Comparative Analysis of AI-Based Agents and Real Individuals in Economic Decision-Making

In recent years, significant research has focused on the potential of AI, particularly large language models (LLMs), to simulate human behavior (Park et al., 2023). Several studies have analyzed differences in decision-making between LLMs and participants in economic experiments. For example, in the article by Horton (2023), the author replicates several classic experiments with LLMs, suggesting that these models could serve as proxies for typical economic agents. However, despite these studies, it remains unclear how closely AI behavior aligns with that of real people.

This project examines the capability of various language models to replicate the behavior of real economic agents in the context of classical behavioral experiments from game theory. Examples of such experiments include investigating loss aversion, trust (Trust Game), and social cooperation (Public Goods Game). These experiments illustrate how individuals deviate from classical rationality when influenced by losses, peer pressure, or limited trust. Experiments on confirmation bias and conformity further demonstrate how beliefs and social influences can distort judgments and decisions.

This project also holds applied significance, as it aims to deepen our understanding of the extent to which contemporary AI models can mimic human cognitive and behavioral traits in complex economic situations. Understanding the differences and similarities between human and AI-agent behavior could have profound implications for the development of economic models and the forecasting of decisions under uncertainty. Moreover, if AI behavior proves sufficiently comparable to human behavior, it could pave the way for AI use in economic experiments, reducing empirical research costs and accelerating analysis. In the long term, the results of this project could influence the application of AI in economic policy formation, management optimization, and modeling human-AI interactions across various economic domains.

The main goal of the research is to determine the extent to which AI-based agents can mimic and predict human behavior under conditions of uncertainty, risk, and limited information.

The research tasks can be outlined as follows:

Within this project, the focus will be on comparing the results of using Large Language Models (LLMs) with the outcomes of classical games in the field of social interaction, as well as with the results of experiments that reveal the specific behaviors of economic agents and are backed by a broad empirical base across various contexts. This will enable the creation of a detailed description of the context and agent characteristics in the LLM prompt and will allow for a comparison of the results with those from real-world experiments. Comparing LLM results with those from experiments highlighting the specific behaviors of economic agents will demonstrate the potential of using LLMs to study the influence of psychological and social factors on decision-making. Thus, the first key objective of the project is to model interactions between economic agents and decision-making strategies using LLMs.

The second task focuses on testing the ability of modern LLMs to propose more effective solutions compared to individuals and groups of individuals. To achieve this, a series of comparative analyses will be conducted, where LLM results will be compared against decisions made by real economic agents in similar situations. This will allow for an objective assessment of the quality of the solutions proposed by the LLMs.

The third task involves analyzing the perception of LLMs and the solutions they propose by economic individuals. A crucial aspect of this analysis will be investigating the extent to which users trust the results obtained from LLMs and how the characteristics of the generated response influence its perception. This direction involves conducting surveys to understand attitudes towards texts generated by LLMs and to identify potential biases.

The scale of the proposed tasks is justified by the rapid development of new, more advanced, and specialized LLMs, as well as the breadth of their potential applications. The project encompasses a wide range of tasks, from theoretical modeling to the practical analysis of AI technology perception, underscoring its comprehensive nature and relevance in the context of the rapid advancement of artificial intelligence and its integration into applied solutions.

This project aims to achieve world-class results by integrating modern and rapidly developing artificial intelligence (AI) technologies into solving classical economic problems.

Results Determining the Theoretical Contribution of the Research:

• Justification for the applicability of foundational language models for modeling the behavior of economic agents.

• Systematization of the key individual characteristics required for creating AI agents.

Results Contributing to Methodological Development:

• Development of a framework for generating synthetic samples that utilizes the capabilities of Large Language Models (LLMs) to simulate behavior comparable to that of real economic agents.

• Development of an experimental design involving human participants and the selection of interaction models.

Empirical Results:

• A comparison of the behavior of LLM-based agents and real economic agents.

• A comparison of the degree to which AI agents based on various LLMs align with the behavior of real economic agents.

• An evaluation of the effectiveness of solutions proposed by LLMs compared to decisions made by real agents and typical benchmarks.

• Evaluation of the perception of solutions proposed by LLMs among economic agents.

Based on the findings, the project aims to develop recommendations for using Large Language Models (LLMs) in economic forecasting, risk management, and decision-making tasks. This is expected to enhance the efficiency of companies, organizations, and government bodies. The reasoned use of LLMs for identifying individual preferences and predicting behavior will accelerate the analytics process and reduce associated costs. Furthermore, the research will also identify future development prospects for LLMs to refine their application in creating advanced AI agents.

• On Decenber 2, 2025,  a research group seminar was held, during which Daria Semenova presented an overview of current studies on the use of artificial intelligence in solving the bride problem (for more details follow the link).

• On October 30-31, 2025, within the framework of the XII International Conference on Applied Economics iCare, project participants presented research study during the poster session (for more details follow the link):

  • Paklina S.N., Parshakov P.A., Shenkman E.A., Matkin N.A. “LLMs vs. Humans: Gender Differences in the Dictator Game”

• On October 28, 2025, G.A. Vasilyev presented a report on the preprint "Humans Expect Rationality and Cooperation from LLM Opponents in Strategic Games" by Barak and Costa-Gomes (2025) at a research group workshop (for more details, follow the link).

• On October 22, 2025, N.A. Matkin presented the study "Comparative Analysis of AI-Based Agent Behavior and Real Individuals in Economic Decision-Making Processes" (co-authored with P.A. Parshakov and E.A. Shenkman) at the VII October International Scientific Conference on Theoretical Economics Problems (for more details, follow the link).

• On September 27, 2025, E.A. Shenkman presented the research "Analysis of LLM-Based Agent Behavior in the Dictator Game" (co-authored with P.A. Parshakov and N.A. Matkin) at the V International Pacific School-Conference on Experimental Economics (for more details, follow the link).

• On August 12, 2025, D.V. Semenova presented a report on the work "The Art of Knowing When to Stop: Analysis of Optimal Stopping in People and Machines" by Zhang and Zhao (2024) at a research group workshop (for more details, follow the link).

• On July 11, 2025, N.A. Matkin presented a report on the topic "Behavior Analysis of LLM-Based AI Agents in a Trust Game" at a laboratory seminar (for more details, follow the link).\

• On July 8, 2025, a research group seminar was held to discuss the following grant-related studies (for more details, follow the link): 
  • N.A. Matkin led a discussion on the article "Do Large Language Models Show Decision Heuristics Similar to Humans?" (Suri et al., 2024). ​​
  • M.A. Usanin presented an analysis of the article "Playing Repeated Games with Large Language Models" (Akata et al., 2025), recently published in Nature Human Behaviour.

• On June 24, 2025, the first working seminar of the research group was held, where the team analyzed current global research on the grant topic (for more details, follow the link):

  • D.A. Dagaev discussed the preprint "Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?" (Horton, 2023).

  • E.A. Shenkman presented an analysis of the article "Can Machines Think Like Humans? A Behavioral Evaluation of LLM-Agents in Dictator Games" (Ma, 2024).

  • P.A. Parshakov presented an overview of the text "Can Large Language Model Agents Simulate Human Trust Behavior?" (Xie et al., 2025).