Facial features, emotions, and the trust game: seminar discusses visual bias in language models

On July 2, the International Laboratory of Intangible-driven Economy hosted a workshop within the framework of RSF project No. 25-18-00539, "Comparative Analysis of AI-Based Agents and Real Individuals in Economic Decision-Making." The seminar focused on research by Junior Research Fellow Egor Ivanov and Senior Research Fellow Marina Zavertiaeva titled "Beyond Demographic Labels: Facial Cues and LLM Bias in Trust-Game Decisions".

The authors investigated whether facial images influence the decisions of multimodal language models in the trust game. The models were presented with photographs of potential partners and asked to determine how many tokens to send them. The analysis accounted for facial structure, perceived demographic attributes, attractiveness, trustworthiness, and expressed emotions.

Facial expression was found to have the strongest impact. The model sent fewer tokens to individuals with angry expressions and more to those with an open smile. At the same time, attractiveness, perceived trustworthiness, and most objective facial characteristics did not show a robust correlation with the model's decisions.

During the discussion, participants outlined potential directions for future work, including refining the research design and testing additional specifications. It was also suggested to identify the most promising research focus and strengthen its connection to existing literature.