Not Quite Human: An Exploratory Study on the Affective Divergence Between MLLMs and Humans in Infographics

Abstract

Infographics shape affective impressions through visual design and narrative framing, yet how MLLMs align with subjective human emotional responses remains under-examined. We compare MLLM-predicted 12-dimensional affect ratings with human ratings from an infographic affect dataset. Our preliminary results show that, although MLLMs provide stable predictions across repeated runs, they match the primary human affect in only about half of high-consensus cases. Through exploratory qualitative content analysis, we identify recurring affective error patterns, including cognitive-load misestimation, missed engagement loss, and fine-grained affect confusion, while examining their co-occurrence with picture-level visual and semantic characteristics. These findings provide a consensus-aware evaluation of MLLM affect prediction and suggest that MLLMs can capture broad affective direction in some cases, but remain limited in modeling human responses shaped by reading burden, viewer interest, prior expectations, and rhetorical framing.