Abstract
Affective computing has progressed from categorical emotion recognition to open-ended affective analysis with large multimodal models. Yet affective science describes emotion as an unfolding process shaped by appraisal, regulation, and social interpretation, which remains underexplored computationally. We propose TRACE, a cognition-oriented framework that formalizes an affective episode through three interrelated stages: Condition, Affect, and Effect, integrating observable cues with cognitive factors such as internal stance and regulation of emotional display. Based on this formulation, TRACE-Bench evaluates multimodal models in real-world social scenes through five tasks spanning grounded affect recognition, regulation decoding, cause reasoning, effect reasoning, and full-chain reconstruction, with 3,746 structured question-answer pairs over 646 videos. A matched human-model comparison reveals a substantial performance gap, while affect-specialized models also generally lag behind general-purpose MLLMs. Model outputs show recurring failures, including treating displayed behavior as genuine feeling and fabricating unsupported events during long-chain generation. We further propose TRACER, a cognition-grounded structured reasoning method that couples each inference with explicit premises from factual observations, cognitive appraisals, and established upstream conclusions, forming a traceable graph of intermediate and target conclusions. TRACER outperforms all evaluated model baselines on each of the five tasks. Project page: https://cogaffc.github.io/TRACE
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Li, H., Zhang, J., Li, B., Lee, M. L., Hsu, W., Wang, Z., Fei, H., & Zhang, M. (2026). Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes. https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes
MLA 9
Li, Hao, et al. "Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes." https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes.
Chicago (author–date)
Li, Hao, Jinye Zhang, Bobo Li, Mong-Li Lee, Wynne Hsu, Zheng Wang, Hao Fei, and Min Zhang. 2026. "Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes." https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes.
Harvard
Li, H., Zhang, J., Li, B., Lee, M. L., Hsu, W., Wang, Z., Fei, H. and Zhang, M. (2026) 'Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes', Available at: https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes.
Vancouver
Li H, Zhang J, Li B, Lee ML, Hsu W, Wang Z, et al. Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes. https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes
IEEE
H. Li, J. Zhang, B. Li, M. L. Lee, W. Hsu, Z. Wang, H. Fei, and M. Zhang, "Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes," https://omanscience.com/en/articles/cognition-oriented-emotion-tracing-from-causes-to-consequences-in-real-world-social-scenes.