Abstract
Sustained emotional support requires generative agents to connect momentary emotion inference and regulation with continuity across encounters. We present PAIR (Perceptual Affective Inference and Regulation), a real-time multimodal agent that reconstructs how an event is appraised into an emotional state. Appraisal scaffolds produce a valence-arousal-dominance estimate and select regulation guidance, delivered through conversation with coordinated speech, color, and avatar cues. Rolling memory carries context across sessions, and the scaffold re-runs after guidance. In a 14-day deployment with 19 participants, 1,093 sessions paired initial and post-guidance estimates with unanchored self-reports. Initial valence reached MAE 1.20 on the 9-point SAM scale (r=.68), dominance reached MAE 1.30, and arousal showed weak agreement even after coarsening. Self-reported emotional change varied with initial state, with the largest valence increases in sessions that began at negative valence. Perceived understanding was associated with greater valence increase and showed little correspondence with numerical prediction error. Over two weeks, helpfulness increased while input shortened; interviews traced personalization and companionship to relevant recall, context updates, and familiar dialogue. These findings connect inference accuracy to conversational and temporal patterns of support through per-event, first-person evaluation.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Quan, K., Ding, Z., Yong, J., Zhang, Q., Wang, D., & Chin, J. (2026). PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent. https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent
MLA 9
Quan, Kexin, et al. "PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent." https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent.
Chicago (author–date)
Quan, Kexin, Zijian Ding, Jiaye Yong, Qinshi Zhang, Dong Wang, and Jessie Chin. 2026. "PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent." https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent.
Harvard
Quan, K., Ding, Z., Yong, J., Zhang, Q., Wang, D. and Chin, J. (2026) 'PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent', Available at: https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent.
Vancouver
Quan K, Ding Z, Yong J, Zhang Q, Wang D, Chin J. PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent. https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent
IEEE
K. Quan, Z. Ding, J. Yong, Q. Zhang, D. Wang, and J. Chin, "PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent," https://omanscience.com/en/articles/pair-perceptual-affective-inference-and-regulation-in-a-real-time-multimodal-conversational-agent.