الملخص
Large language models often lose track of persistent instructions and relevant information as multi-turn conversations grow. We study this cumulative contextual decay through three related failure modes: attention pollution, dilution, and drift. We propose REA (Role-aware Heuristic Episodic Attention), a context-management framework that assigns different persistence and representation policies to instructions and episodic interactions. Instructional Memory retains identified global constraints in a dedicated prefix. Episodic Memory preserves user inputs and compresses model replies, while heuristic retrieval selects raw text, compressed representations, or omission for each historical turn. On Long-MT-Bench+, REA improves the judge score from 6.32 to 7.36 on a 10-point scale, a 16.5% relative gain over the Vanilla baseline, and reduces average latency by 2.91$\times$. Additional evaluations show aggregate gains on three backbones spanning 1.7B-7B parameters and on Chinese and English role-playing tasks. These results support role-aware context management as a practical approach to maintaining conversational continuity and instruction adherence.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Hong, W., Zhang, Z., Chen, Y., Zhang, L., Liu, B., Qiao, L., Tian, Z., & Li, D. (2026). Role-aware Heuristic Episodic Attention for Conversational LLMs. https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms
MLA 9
Hong, Wanyang, et al. "Role-aware Heuristic Episodic Attention for Conversational LLMs." https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms.
شيكاغو (المؤلف–التاريخ)
Hong, Wanyang, Zhaoning Zhang, Yi Chen, Libo Zhang, Baihui Liu, Linbo Qiao, Zhiliang Tian, and Dongsheng Li. 2026. "Role-aware Heuristic Episodic Attention for Conversational LLMs." https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms.
هارفارد
Hong, W., Zhang, Z., Chen, Y., Zhang, L., Liu, B., Qiao, L., Tian, Z. and Li, D. (2026) 'Role-aware Heuristic Episodic Attention for Conversational LLMs', Available at: https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms.
فانكوفر
Hong W, Zhang Z, Chen Y, Zhang L, Liu B, Qiao L, et al. Role-aware Heuristic Episodic Attention for Conversational LLMs. https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms
IEEE
W. Hong, Z. Zhang, Y. Chen, L. Zhang, B. Liu, L. Qiao, Z. Tian, and D. Li, "Role-aware Heuristic Episodic Attention for Conversational LLMs," https://omanscience.com/ar/articles/role-aware-heuristic-episodic-attention-for-conversational-llms.