الملخص
Agentic reinforcement learning (RL) has emerged as a powerful approach for training large language model agents on multi-step tasks, yet reliance on terminal outcome rewards creates two credit-assignment problems, particularly in long-horizon tasks. First, same-outcome rollout groups provide no learning signal from terminal rewards. Second, terminal rewards provide only trajectory-wide feedback, making it difficult to identify which decisions caused a failure. Recent work supplements terminal rewards with finer-grained information from trajectory analysis, such as natural-language reflections on intermediate decisions and errors. However, natural-language diagnoses are difficult to use directly for credit assignment: their error claims may be unreliable, and they do not quantify how much each error should affect learning. We propose Self-Diagnosis-guided Terminal Credit Redistribution (FAULT), which turns diagnosed errors into explicit step-level credit anchored by terminal outcomes. FAULT checks diagnostic evidence and learns relative error costs from task outcomes. During training, the policy and self-diagnoser co-evolve, while error costs are updated online from recent outcomes. On ALFWorld, FAULT recovers learning signals from same-outcome groups, reaching 95% signal coverage versus 41% for GRPO and 72% for GiGPO, while better localizing credit to specific error steps. Across two model scales, FAULT delivers strong. improvements on the long-horizon ALFWorld and WebShop tasks while remaining competitive on short-horizon Search-based QA.
الكلمات المفتاحية
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Zhu, Y., Liu, Q., Qiao, W., Ren, X., Xu, W., Li, W., Wang, W., Chen, R., Luan, X., Luo, Y., Huang, H., Zheng, X., & Shimodaira, H. (2026). My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning. https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning
MLA 9
Zhu, Yihua, et al. "My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning." https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning.
شيكاغو (المؤلف–التاريخ)
Zhu, Yihua, Qianying Liu, Weixu Qiao, Xuan Ren, Weiwei Xu, Wenbo Li, Wei Wang, Ruijia Chen, Xinmiao Luan, Yin Luo, Hao Huang, Xiang Zheng, and Hidetoshi Shimodaira. 2026. "My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning." https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning.
هارفارد
Zhu, Y., Liu, Q., Qiao, W., Ren, X., Xu, W., Li, W., Wang, W., Chen, R., Luan, X., Luo, Y., Huang, H., Zheng, X. and Shimodaira, H. (2026) 'My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning', Available at: https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning.
فانكوفر
Zhu Y, Liu Q, Qiao W, Ren X, Xu W, Li W, et al. My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning. https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning
IEEE
Y. Zhu, Q. Liu, W. Qiao, X. Ren, W. Xu, W. Li, W. Wang, R. Chen, X. Luan, Y. Luo, H. Huang, X. Zheng, and H. Shimodaira, "My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning," https://omanscience.com/ar/articles/my-fault-self-diagnosis-as-credit-assignment-in-self-evolving-agentic-reinforcement-learning.