Abstract
On-policy distillation (OPD) trains compact language agents with teacher feedback on student-generated trajectories. In multi-turn tasks, compounding errors can move students beyond the teacher's effective supervision. We introduce Graph-Conditioned On-Policy Agent Distillation (GC-OPD), which enriches an off-the-shelf teacher's scoring context with execution evidence. A graph indexes repeated teacher executions by shared states while preserving complete successful and failed histories. After each student episode, GC-OPD retrieves current-state references or historical alternatives and combines them with student hindsight to score the original thought-action tokens. Using the same original teachers, GC-OPD improves mean success over vanilla OPD from 24.70% to 48.78% on ScienceWorld (4B student), from 53.36% to 85.26% on ALFWorld Unseen, and from 29.10% to 37.65% on WebShop. At matched student sizes, it also achieves higher mean success than every evaluated OPD baseline using GRPO-trained teachers on ScienceWorld and ALFWorld; the strongest such ScienceWorld 4B baseline reaches 46.66%. GC-OPD requires no task-specific teacher optimization.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Yi, X., Luo, W., Huang, Y., Zhan, J., Qin, A., Zhao, P., & Xiao, X. (2026). Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers. https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers
MLA 9
Yi, Xiaohan, et al. "Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers." https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers.
Chicago (author–date)
Yi, Xiaohan, Wen Luo, Yani Huang, Junfeng Zhan, Asher Qin, Peilin Zhao, and Xi Xiao. 2026. "Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers." https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers.
Harvard
Yi, X., Luo, W., Huang, Y., Zhan, J., Qin, A., Zhao, P. and Xiao, X. (2026) 'Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers', Available at: https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers.
Vancouver
Yi X, Luo W, Huang Y, Zhan J, Qin A, Zhao P, et al. Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers. https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers
IEEE
X. Yi, W. Luo, Y. Huang, J. Zhan, A. Qin, P. Zhao, and X. Xiao, "Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers," https://omanscience.com/en/articles/graph-conditioned-on-policy-agent-distillation-from-off-the-shelf-teachers.