Abstract
Multi-teacher on-policy distillation allows a student to learn from complementary specialists on its own trajectories. Domain-routed approaches, however, select one teacher per example and keep it fixed throughout the response. This design both depends on domain labels that mixed training corpora often lack and cannot adapt teacher selection when the expertise required changes within a trajectory. We observe that each specialist deviates more from a shared reference on in-domain prompts than on out-of-domain prompts, on average. Based on this observation, we propose \textbf{TrustMOPD}, which replaces example-level teacher selection with label-free, token-level supervision allocation. At each student-generated prefix, TrustMOPD measures this displacement in next-token preferences, calibrates its magnitude across teachers, and uses the resulting scores as proxies for local reliability to weight teacher-specific distillation losses. Evaluated across mathematics, code, and instruction following, TrustMOPD closes 91.5\% and 98.0\% of the overall-score gap between the initial student and oracle-routed teachers when trained on \textsc{SingleCap} and \textsc{MultiCap}, respectively, compared with 54.4\% and 54.5\% for the strongest label-free baseline in each setting. On \textsc{SingleCap}, it approaches label-based MOPD without using domain labels.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Sun, J., Zheng, M., Song, M., Liu, Z., Li, G., Jiang, H., Cheng, Y., Feng, B., Cai, Y., Fang, J., & Wang, X. (2026). Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation. https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation
MLA 9
Sun, Jie, et al. "Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation." https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation.
Chicago (author–date)
Sun, Jie, Mao Zheng, Mingyang Song, Zeyuan Liu, Gengsheng Li, Houcheng Jiang, Yilin Cheng, Bichuan Feng, Yuchen Cai, Junfeng Fang, and Xiang Wang. 2026. "Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation." https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation.
Harvard
Sun, J., Zheng, M., Song, M., Liu, Z., Li, G., Jiang, H., Cheng, Y., Feng, B., Cai, Y., Fang, J. and Wang, X. (2026) 'Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation', Available at: https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation.
Vancouver
Sun J, Zheng M, Song M, Liu Z, Li G, Jiang H, et al. Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation. https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation
IEEE
J. Sun, M. Zheng, M. Song, Z. Liu, G. Li, H. Jiang, Y. Cheng, B. Feng, Y. Cai, J. Fang, and X. Wang, "Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation," https://omanscience.com/en/articles/distill-what-you-trust-reliability-aware-multi-teacher-on-policy-distillation.