Authors

Yan Zheng

Publications 2

Preprint Open access

LastOPD: Taming Collapse in Latent On-Policy Distillation

Jie Yang, Zhengyu Fang, Zelin Xu et al. · 2026

On-policy distillation (OPD) corrects a student on the responses it writes, but its signal is the teacher's next-token distribution: it tells the student what the teacher says but misses how it thinks. Latent supervision promises the missing part by aligning the student's latent states to the teacher's. Recent methods …

Co-authors