Preprint Open access
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
On-policy distillation (OPD) has become increasingly popular for transferring teacher capabilities to student models. In this work, we ask a critical research question: Is on-policy sampling always beneficial for distilling arbitrary teacher-student pairs? We show that a simple alternative, Semi-OPD, which distills fro …