نسخة أولية وصول مفتوح
SAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation
On-policy distillation (OPD) reduces train-test state mismatch by training a student on its own generated trajectories, but weak students may visit teacher-misaligned prefixes where supervision is less representative. We introduce SAKI (Supervision Allocation with KL-constrained Interpolation), which combines a KL-cons …