Preprint Open access
Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
On-policy distillation (OPD) has emerged as an effective post-training paradigm for language models, with recent efforts extending it to block diffusion language models (BDLMs). However, existing studies focus almost exclusively on small block sizes, leaving distillation into student models with larger blocks underexpl …