Preprint Open access
Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting
Supervised fine-tuning of discrete diffusion language models masks some response tokens and trains the model to recover their original values from the visible context. The masking pattern therefore determines both the context available to the model and the tokens it learns to predict. Uniform random masking does not ex …