Authors

Samyak Jha

Publications 2

Preprint Open access

Masking Frequent Tokens Sharpens Direct Preference Optimization

Harshvardhan Saini, Samyak Jha, Yiming Tang et al. · 2026

Direct Preference Optimization (DPO) aligns language models by optimizing over sequence-level sums of token-wise implicit reward differences. However, we identify a pervasive pathology in this formulation: a disproportionately small subset of high-frequency token types dominates cumulative sequence scores while appeari …

Preprint Open access

Understanding and Exploiting Anisotropy in Post-Training

Samyak Jha, Harshvardhan Saini, Yizhen Liao et al. · 2026

LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective. Frequency-weighted likelihood training leaves a well-known signature: \emph{anisotropy}, in which a few residual channels carry disproportionately large a …

Co-authors