الباحثون

Samyak Jha

المنشورات 2

نسخة أولية وصول مفتوح

Masking Frequent Tokens Sharpens Direct Preference Optimization

Harshvardhan Saini, Samyak Jha, Yiming Tang وآخرون · 2026

Direct Preference Optimization (DPO) aligns language models by optimizing over sequence-level sums of token-wise implicit reward differences. However, we identify a pervasive pathology in this formulation: a disproportionately small subset of high-frequency token types dominates cumulative sequence scores while appeari …

نسخة أولية وصول مفتوح

Understanding and Exploiting Anisotropy in Post-Training

Samyak Jha, Harshvardhan Saini, Yizhen Liao وآخرون · 2026

LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective. Frequency-weighted likelihood training leaves a well-known signature: \emph{anisotropy}, in which a few residual channels carry disproportionately large a …

المؤلفون المشاركون