Preprint Open access
Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features
Supervised fine-tuning (SFT) learns most aggressively from tokens that the model deems least likely. This helps acquire new behaviors, but also amplifies noisy or conflicting supervision and can overwrite useful pretrained knowledge. Through a unified policy-loss view, we revisit existing token-reweighting methods and …