Preprint Open access
A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards
When post-training large language models on tasks with semi-verifiable rewards, there are many factors (training steps, base model size, training order, data quality, verifier accuracy, etc.) that practitioners must contend with to maximize model performance. Yet, it remains unclear how well verifier agreement predicts …