Preprint Open access
RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning
Reinforcement learning is crucial for improving large language models' reasoning and generalization. It relies on massive rollouts whose lengths become increasingly long-tailed as context windows grow. In on-policy training, these long-tail rollouts can result in GPU bubbles, reducing system utilization and limiting RL …