Preprint Open access
COPC: Coupled Off-Policy Correction for Asynchronous LLM Reinforcement Learning
Asynchronous RL accelerates large language model post-training by decoupling rollout generation from optimization, but trains on stale trajectories. Existing methods primarily correct token-level policy mismatch through importance-ratio control in the actor objective. We show that this \emph{policy-side correction} alo …