Preprint Open access
Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR
Assigning credit to intermediate steps remains a central challenge in training Large Language Models (LLMs) on multi-step reasoning tasks with sparse terminal rewards, and actor-critic methods such as PPO address this by learning value functions to construct token-level advantages. Their effectiveness, however, hinges …