Preprint Open access
OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We study whether tree search improves their coverage within a fixed budget while controlling gradient bias. We introduce On-Policy Parallel Tree Search (OPTS) and Tree Traj …