Abstract
Reliable action evaluation in contact-rich manipulation requires looking beyond the current observation to future visual and contact consequences. Existing noise-space reinforcement learning efficiently steers a frozen Vision-Language-Action (VLA) policy, but its critics largely ignore these consequences. We present Imagine-RL, which augments noise-space VLA post-training with action-conditioned visual-torque imagination. For each candidate action chunk, a frozen visual-torque latent world model (VTLWM) autoregressively predicts compact future representations without pixel reconstruction. A current image-state-action query attends to observed histories and predicted futures, while previous-window prediction residuals provide token-wise confidence priors that suppress unreliable future tokens. By combining current evidence with predicted consequences, the action critic better evaluates candidate actions and supervises the actor, while the VLA and VTLWM remain frozen. Across four real-robot tasks with 50 evaluation trials per task, Imagine-RL uses only 100 RL trajectories and improves the average success rate by (23.6%) over DSRL and by (60%) over VLA baselines.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Hu, K., Zhai, W., Zhao, B., & Liang, S. (2026). Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning. https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning
MLA 9
Hu, Kejia, et al. "Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning." https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning.
Chicago (author–date)
Hu, Kejia, Wentong Zhai, Bo Zhao, and Shuai Liang. 2026. "Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning." https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning.
Harvard
Hu, K., Zhai, W., Zhao, B. and Liang, S. (2026) 'Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning', Available at: https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning.
Vancouver
Hu K, Zhai W, Zhao B, Liang S. Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning. https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning
IEEE
K. Hu, W. Zhai, B. Zhao, and S. Liang, "Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning," https://omanscience.com/en/articles/imagine-rl-residual-confidence-guided-cross-attention-for-world-model-augmented-vla-reinforcement-learning.