الملخص
While action-conditioned video prediction provides an intuitive world model for robotics, purely data-driven predictors often suffer from compounding errors and physically implausible hallucinations in long-horizon rollouts, severely undermining downstream action planning. We propose VPTwin, a Real-Sim-Real video prediction framework that anchors real-world future prediction using real-synchronized simulation twins. For a target manipulation task, a VLM reconstructs an executable digital twin from a real demonstration episode. To accommodate the ill-posed estimation of unobserved physical properties, Isaac Sim simulates multiple forward dynamic rollouts across randomized physical configurations under candidate action trajectories. Using these rollouts as in-context references, VPTwin harmonizes both domains, using simulation dynamics to enforce physical plausibility while capturing unmodeled contact interactions from real video. Furthermore, we establish a predictive planning loop using VPTwin to visually verify VLM-proposed actions and guide reliable real-world execution. Evaluations show substantial reductions in physical hallucinations during video prediction and marked improvements in manipulation planning performance.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Xiao, Z., Pan, M., He, N., Zhou, D., & Wang, Y. (2026). VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning. https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning
MLA 9
Xiao, Zhenghao, et al. "VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning." https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning.
شيكاغو (المؤلف–التاريخ)
Xiao, Zhenghao, Minting Pan, Nantian He, Dongzhan Zhou, and Yunbo Wang. 2026. "VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning." https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning.
هارفارد
Xiao, Z., Pan, M., He, N., Zhou, D. and Wang, Y. (2026) 'VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning', Available at: https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning.
فانكوفر
Xiao Z, Pan M, He N, Zhou D, Wang Y. VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning. https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning
IEEE
Z. Xiao, M. Pan, N. He, D. Zhou, and Y. Wang, "VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning," https://omanscience.com/ar/articles/vptwin-real-sim-real-video-prediction-for-robotic-manipulation-planning.