الملخص
Generative models, such as diffusion and flow-based models, have shown strong promise for robot policy learning by capturing complex and multimodal action distributions from demonstrations. However, policies trained solely with imitation learning often suffer from imperfect demonstrations and distributional shifts, while further improvement typically requires additional expert data. Reinforcement learning offers a natural solution through environment interaction, but effectively finetuning generative robot policies remains challenging due to the intractability of likelihood estimation. In this work, we propose RoboFFT, a forward-process reinforcement learning framework for finetuning generative robot policies, which applies forward noising to sampled actions and uses the weighted score / flow matching loss to construct a surrogate policy ratio for PPO-style updates. We evaluate RoboFFT with popular generative robot policies on representative simulation benchmarks, including long-horizon planning and sparse reward settings. Extensive experiments and analysis demonstrate that RoboFFT consistently improves performance while achieving better stability and training efficiency. We further integrate RoboFFT into a real world RL framework and demonstrate its effectiveness in real world tasks. Project website: https://student-of-holmes.github.io/RoboFFT/.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Li, Y., Hu, S., Wang, Y., Pu, Y., Zhang, H., Chen, Y., & Yang, Y. (2026). RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process. https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process
MLA 9
Li, Yu, et al. "RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process." https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process.
شيكاغو (المؤلف–التاريخ)
Li, Yu, Shenghe Hu, Yuhan Wang, Yaoxiang Pu, Haotong Zhang, Yuanpei Chen, and Yaodong Yang. 2026. "RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process." https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process.
هارفارد
Li, Y., Hu, S., Wang, Y., Pu, Y., Zhang, H., Chen, Y. and Yang, Y. (2026) 'RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process', Available at: https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process.
فانكوفر
Li Y, Hu S, Wang Y, Pu Y, Zhang H, Chen Y, et al. RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process. https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process
IEEE
Y. Li, S. Hu, Y. Wang, Y. Pu, H. Zhang, Y. Chen, and Y. Yang, "RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process," https://omanscience.com/ar/articles/robofft-finetuning-generative-robot-policy-via-online-reinforcement-learning-with-forward-process.