الملخص
Human-in-the-loop reinforcement learning (HIL-RL) offers a promising route to efficient training of robotic manipulation policies by combining autonomous learning with human demonstrations and online corrections. However, insufficient use of successful human experience in value learning prolongs costly real-world training, while persistent imitation penalties can limit value-driven policy improvement. To address these limitations, we propose ReF-HIL, an efficient HIL-RL framework that uses human guidance to accelerate the learning process. Human-Reference-Guided Value Shaping learns an independent value reference from successful human experience to guide online value learning, while incorporating local corrective feedback. A Human Action Fence defines a learned human-action neighborhood, allowing value-driven optimization for better performance without imitation penalties inside while constraining policy and value updates outside. Experiments on five diverse and challenging real-world manipulation tasks demonstrate improved overall learning efficiency and higher success rates compared with the evaluated baselines. Specifically, ReF-HIL reaches 90% autonomous success in only 18-63 minutes of active training and achieves final success rates of 91.7-100%. These results highlight the potential of human-guided reinforcement learning to acquire reliable manipulation skills efficiently in the real world. Project website: https://anonymous.4open.science/w/ReF-HIL-7762/
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Luo, S., Wang, S., Xia, S., Zhang, T., Liang, Z., Shen, G., Wang, B., & Wu, D. (2026). ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning. https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning
MLA 9
Luo, Shaoyin, et al. "ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning." https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning.
شيكاغو (المؤلف–التاريخ)
Luo, Shaoyin, Song Wang, Shibo Xia, Tianle Zhang, Zhaowei Liang, Guanghui Shen, Bin Wang, and Dan Wu. 2026. "ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning." https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning.
هارفارد
Luo, S., Wang, S., Xia, S., Zhang, T., Liang, Z., Shen, G., Wang, B. and Wu, D. (2026) 'ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning', Available at: https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning.
فانكوفر
Luo S, Wang S, Xia S, Zhang T, Liang Z, Shen G, et al. ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning. https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning
IEEE
S. Luo, S. Wang, S. Xia, T. Zhang, Z. Liang, G. Shen, B. Wang, and D. Wu, "ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning," https://omanscience.com/ar/articles/ref-hil-shaping-the-critic-around-human-action-neighborhoods-for-efficient-human-in-the-loop-reinforcement-learning.