الملخص
Adapting to changes in robot dynamics requires learning from new data without discarding experience that may still be useful. In continual model-based reinforcement learning (RL), replay collected before a dynamics change can slow adaptation, while removing it unnecessarily reduces available training data and can be especially costly if earlier dynamics return. We study when recent transitions are preferable to the full replay history. Two quantities characterize this trade-off: change magnitude and age-staleness area under the curve (AUC), measuring how well transition age separates stale from fresh data. Forgetting stale data helps after large permanent shifts but hurts when dynamics recur and older data becomes useful again. Choosing a replay strategy therefore depends on predicting when older data will help or hurt. We test these effects across two locomotion morphologies, two model-based RL algorithms, and Real-World RL benchmark perturbations. Because ground-truth staleness labels are unavailable on deployed robots, we evaluate whether an estimator built from interaction data can still provide the quantities needed to choose a replay strategy after permanent changes. Our results show that replay retention depends on change magnitude and on how the dynamics evolve.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Yang, E., Thompson, S., & Konidaris, G. D. (2026). Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning. https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning
MLA 9
Yang, Everest, et al. "Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning." https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning.
شيكاغو (المؤلف–التاريخ)
Yang, Everest, Skye Thompson, and George D. Konidaris. 2026. "Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning." https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning.
هارفارد
Yang, E., Thompson, S. and Konidaris, G. D. (2026) 'Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning', Available at: https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning.
فانكوفر
Yang E, Thompson S, Konidaris GD. Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning. https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning
IEEE
E. Yang, S. Thompson, and G. D. Konidaris, "Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning," https://omanscience.com/ar/articles/characterizing-replay-retention-under-dynamics-shift-in-model-based-reinforcement-learning.