Abstract
Reinforcement learning (RL) has become an effective post-training paradigm for long-horizon large language model (LLM) agents. However, we find that the resulting policies can be sensitive to various policy perturbations, such as hidden-state noise, pruning, and quantization. In this work, we study how to improve perturbation robustness during policy optimization. We first introduce the notion of a perturbation robust policy and analyze conditions under which perturbed policy updates preserve stable monotonic improvement. Based on this analysis, we introduce Stable Perturbation-Robust Policy Optimization (SPrPO), which applies adaptive and sensitivity-aware perturbations during RL training. We evaluate SPrPO on ALFWorld and WebShop and conduct systematic experiments across multiple perturbation types and scales, showing improved perturbation robustness while maintaining stable policy optimization.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wang, P., Li, Y., Ren, Y., Yang, H., & Chen, J. (2026). Learning Perturbation Robust Policies for LLM Agents with Stable Optimization. https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization
MLA 9
Wang, Pengxin, et al. "Learning Perturbation Robust Policies for LLM Agents with Stable Optimization." https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization.
Chicago (author–date)
Wang, Pengxin, Yuanzhe Li, Yuxin Ren, Huanrui Yang, and Jingdi Chen. 2026. "Learning Perturbation Robust Policies for LLM Agents with Stable Optimization." https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization.
Harvard
Wang, P., Li, Y., Ren, Y., Yang, H. and Chen, J. (2026) 'Learning Perturbation Robust Policies for LLM Agents with Stable Optimization', Available at: https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization.
Vancouver
Wang P, Li Y, Ren Y, Yang H, Chen J. Learning Perturbation Robust Policies for LLM Agents with Stable Optimization. https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization
IEEE
P. Wang, Y. Li, Y. Ren, H. Yang, and J. Chen, "Learning Perturbation Robust Policies for LLM Agents with Stable Optimization," https://omanscience.com/en/articles/learning-perturbation-robust-policies-for-llm-agents-with-stable-optimization.