Abstract
Textual-gradient methods automate prompt optimization through natural-language feedback, but their iterative updates can be unstable. We identify two sources of this instability: noisy gradients produced from already-correct examples and over-specialization to hard cases that degrades performance on simpler inputs. We introduce STEVE, a stabilization framework with two coupled mechanisms. Error-Driven Refinement generates gradients only from incorrectly handled examples, concentrating updates on informative failures. Regularized Verification treats every update as provisional and accepts it only when improvement on hard cases does not cause unacceptable regression on a preservation set. Across ten reasoning benchmarks, three evaluator/optimizer models, and established prompt-optimization baselines, STEVE reduces degradation and produces more robust prompts. Additional evaluations with gpt-5.4-mini/gpt-5.4 on symbolic reasoning, GSM8K-Platinum, and DS-1000 show that these gains persist with newer models and larger test sets. STEVE therefore provides a practical way to improve the stability and effectiveness of textual-gradient prompt optimization.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Xu, Y., Li, Y., Li, X., Gu, Y., Shen, Y., Yu, L., & Wang, H. (2026). STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification. https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification
MLA 9
Xu, Yifan, et al. "STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification." https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification.
Chicago (author–date)
Xu, Yifan, Yixuan Li, Xinzhuo Li, Yixin Gu, Yifan Shen, Lijun Yu, and Haohan Wang. 2026. "STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification." https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification.
Harvard
Xu, Y., Li, Y., Li, X., Gu, Y., Shen, Y., Yu, L. and Wang, H. (2026) 'STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification', Available at: https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification.
Vancouver
Xu Y, Li Y, Li X, Gu Y, Shen Y, Yu L, et al. STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification. https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification
IEEE
Y. Xu, Y. Li, X. Li, Y. Gu, Y. Shen, L. Yu, and H. Wang, "STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification," https://omanscience.com/en/articles/steve-stabilizing-textual-gradient-based-prompt-optimization-via-error-driven-refinement-and-regularized-verification.