Abstract
Stochastic recursive momentum (STORM) achieves fast convergence for nonconvex optimization via the variance reduction effect, but existing analyses rely on the strong average smoothness assumption. In this paper, we study the convergence of STORM for different objectives without average smoothness. We first revisit the results under average smoothness, obtaining the $O(T^{-1/3})$ bound for nonconvex objectives and the $O(σ^2/(μT))$ bound for last-iterate output under the $μ$-Polyak--Łojasiewicz~(PL) condition. Without average smoothness, we design an auxiliary sequence and compare the STORM update with it in the analysis. With the help of this sequence, we prove that STORM still attains an $O(T^{-1/4})$ rate for nonconvex objectives, which is optimal under standard smoothness. For convex and $λ$-strongly convex objectives, we further prove averaged and last-iterate bounds with optimal rates of $O(σR/\sqrt T)$ and $O(σ^2/(λT))$, respectively. All the obtained results use the same STORM recursion with different hyperparameter choices.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Jiang, W., Wang, Y., Yang, W., Yan, R., Zhang, L., & Li, Z. (2026). Convergence Analysis of STORM Under Different Geometries. https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries
MLA 9
Jiang, Wei, et al. "Convergence Analysis of STORM Under Different Geometries." https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries.
Chicago (author–date)
Jiang, Wei, Yibo Wang, Wenhao Yang, Rui Yan, Lijun Zhang, and Zechao Li. 2026. "Convergence Analysis of STORM Under Different Geometries." https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries.
Harvard
Jiang, W., Wang, Y., Yang, W., Yan, R., Zhang, L. and Li, Z. (2026) 'Convergence Analysis of STORM Under Different Geometries', Available at: https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries.
Vancouver
Jiang W, Wang Y, Yang W, Yan R, Zhang L, Li Z. Convergence Analysis of STORM Under Different Geometries. https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries
IEEE
W. Jiang, Y. Wang, W. Yang, R. Yan, L. Zhang, and Z. Li, "Convergence Analysis of STORM Under Different Geometries," https://omanscience.com/en/articles/convergence-analysis-of-storm-under-different-geometries.