Abstract
Local Gaussian models of constant-step learning predict output variability and expected losses, but weak convergence alone does not justify these moment predictions. We establish moment-accurate Gaussian mixtures by matching stationary energy with local Ornstein--Uhlenbeck limits, ruling out quadratic tail mass invisible to weak convergence. For step size $a$, the second-order Wasserstein error is $o(\sqrt a)$, uniformly over invariant laws, using each law's actual root weights. The assumptions combine confinement, descent, finitely many hyperbolic equilibria and root continuity with finite-variance innovations. The result yields observable covariances, expected objective gaps and first-order mean shifts, while allowing singular covariances, compatible saddles and weights without a limit. For additive noise given by a fixed invertible transform of independent standardized Student $t_3$ coordinates, symmetry gives an order-sharp $\sqrt a$ smooth-test bound. Numerical transport calculations demonstrate the value of root-specific covariances; controlled SGD studies assess observable predictions across step sizes, batch sizes and model geometries.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Li, X., & Wu, W. B. (2026). Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation. https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation
MLA 9
Li, Xiaoli, and Wei Biao Wu. "Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation." https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation.
Chicago (author–date)
Li, Xiaoli, and Wei Biao Wu. 2026. "Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation." https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation.
Harvard
Li, X. and Wu, W. B. (2026) 'Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation', Available at: https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation.
Vancouver
Li X, Wu WB. Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation. https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation
IEEE
X. Li, and W. B. Wu, "Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation," https://omanscience.com/en/articles/moment-accurate-gaussian-mixtures-for-constant-step-stochastic-approximation.