Abstract
Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric $α$-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Takanami, K., Takahashi, T., & Kabashima, Y. (2026). Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data. https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data
MLA 9
Takanami, Kaito, et al. "Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data." https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data.
Chicago (author–date)
Takanami, Kaito, Takashi Takahashi, and Yoshiyuki Kabashima. 2026. "Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data." https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data.
Harvard
Takanami, K., Takahashi, T. and Kabashima, Y. (2026) 'Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data', Available at: https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data.
Vancouver
Takanami K, Takahashi T, Kabashima Y. Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data. https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data
IEEE
K. Takanami, T. Takahashi, and Y. Kabashima, "Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data," https://omanscience.com/en/articles/asymptotic-analysis-of-empirical-risk-minimization-on-entry-wise-i-i-d-heavy-tailed-data.