Abstract
This paper revisits the distributed learning problem for training a multinomial logistic regression model with the Federated Averaging ($\texttt{FedAvg}$) algorithm. We concentrate on a scenario with arbitrarily large stepsizes and heterogeneous update rules where the devices may perform a different number of local updates in each round. We show that, with linearly separable data, $\texttt{FedAvg}$ is stable with any stepsizes and the objective values converge to zero at the rate of ${\cal O}(1/R)$, where $R$ is the number of communication rounds. Our result also demonstrates that the effects of device heterogeneity vanish asymptotically. For sufficiently large $R$, the objective values decrease monotonically and is bounded by ${\cal O}( 1 / (R T_{\rm avg}))$, where $T_{\rm avg}$ is the average number of local update steps per communication round across devices. Numerical experiments support our findings.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wong, H. F., Wai, H. T., & Yau, C. Y. (2026). Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices. https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices
MLA 9
Wong, Hok Fong, et al. "Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices." https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices.
Chicago (author–date)
Wong, Hok Fong, Hoi-To Wai, and Chung-Yiu Yau. 2026. "Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices." https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices.
Harvard
Wong, H. F., Wai, H. T. and Yau, C. Y. (2026) 'Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices', Available at: https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices.
Vancouver
Wong HF, Wai HT, Yau CY. Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices. https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices
IEEE
H. F. Wong, H. T. Wai, and C. Y. Yau, "Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices," https://omanscience.com/en/articles/large-stepsizes-federated-learning-on-logistic-regression-with-linearly-separable-data-the-case-of-heterogeneous-devices.