Abstract
We present a comprehensive benchmark of Federated Learning (FL) for multilingual Automatic Speech Recognition (ASR), evaluating four Speech-LLM architectures on the Multilingual LibriSpeech dataset. We compare FedAvg and FedProx across frozen and unfrozen encoder configurations, demonstrating that optimized learning rates are critical for performance. Specifically, independently tuning the learning rates for the speech encoder, connector, and decoder yields the lowest error rates, with full three-component adaptation (LoRA for encoder and decoder, full training for the connector) producing the best FL results. We observe that FedProx efficacy is architecture-dependent, providing notable advantages in multilingual pre-trained architectures (e.g., EuroLLM over TinyLlama when keeping the encoder fixed); this indicates that LLM backbone capacity plays a key role in mediating resilience to heterogeneous data distributions. These findings offer concrete design guidance for deploying multilingual Speech-LLMs in privacy-sensitive, distributed environments.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Luque, J., Sant, A., & López, F. (2026). Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking. https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking
MLA 9
Luque, Jordi, et al. "Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking." https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking.
Chicago (author–date)
Luque, Jordi, Aleix Sant, and Fernando López. 2026. "Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking." https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking.
Harvard
Luque, J., Sant, A. and López, F. (2026) 'Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking', Available at: https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking.
Vancouver
Luque J, Sant A, López F. Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking. https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking
IEEE
J. Luque, A. Sant, and F. López, "Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarking," https://omanscience.com/en/articles/federated-multilingual-speech-llms-architecture-and-aggregation-strategy-benchmarking.