Preprint Open access
Shared Low-rank Basis Factorization for Data-free Mixture-of-Experts Compression
Mixture-of-Experts (MoE) large language models decouple capacity from compute through sparse routing, but their large parameter count creates storage and serving challenges. We analyze three MoE compression families: expert pruning, expert merging, and weight reconstruction, and derive structural error bounds showing t …