نسخة أولية وصول مفتوح
Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing
In this paper, we present a Mixture-of-Experts (MoE) quantization method based on activation entropy. Although quantization reduces memory and computational costs, it can substantially degrade performance. In particular, performance decline is pronounced in quantized MoE models, where individual experts have a small nu …