Abstract
The Straight-Through Estimator (STE) is a widely used heuristic for Quantization-Aware Training (QAT), but its surrogate gradients can exhibit substantial mismatch with the underlying quantized objective, leading to noisy updates and parameter oscillations, particularly in ultra-low-bit regimes. We propose the Quantization-Aware Minimal-Norm Optimizer (Q-MINO), a temporal bundle method that combines gradient consensus, state-drift regularization, and an alignment constraint to construct stabilized, minimum-norm update directions from recent optimization states. Q-MINO solves the resulting constrained subproblem using a warm-started Frank--Wolfe procedure with a feasible fallback initialization. Theoretically, via a stochastic Lyapunov Kurdyka--Łojasiewicz (KL) framework, we show that Q-MINO achieves asymptotic neighborhood convergence. Moreover, we detail numerical experiments with Q-MINO at various quantizations.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Li, D. (2026). Q-MINO: A Minimal-Norm Method for Quantization-Aware Training. https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training
MLA 9
Li, Don. "Q-MINO: A Minimal-Norm Method for Quantization-Aware Training." https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training.
Chicago (author–date)
Li, Don. 2026. "Q-MINO: A Minimal-Norm Method for Quantization-Aware Training." https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training.
Harvard
Li, D. (2026) 'Q-MINO: A Minimal-Norm Method for Quantization-Aware Training', Available at: https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training.
Vancouver
Li D. Q-MINO: A Minimal-Norm Method for Quantization-Aware Training. https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training
IEEE
D. Li, "Q-MINO: A Minimal-Norm Method for Quantization-Aware Training," https://omanscience.com/en/articles/q-mino-a-minimal-norm-method-for-quantization-aware-training.