Abstract
Muon, an optimizer known for its efficiency, has a clear interpretation for matrix-valued updates, but convolutional kernels are stored as four-dimensional tensors. Standard implementations reshape these tensors into matrices, a shortcut which breaks the theoretical understanding behind Muon. To investigate this, we formalize the corresponding optimization objective directly in convolutional operator geometry and introduce Convolutional Newton-Schulz (Conv-NS), which approximates the polar factor in this geometry while preserving kernel support. When applied in fast training experiments, Conv-NS and reshape-based Muon are both computationally efficient and achieve comparable accuracy on CIFAR-10 and ImageNet classification tasks. However, as one could expect a theoretically aligned Conv-NS to outperform reshape-based Muon, we investigate this mismatch between practice and theoretical understanding, with the hypothesis that exact convolutional orthogonalization may overconstrain updates. These findings highlight Muon's strong practical performance while opening directions for its further development on convolutions. Our code is publicly available at \href{https://github.com/thib-s/muonconv-cifar10-airbench}{github conv-muon}.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Boissin, T., Massena, T., Serrurier, M., & Mamalet, F. (2026). Muon Is Theoretically Wrong For Convolutions, But Empirically Effective. https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective
MLA 9
Boissin, Thibaut, et al. "Muon Is Theoretically Wrong For Convolutions, But Empirically Effective." https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective.
Chicago (author–date)
Boissin, Thibaut, Thomas Massena, Mathieu Serrurier, and Franck Mamalet. 2026. "Muon Is Theoretically Wrong For Convolutions, But Empirically Effective." https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective.
Harvard
Boissin, T., Massena, T., Serrurier, M. and Mamalet, F. (2026) 'Muon Is Theoretically Wrong For Convolutions, But Empirically Effective', Available at: https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective.
Vancouver
Boissin T, Massena T, Serrurier M, Mamalet F. Muon Is Theoretically Wrong For Convolutions, But Empirically Effective. https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective
IEEE
T. Boissin, T. Massena, M. Serrurier, and F. Mamalet, "Muon Is Theoretically Wrong For Convolutions, But Empirically Effective," https://omanscience.com/en/articles/muon-is-theoretically-wrong-for-convolutions-but-empirically-effective.