Preprint Open access
DGA-Muon: Decoupled Geometry-Aligned Adaptive Scaling for Muon
While NorMuon has achieved strong empirical performance in pretraining, its underlying adaptive mechanism remains largely heuristic and poorly understood. In this work, we provide the first systematic theoretical analysis of NorMuon's adaptivity, revealing that it primarily arises from orthogonalization-induced geometr …