نسخة أولية وصول مفتوح
Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum
Practical Muon maintains momentum and performs a small, fixed number of Newton--Schulz iterations separately for each parameter matrix, often with a Nesterov correction. We analyze these layer-wise finite-step updates jointly on a coupled nonconvex objective, rather than replacing them by exact polar factors or one glo …