Preprint Open access
QAM: Quadratic-Accurate Checkpoint Merging via Sequential Consistency
Saved checkpoints record states along a training trajectory, but generally do not determine the updates at states that would be visited under a different schedule. We study how accurately these checkpoints can reconstruct the endpoint of a sequential reference with prescribed update strengths. Under a common local tran …