Preprint Open access
$λ$-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
Joint-embedding self-supervised learning typically combines an invariance objective across augmented views with additional mechanisms to prevent representational collapse. These objectives are often applied after a projection head, while downstream tasks use the backbone representation before the projector. We find tha …