Preprint Open access
Quadratic Weak-to-Strong Generalization in Random Feature Networks via Random Matrix Theory
Weak-to-strong generalization is the phenomenon where a strong student model trained with labels produced by a weak teacher model is able to generalize better than the teacher. In this paper, we study this phenomenon in two-layer random feature networks where the model strength is determined by its width. Using tools f …