Preprint Open access
ReTeach: Building a Self-Teacher through Multi-Round Reflection and Retry
Self-distillation can improve reasoning without a separately trained, more capable teacher, but its effectiveness depends on how the self-teacher gains an advantage over the student. Conditioning the teacher on reference answers or solutions can provide such an advantage, but this information may be unavailable. Reflec …