Preprint Open access
Adaptive Mutual Distillation for Balanced Multi-Task Post-Training of Large Language Models
Multi-task post-training of large language models (LLMs) aims to improve performance across tasks with unequal amounts of training data. Existing methods focus primarily on balancing task contributions during single-model training. Different task-balancing strategies can produce models with complementary strengths, cre …