الملخص
Task vectors provide a simple mechanism for composing learned capabilities through model merging. However, the composability of task vectors produced by on-policy distillation (OPD) remains largely unexplored. OPD trains a student using teacher feedback on student-generated trajectories, yielding parameter updates that differ from those produced by the teacher model, usually by reinforcement learning (RL). We therefore ask whether OPD task vectors can complement their RL teacher updates and compose effectively across tasks. Across five domains and two model architectures, we find evidence for both forms of composability. Within a task, merging OPD and RL task vectors can outperform both constituent models, even when the OPD student is weaker than its RL teacher. Across tasks, OPD task-vector compositions achieve higher average scores than corresponding RL compositions in seven of eight backbone-merging-rule comparisons. Parameter-space analyses reveal substantial non-collinearity between OPD and RL updates. Experiment in CODE domain on SMOLLM3-3B shows that the combined direction outperforms either constituent direction at the tested global update norm, supporting directional complementarity in this configuration. Across tasks, OPD updates also show lower overlap among the top-10% feed-forward channels ranked by update energy. Together, these results show that weaker standalone performance does not imply weaker task-vector composability. OPD task vectors can complement stronger RL teacher updates and combine effectively across tasks, highlighting composability as a distinct property for understanding and evaluating post-training updates.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Zhou, J., Han, F., Zhu, H., Zhou, Y., Zhang, R. Y., Xu, J., Gao, S., Guo, Q., & Zhang, X. Y. (2026). No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation. https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation
MLA 9
Zhou, Jingang, et al. "No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation." https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation.
شيكاغو (المؤلف–التاريخ)
Zhou, Jingang, Feiyu Han, Han Zhu, Yuyi Zhou, Rui-Yang Zhang, Jian Xu, Sirui Gao, Qingpei Guo, and Xu-Yao Zhang. 2026. "No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation." https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation.
هارفارد
Zhou, J., Han, F., Zhu, H., Zhou, Y., Zhang, R. Y., Xu, J., Gao, S., Guo, Q. and Zhang, X. Y. (2026) 'No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation', Available at: https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation.
فانكوفر
Zhou J, Han F, Zhu H, Zhou Y, Zhang RY, Xu J, et al. No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation. https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation
IEEE
J. Zhou, F. Han, H. Zhu, Y. Zhou, R. Y. Zhang, J. Xu, S. Gao, Q. Guo, and X. Y. Zhang, "No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation," https://omanscience.com/ar/articles/no-task-vector-is-an-island-a-comprehensive-study-on-the-composability-of-task-vectors-from-on-policy-distillation.