الملخص
Model merging offers a promising solution for combining multiple fine-tuned checkpoints into a single model through parameter arithmetic. However, finding optimal merging coefficients requires an extensive search that becomes prohibitively expensive as models scale in both size and number, due to high memory requirements and combinatorial growth in the search space. We show that, within the same model family, models exhibit highly congruent performance distributions over merging coefficients across different model sizes. This distributional similarity enables a practical paradigm we call \textit{$α$Transfer}: searching for optimal coefficients on a small proxy model, then directly transfer them to larger target models. We verify $α$Transfer across multiple merging methods, model families, and tasks. Experimental results demonstrate a 6$\times$ speedup and 70\% memory reduction on vision transformers, and a 20$\times$ speedup and 85\% memory reduction on large language models, while maintaining comparable performance. Our findings establish $α$Transfer as an efficient and generalizable approach to scaling model merging.
الكلمات المفتاحية
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Huang, S. C., Tam, Z. R., Lin, C. Y., Chen, Y. N., Lee, H. Y., & Sun, S. H. (2026). $α$Transfer: Coefficient Transfer for Efficient Model Merging. https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging
MLA 9
Huang, Shih-Cheng, et al. "$α$Transfer: Coefficient Transfer for Efficient Model Merging." https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging.
شيكاغو (المؤلف–التاريخ)
Huang, Shih-Cheng, Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Hung-yi Lee, and Shao-Hua Sun. 2026. "$α$Transfer: Coefficient Transfer for Efficient Model Merging." https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging.
هارفارد
Huang, S. C., Tam, Z. R., Lin, C. Y., Chen, Y. N., Lee, H. Y. and Sun, S. H. (2026) '$α$Transfer: Coefficient Transfer for Efficient Model Merging', Available at: https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging.
فانكوفر
Huang SC, Tam ZR, Lin CY, Chen YN, Lee HY, Sun SH. $α$Transfer: Coefficient Transfer for Efficient Model Merging. https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging
IEEE
S. C. Huang, Z. R. Tam, C. Y. Lin, Y. N. Chen, H. Y. Lee, and S. H. Sun, "$α$Transfer: Coefficient Transfer for Efficient Model Merging," https://omanscience.com/ar/articles/transfer-coefficient-transfer-for-efficient-model-merging.