Abstract
Specialized models encode task-oriented behavior, but transferring that behavior to a general language model usually requires training, distillation, or representation alignment. We study whether such ability can instead be transferred directly at the parameter level. We apply two existing training-free heterogeneous merging methods, previously shown to transfer knowledge between general language models, to specialist-to-general transfer, projecting a specialist donor into the recipient's shape and interpolating backbone parameters without gradient updates or semantic alignment. Intersection-Merge (IM) injects a prefix-aligned donor slice matching the recipient shape, while Activate-Prune-Merge (APM) uses forward-pass activation statistics to select which donor dimensions to retain before injection. Across embedding, reranking, reward modeling, and MoE code-specialist transfer, both methods improve the general recipient, showing that simple heterogeneous merging can move capabilities across diverse specialist roles.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Fan, J., Chen, S., Hou, Y., Xia, W., Xu, K., Xie, H., & Chen, E. (2026). Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer. https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer
MLA 9
Fan, Jiahe, et al. "Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer." https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer.
Chicago (author–date)
Fan, Jiahe, Si Chen, Yinghao Hou, Wenbo Xia, Ke Xu, Hong Xie, and Enhong Chen. 2026. "Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer." https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer.
Harvard
Fan, J., Chen, S., Hou, Y., Xia, W., Xu, K., Xie, H. and Chen, E. (2026) 'Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer', Available at: https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer.
Vancouver
Fan J, Chen S, Hou Y, Xia W, Xu K, Xie H, et al. Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer. https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer
IEEE
J. Fan, S. Chen, Y. Hou, W. Xia, K. Xu, H. Xie, and E. Chen, "Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer," https://omanscience.com/en/articles/exploring-heterogeneous-model-merging-approach-for-complex-knowledge-transfer.