الملخص
Large vision-language models (LVLMs) exhibit strong multimodal in-context learning (ICL) capabilities, yet this ability degrades substantially as model size decreases. Knowledge distillation offers a natural way to bridge this gap, but existing methods primarily align output distributions or hidden representations directly. Such alignment teaches the student what the teacher predicts without revealing which evidence in the complex context causally supports that prediction. Consequently, a student can imitate the teacher's answer while continuing to rely on language priors, prompt structure, or other spurious cues. To address this limitation, we introduce Multimodal Causal Distillation (MCD), a distillation framework that transfers how a strong teacher uses multimodal evidence during ICL. MCD uses structure-preserving token interventions to identify and verify causal evidence, then transfers how the teacher responds when that evidence is retained or removed. This design connects distillation to the causal patterns by which the model uses contextual evidence during multimodal ICL. Experiments across three LVLM families and seven benchmarks show that MCD improves student performance by 7.23 points on average and outperforms vanilla distillation by 4.68 points, while further analyses confirm the generalizability of these gains.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Li, Y., Li, J., Xiao, C., Xiao, X., Wang, T., & Liu, Y. (2026). MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models. https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models
MLA 9
Li, Yanshu, et al. "MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models." https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models.
شيكاغو (المؤلف–التاريخ)
Li, Yanshu, Jiaqian Li, Canran Xiao, Xi Xiao, Tianyang Wang, and Yongtai Liu. 2026. "MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models." https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models.
هارفارد
Li, Y., Li, J., Xiao, C., Xiao, X., Wang, T. and Liu, Y. (2026) 'MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models', Available at: https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models.
فانكوفر
Li Y, Li J, Xiao C, Xiao X, Wang T, Liu Y. MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models. https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models
IEEE
Y. Li, J. Li, C. Xiao, X. Xiao, T. Wang, and Y. Liu, "MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models," https://omanscience.com/ar/articles/mcd-causal-distillation-of-multimodal-in-context-learning-in-large-vision-language-models.