الملخص
Multimodal Large Language Models (MLLMs) suffer from hallucinations, creating a critical need for Uncertainty Quantification (UQ) to ensure reliable deployment. However, existing approaches struggle to detect uncertainty caused by superficial associations, especially when the query-relevant signal is weak. We mainly attribute this issue to their bias toward aleatoric uncertainty arising from data ambiguity, overlooking epistemic uncertainty stemming from model limitations. To further decompose uncertainty types for a comprehensive UQ, we propose Causal-Invariant Masking (CIM), which measures the semantic shift between the original predictions and those conditioned on a causally-focused view. Based on this framework, we introduce Semantic Divergence as our core metric for UQ and provide theoretical evidence that it converges to the variance of model's sensitivity to non-causal correlations, establishing its ability to capture MLLM's limitation. To accelerate UQ in MLLMs, we further propose Expected Embedding Drift (EED), a fast geometric proxy metric that estimates semantic shift directly within the hyperspherical embedding space. Experiments show that our method achieves state-of-the-art performance on various benchmarks, while the proposed EED accelerates by nearly 50% with comparable performance.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Luo, H., Tao, L., Gui, J., Chen, X., Xu, C., Guo, J., & Dong, M. (2026). Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking. https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking
MLA 9
Luo, Haoyang, et al. "Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking." https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking.
شيكاغو (المؤلف–التاريخ)
Luo, Haoyang, Linwei Tao, Jie Gui, Xinghao Chen, Chang Xu, Jianyuan Guo, and Minjing Dong. 2026. "Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking." https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking.
هارفارد
Luo, H., Tao, L., Gui, J., Chen, X., Xu, C., Guo, J. and Dong, M. (2026) 'Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking', Available at: https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking.
فانكوفر
Luo H, Tao L, Gui J, Chen X, Xu C, Guo J, et al. Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking. https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking
IEEE
H. Luo, L. Tao, J. Gui, X. Chen, C. Xu, J. Guo, and M. Dong, "Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking," https://omanscience.com/ar/articles/revealing-epistemic-uncertainty-in-mllms-via-causal-invariant-masking.