Abstract
Vision-language models (VLMs) exhibit strong multimodal capabilities but remain vulnerable to backdoors implanted through poisoned fine-tuning data. Existing defenses often require extensive parameter updates during fine-tuning or incur per-query overhead during inference. To address these limitations, we propose Perturb-Select-Restore (PSR), a post-training defense that performs sparse updates to the projection interface and introduces no additional computation during inference. We reveal that backdoored VLM projectors are substantially more sensitive to bounded perturbations than clean VLM projectors, a phenomenon we term projection fragility. Building on this finding, PSR identifies the output channels most sensitive to perturbations in each projection layer of a backdoored VLM and restores their parameters to the corresponding pretrained values. Experiments across multiple tasks show that PSR reduces attack success rates to near zero while preserving clean-task performance.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Liu, S., Zhang, Z., Yuan, S., Aung, K. M. M., Lin, Z., & Feng, L. (2026). Selective Channel Restoration for Backdoored Vision-Language Models. https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models
MLA 9
Liu, Shuming, et al. "Selective Channel Restoration for Backdoored Vision-Language Models." https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models.
Chicago (author–date)
Liu, Shuming, Zhifang Zhang, Suqin Yuan, Khin Mi Mi Aung, Zhuoyi Lin, and Lei Feng. 2026. "Selective Channel Restoration for Backdoored Vision-Language Models." https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models.
Harvard
Liu, S., Zhang, Z., Yuan, S., Aung, K. M. M., Lin, Z. and Feng, L. (2026) 'Selective Channel Restoration for Backdoored Vision-Language Models', Available at: https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models.
Vancouver
Liu S, Zhang Z, Yuan S, Aung KMM, Lin Z, Feng L. Selective Channel Restoration for Backdoored Vision-Language Models. https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models
IEEE
S. Liu, Z. Zhang, S. Yuan, K. M. M. Aung, Z. Lin, and L. Feng, "Selective Channel Restoration for Backdoored Vision-Language Models," https://omanscience.com/en/articles/selective-channel-restoration-for-backdoored-vision-language-models.