Abstract
Processing long visual token sequences from high-resolution images makes multi-step reasoning computationally expensive for multimodal Large Language Models (MLLMs). Existing one-shot pruning and aggregation methods compress visual tokens into a fixed context before decoding. However, visual evidence needs can shift as reasoning unfolds, making it difficult for a fixed compressed context to retain all the details needed across stages. To address this challenge, we propose ViMoD, a lightweight framework that maintains a compact visual context while preserving access to original fine-grained evidence as reasoning needs evolve. Deformable Aggregation of Region-wise Tokens (DART) learns content-adaptive groups and aggregation capacities, constructing compact Coarse representations linked to recoverable original Fine tokens. Temporal Routing for Adaptive Contextual Evidence (TRACE) integrates decoding history to anticipate upcoming evidence needs and select, retain, or replace active Fine-token groups. Selected Fine tokens augment the persistent Coarse context in the frozen backbone, enabling stage-specific evidence access without continuously attending to all visual tokens. On Qwen3-VL-4B, ViMoD outperforms all evaluated baselines on all eight reasoning benchmarks at a 20% target visual token budget, improving the mean normalized score by 39.0% over the strongest evaluated one-shot baseline. These gains are achieved with only 0.0546% additional trainable parameters relative to the frozen backbone.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Xue, Y., Wu, H., Yang, J., Dong, M., Chen, X., Chen, H., & Guo, J. (2026). Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning. https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning
MLA 9
Xue, Yicheng, et al. "Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning." https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning.
Chicago (author–date)
Xue, Yicheng, Han Wu, Jufeng Yang, Minjing Dong, Xinghao Chen, Hanting Chen, and Jianyuan Guo. 2026. "Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning." https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning.
Harvard
Xue, Y., Wu, H., Yang, J., Dong, M., Chen, X., Chen, H. and Guo, J. (2026) 'Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning', Available at: https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning.
Vancouver
Xue Y, Wu H, Yang J, Dong M, Chen X, Chen H, et al. Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning. https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning
IEEE
Y. Xue, H. Wu, J. Yang, M. Dong, X. Chen, H. Chen, and J. Guo, "Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning," https://omanscience.com/en/articles/look-back-think-ahead-visual-memory-on-demand-for-efficient-multimodal-reasoning.