Abstract
Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. In this work, we present a mechanistic analysis of this failure mode, identifying two critical bottlenecks inherent to the global attention pipeline of MLLMs. First, we reveal an individuation bottleneck stemming from image patchification: because Vision Transformers process patches independently, they struggle to group fragmented geometric features across boundaries into distinct object representations. Second, we identify a collapse in the subsequent counting aggregation process, where representation separation rapidly diminishes as numerosity increases due to attention compression. Identifying and formalizing these twin bottlenecks constitutes our first major contribution. To overcome them, we propose ConvStack, a lightweight architecture that operates directly in the visual token space to explicitly aggregate and inject local spatial structures via zero-initialized residual connections. By explicitly addressing the individuation bottleneck, ConvStack provides unambiguous geometric evidence for downstream aggregation. Remarkably, by fine-tuning exclusively on counting tasks, the model achieves substantial improvements in dense object counting and broader spatial understanding benchmarks, without compromising on general visual capabilities.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Che, L., Quan, Y., Fang, S., Wang, H., Krishna, R., Tang, R., & Pavlovic, V. (2026). Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack. https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack
MLA 9
Che, Liwei, et al. "Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack." https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack.
Chicago (author–date)
Che, Liwei, Yihao Quan, Sen Fang, Hongyi Wang, Ranjay Krishna, Ruixiang Tang, and Vladimir Pavlovic. 2026. "Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack." https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack.
Harvard
Che, L., Quan, Y., Fang, S., Wang, H., Krishna, R., Tang, R. and Pavlovic, V. (2026) 'Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack', Available at: https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack.
Vancouver
Che L, Quan Y, Fang S, Wang H, Krishna R, Tang R, et al. Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack. https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack
IEEE
L. Che, Y. Quan, S. Fang, H. Wang, R. Krishna, R. Tang, and V. Pavlovic, "Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack," https://omanscience.com/en/articles/why-mllms-struggle-to-count-overcoming-individuation-and-aggregation-bottlenecks-with-convstack.