Abstract
Large vision-language models (VLMs) have emerged as a powerful paradigm for urban and spatial AI. However, current state-of-the-art large VLMs still struggle with quantitative reasoning on remote sensing imagery. Existing benchmarks and algorithms are predominantly based on qualitative Visual Question Answering (VQA), providing limited insights into the quantitative reasoning capabilities of VLMs for built environment metrics. To address this gap, we develop Quantitative Urban and Spatial AI benchmark (USAI-Quant), the first benchmark designed to quantitatively evaluate VLM's reasoning capabilities on built environment metrics via remote sensing imagery. USAI-Quant is curated from the 335 largest U.S. cities, aligning high-resolution remote sensing images with quantitative built environment metrics. We then evaluate both general-purpose and remote sensing VLMs (RS-VLMs) by applying VQAs to tens of built environment metrics across three complexity levels. Our results reveal that current state-of-the-art models consistently fall short on numeric reasoning tasks. We further conduct in-depth analyses across models, question types, and geographic locations, uncovering insights into performance variability and task-specific challenges.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wang, D., Song, Q., Chen, Y., Balakrishnan, D., Srinivasan, R. S., & Wang, S. (2026). USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments. https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments
MLA 9
Wang, Dongdong, et al. "USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments." https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments.
Chicago (author–date)
Wang, Dongdong, Qingqi Song, Yuzhou Chen, Deepak Balakrishnan, Ravi Shankar Srinivasan, and Shenhao Wang. 2026. "USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments." https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments.
Harvard
Wang, D., Song, Q., Chen, Y., Balakrishnan, D., Srinivasan, R. S. and Wang, S. (2026) 'USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments', Available at: https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments.
Vancouver
Wang D, Song Q, Chen Y, Balakrishnan D, Srinivasan RS, Wang S. USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments. https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments
IEEE
D. Wang, Q. Song, Y. Chen, D. Balakrishnan, R. S. Srinivasan, and S. Wang, "USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments," https://omanscience.com/en/articles/usai-quant-a-quantitative-reasoning-benchmark-for-vision-language-models-in-built-environments.