Abstract
Embodied agents must reason about 3D space while the video is still arriving, answering questions as soon as they have observed enough of the scene. VLMs that incorporate 3D geometric priors achieve strong spatial reasoning, but they operate offline, i.e., the full video must be available before they produce an answer. Streaming VLMs process frames causally and decide for themselves when to respond, yet they lack explicit 3D representations. We present SpaTime, a streaming VLM that fuses causal geometry tokens into the language model at every frame, using only the frames observed so far. To supervise when the model answers, we propose a response-time loss that maps per-frame response probabilities to a differentiable expected response time and penalizes the distance from the ground-truth frame. For evaluation, we construct StreamVSTI-Bench and StreamVSI-Bench, streaming adaptations of VSTI-Bench and VSI-Bench. On StreamVSTI-Bench, SpaTime reaches 49.2% overall accuracy and reduces the mean response-time error by 66% relative to the strongest streaming baseline.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Yin, H., Zhan, H., Chng, S. F., Xu, Y., & Yeh, R. A. (2026). SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning. https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning
MLA 9
Yin, Hairong, et al. "SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning." https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning.
Chicago (author–date)
Yin, Hairong, Huangying Zhan, Shin-Fang Chng, Yi Xu, and Raymond A. Yeh. 2026. "SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning." https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning.
Harvard
Yin, H., Zhan, H., Chng, S. F., Xu, Y. and Yeh, R. A. (2026) 'SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning', Available at: https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning.
Vancouver
Yin H, Zhan H, Chng SF, Xu Y, Yeh RA. SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning. https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning
IEEE
H. Yin, H. Zhan, S. F. Chng, Y. Xu, and R. A. Yeh, "SpaTime: Streaming Vision-Language Models for Spatio-temporal Reasoning," https://omanscience.com/en/articles/spatime-streaming-vision-language-models-for-spatio-temporal-reasoning.