Abstract
Robotic manipulation often requires inferring task-relevant states from past interactions when the current observation alone is insufficient to determine the appropriate action. Despite progress in benchmarking memory-augmented vision-language-action (VLA) models, application-oriented tasks requiring history-dependent semantic inference remain underrepresented. We introduce GiT (Grounded in Time), a dataset and benchmark for grounding manipulation decisions in past events across biolaboratory, household, and industrial scenarios. It includes real-robot and Universal Manipulation Interface (UMI) style demonstrations covering 18 bimanual tasks, together with simulation data and a ManiSkill-based evaluation suite covering nine tasks. Fine-grained subtask annotations and annotated counterfactual task pairs, in which similar current observations require different actions depending on prior events, support policy learning and targeted evaluation of history use. Evaluations of representative end-to-end VLA models in simulation and on selected real-world tasks reveal substantial room for improvement in history-dependent manipulation. The dataset and benchmark are available at the project page.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wang, Y., Yang, Y., Xu, G., Lin, S., Kang, N., Lu, P., Chen, X., Xue, Z., Deng, P., Liu, X., Yang, C., & Lu, Z. (2026). Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation. https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation
MLA 9
Wang, Yi, et al. "Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation." https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation.
Chicago (author–date)
Wang, Yi, Yang Yang, Guangqi Xu, Sumin Lin, Ning Kang, Pengxiang Lu, Xiaotong Chen, Zeyu Xue, Ping Deng, Xing Liu, Chenguang Yang, and Zhenyu Lu. 2026. "Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation." https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation.
Harvard
Wang, Y., Yang, Y., Xu, G., Lin, S., Kang, N., Lu, P., Chen, X., Xue, Z., Deng, P., Liu, X., Yang, C. and Lu, Z. (2026) 'Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation', Available at: https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation.
Vancouver
Wang Y, Yang Y, Xu G, Lin S, Kang N, Lu P, et al. Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation. https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation
IEEE
Y. Wang, Y. Yang, G. Xu, S. Lin, N. Kang, P. Lu, X. Chen, Z. Xue, P. Deng, X. Liu, C. Yang, and Z. Lu, "Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation," https://omanscience.com/en/articles/grounded-in-time-a-multi-source-dataset-and-benchmark-for-temporal-grounding-in-robotic-manipulation.