Abstract
Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events. Retrieving relevant events therefore does not necessarily recover the "biography" of the particular entity a question concerns. To address this, we introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment. During question answering, the biography is retrieved alongside episodic evidence, allowing the model to follow an entity through events using identity links established during memory construction. Evaluations across four benchmarks, including day-long and week-long recordings, demonstrate improvements over prior memory frameworks in both multiple-choice and open-ended question answering. On EgoLifeQA, GEB achieves 72.0% accuracy, 4.4 percentage points above the best published result. Ablations show that grounded identity association and biography reading both contribute to the gains, which additional descriptions alone do not fully recover.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Ren, H., Fan, L., Pao, H., Guo, H., Zia, Z., Chen, Y., Schwing, A., & Hua, G. (2026). Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies. https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies
MLA 9
Ren, Hui, et al. "Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies." https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies.
Chicago (author–date)
Ren, Hui, Lei Fan, Henry Pao, Han Guo, Zeeshan Zia, Ying Chen, Alexander Schwing, and Gang Hua. 2026. "Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies." https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies.
Harvard
Ren, H., Fan, L., Pao, H., Guo, H., Zia, Z., Chen, Y., Schwing, A. and Hua, G. (2026) 'Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies', Available at: https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies.
Vancouver
Ren H, Fan L, Pao H, Guo H, Zia Z, Chen Y, et al. Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies. https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies
IEEE
H. Ren, L. Fan, H. Pao, H. Guo, Z. Zia, Y. Chen, A. Schwing, and G. Hua, "Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies," https://omanscience.com/en/articles/beyond-the-timeline-augmenting-long-video-memory-with-grounded-entity-biographies.