Abstract
While recent video generative models can synthesize high-fidelity videos, they struggle to portray plausible physical interactions and the resulting state transitions, a critical bottleneck for applications in robotics and VR/AR. To address this, we introduce a framework to generate a scalable synthetic dataset of controllable interactions. Our pipeline leverages a structured taxonomy and state-of-the-art image editing models to create explicit `start' and `end' state images, which serve as visual anchors for the interaction. To generate a seamless video utilizing these anchors, we propose State-Guided Sampling (SGS), a novel sampling technique that mitigates artifacts common in naive conditional generation. Furthermore, we develop and validate a new automated evaluation system that aligns with human judgments to ensure data quality. Experiments show that fine-tuning a base model on our dataset significantly enhances its ability to generate plausible interactions.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Jang, J., Kim, J., Kwak, N., & Kim, K. (2026). Bootstrapping Video Interaction Generation with Synthetic State Transitions. https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions
MLA 9
Jang, Jiho, et al. "Bootstrapping Video Interaction Generation with Synthetic State Transitions." https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions.
Chicago (author–date)
Jang, Jiho, JinYoung Kim, Nojun Kwak, and Kyungjune Kim. 2026. "Bootstrapping Video Interaction Generation with Synthetic State Transitions." https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions.
Harvard
Jang, J., Kim, J., Kwak, N. and Kim, K. (2026) 'Bootstrapping Video Interaction Generation with Synthetic State Transitions', Available at: https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions.
Vancouver
Jang J, Kim J, Kwak N, Kim K. Bootstrapping Video Interaction Generation with Synthetic State Transitions. https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions
IEEE
J. Jang, J. Kim, N. Kwak, and K. Kim, "Bootstrapping Video Interaction Generation with Synthetic State Transitions," https://omanscience.com/en/articles/bootstrapping-video-interaction-generation-with-synthetic-state-transitions.