Abstract
We propose UniDynamics, a diffusion-based framework for future 4D dynamic scenes (RGB, depth, and optical flow) generation from a single event-RGB pair, without requiring long histories or control priors as in existing methods, while explicitly modeling future motion fields. The core idea is to leverage event streams to offer an alternative motion prior for single-RGB extrapolation, and to enforce geometric and motion constraints throughout generation via multimodal modeling. Specifically, we design an Event Latent Enhancement (ELE) module to align and enhance event latents into diffusion-injectable conditioning features, providing robust initial motion priors and reliable texture/structure cues. We further introduce a Perceptual Dynamics Space (PDS) embedded in the multi-scale U-Net, which decouples and adaptively interacts depth and flow while continuously feeding back constraints to appearance features, improving geometric-motion consistency for physically plausible and spatiotemporally coherent prediction. Experiments on VKitti2 and DSEC demonstrate state-of-the-art performance, producing high-quality, temporally coherent, and 4D-consistent future predictions, especially under challenging high-speed motion blur.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Liu, D., Zhan, X., Wang, T., Wang, X., & Sun, C. (2026). UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation. https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation
MLA 9
Liu, Daikun, et al. "UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation." https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation.
Chicago (author–date)
Liu, Daikun, Xin Zhan, Teng Wang, Xiaoping Wang, and Changyin Sun. 2026. "UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation." https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation.
Harvard
Liu, D., Zhan, X., Wang, T., Wang, X. and Sun, C. (2026) 'UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation', Available at: https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation.
Vancouver
Liu D, Zhan X, Wang T, Wang X, Sun C. UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation. https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation
IEEE
D. Liu, X. Zhan, T. Wang, X. Wang, and C. Sun, "UniDynamics: Event-RGB Fusion for Unified Future 4D Dynamic Scene Generation," https://omanscience.com/en/articles/unidynamics-event-rgb-fusion-for-unified-future-4d-dynamic-scene-generation.