Abstract
Camera-based 3D perception for autonomous driving relies heavily on large annotated datasets, and deploying such a system to a new target region typically requires data collection and annotation. Generative augmentation has been proposed to reduce this cost, but existing approaches face a fundamental trade-off: label-conditioned methods consume the very annotations they aim to replace, while simulator-conditioned methods offer free annotations but lack visual grounding to specific real environments. This work investigates the extent to which a digital-twin-driven Real2Sim2Real pipeline (DT-R2S2R) can substitute for target-region real data. By reconstructing recorded driving clips inside a georeferenced digital twin (DT-R2S), we condition a diffusion model on geometrically aligned simulator renderings, establishing a digital twin-grounded Sim2Real model (DT-S2R). As a result, DT-S2R synthesizes photorealistic driving images given low-cost yet georeferenced simulator data across both reconstructed and novel simulator scenes within digital-twin coverage. The efficacy of generated data is verified on diverse 3D detectors. DETR3D, especially, reports 93.18% of mAP obtained by a target-region real-data oracle, without employing target images for detector training. Furthermore, simple co-training with existing out-of-target real data outperforms the oracle. Thus, DT-R2S2R can substantially reduce the cost of manual on-site data collection and annotation in digital twin-available districts, providing a practical foundation for scaling 3D perception.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Lim, H., Jeon, H., Kim, D., Jung, S., & Yoo, H. (2026). Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction. https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction
MLA 9
Lim, Hojun, et al. "Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction." https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction.
Chicago (author–date)
Lim, Hojun, Hyeongseok Jeon, Donghyun Kim, Soonyoung Jung, and Heecheol Yoo. 2026. "Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction." https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction.
Harvard
Lim, H., Jeon, H., Kim, D., Jung, S. and Yoo, H. (2026) 'Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction', Available at: https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction.
Vancouver
Lim H, Jeon H, Kim D, Jung S, Yoo H. Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction. https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction
IEEE
H. Lim, H. Jeon, D. Kim, S. Jung, and H. Yoo, "Digital Twin-Driven Real2Sim2Real: Simulator-Conditioned Generation via Paired Driving-Scene Reconstruction," https://omanscience.com/en/articles/digital-twin-driven-real2sim2real-simulator-conditioned-generation-via-paired-driving-scene-reconstruction.