Abstract
Graphical User Interface (GUI) agents have emerged as a promising paradigm for automating complex digital workflows across diverse applications. However, training highly capable and generalizable agents fundamentally relies on massive, high-fidelity visual-action trajectories, which are notoriously difficult to acquire. While human demonstrations are unscalable, existing GUI world models rely on text descriptions or HTML rendering, discarding crucial pixel-level visual details like icons and layout styles. To address this issue, we introduce Infinite-Dreamer, a simulation-free data synthesis method powered by a pixel-level Image Editing World Model. By conceptualizing GUI transitions as image editing tasks, we leverage Vision-Language Models (VLMs) to describe action-induced UI changes as structured delta-text. We then fine-tune an image editing backbone to controllably synthesize realistic screenshot transitions. We utilize this model to generate both single-frame visual robustness data and multi-step imaginary trajectories. To validate the effectiveness of our approach, we fine-tune the Qwen3-VL baseline solely on the synthesized data to obtain Infinite-Actor, and evaluate it on AndroidWorld, MobileWorld, and AndroidControl-Curated benchmarks. Infinite-Actor consistently outperforms the Qwen3-VL baselines across scales: Infinite-Actor-8B improves AndroidWorld Pass@1 by +4.45 and nearly doubles the MobileWorld Pass@3 success rate, while Infinite-Actor-2B improves Pass@1 by +9.05. Code is available at https://github.com/swaydy-n/Infinite-Dreamer.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Ning, Y., Niu, R., Xing, Q., Duan, Z., He, Q., Wang, P., & Wang, Q. (2026). Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training. https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training
MLA 9
Ning, Yongxin, et al. "Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training." https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training.
Chicago (author–date)
Ning, Yongxin, Runliang Niu, Qianli Xing, Zhiyi Duan, Qingzu He, Pan Wang, and Qi Wang. 2026. "Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training." https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training.
Harvard
Ning, Y., Niu, R., Xing, Q., Duan, Z., He, Q., Wang, P. and Wang, Q. (2026) 'Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training', Available at: https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training.
Vancouver
Ning Y, Niu R, Xing Q, Duan Z, He Q, Wang P, et al. Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training. https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training
IEEE
Y. Ning, R. Niu, Q. Xing, Z. Duan, Q. He, P. Wang, and Q. Wang, "Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training," https://omanscience.com/en/articles/imagine-to-act-high-fidelity-data-synthesis-via-image-editing-world-model-for-scalable-gui-agent-training.