Abstract
Reinforcement learning is increasingly used to post-train vision-language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Kaduri, O., Feingold, K., Isola, P., & Dekel, T. (2026). Reinforcement Learning from Intermediate Renders for Image-to-Code Generation. https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation
MLA 9
Kaduri, Omri, et al. "Reinforcement Learning from Intermediate Renders for Image-to-Code Generation." https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation.
Chicago (author–date)
Kaduri, Omri, Kate Feingold, Phillip Isola, and Tali Dekel. 2026. "Reinforcement Learning from Intermediate Renders for Image-to-Code Generation." https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation.
Harvard
Kaduri, O., Feingold, K., Isola, P. and Dekel, T. (2026) 'Reinforcement Learning from Intermediate Renders for Image-to-Code Generation', Available at: https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation.
Vancouver
Kaduri O, Feingold K, Isola P, Dekel T. Reinforcement Learning from Intermediate Renders for Image-to-Code Generation. https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation
IEEE
O. Kaduri, K. Feingold, P. Isola, and T. Dekel, "Reinforcement Learning from Intermediate Renders for Image-to-Code Generation," https://omanscience.com/en/articles/reinforcement-learning-from-intermediate-renders-for-image-to-code-generation.