Abstract
Reference-guided diffusion editors struggle to faithfully reproduce user-provided references. We identify a potential bottleneck in diffusion editors: many methods provide limited reference-attention allocation. For example, in LoomVideo, edit-region queries assign less than 1% of their attention mass to the reference. We introduce RefGAP, a training-free correction that determines logit-offset magnitudes online at each layer from the reference-attention mass measured during the forward pass. Positive offsets to reference logits strengthen reference usage by edit-region queries, while negative offsets for keep-region queries limit reference-induced changes outside the edit. Two global coefficients control the correction; they are selected once on validation data from four development diffusion editors and held fixed. Across seven diffusion-based image/video editors, RefGAP improves identity fidelity in head swapping and face swapping. RefGAP achieves a fidelity-preservation trade-off comparable to separately tuned constant edit-side biases, without per-approach strength sweeps. Additional experiments on virtual try-on and background replacement evaluate transfer beyond identity editing.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Wang, Y., Liao, S., Liu, G., & Liang, X. (2026). Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing. https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing
MLA 9
Wang, Yanan, et al. "Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing." https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing.
Chicago (author–date)
Wang, Yanan, Shengcai Liao, Guangyi Liu, and Xiaodan Liang. 2026. "Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing." https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing.
Harvard
Wang, Y., Liao, S., Liu, G. and Liang, X. (2026) 'Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing', Available at: https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing.
Vancouver
Wang Y, Liao S, Liu G, Liang X. Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing. https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing
IEEE
Y. Wang, S. Liao, G. Liu, and X. Liang, "Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing," https://omanscience.com/en/articles/mind-the-refgap-correcting-reference-attention-in-diffusion-based-visual-editing.