Abstract
Adapting image restoration models to a stream of new tasks without revisiting past data remains challenging due to catastrophic forgetting. In this work, we propose Restoring without Forgetting (RwF), a filter-level continual adaptation framework for image restoration built upon a critical observation: task-specific knowledge is centered in a small subset of filters and can be separated from those reconstructing general content. RwF first performs parameter-space integrated gradients attribution to localize degradation-critical filters in a coarse-to-fine manner. It then adapts to new tasks by generating task-specific filters from a filter bank using compact factorized low-rank transformations, further augmented with cross-task attention and prototypical contrastive learning, and lastly assembles them back only at localized positions. Experiments on six restoration tasks show that RwF effectively avoids forgetting and achieves competitive restoration quality against all-in-one methods that have full data access, and outperforms LoRA-style adaptation with $\sim$10$\times$ fewer additional parameters. Code is available at https://github.com/funkdub/Restoring-without-Forgetting.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Feng, X., Zhao, J., Zhang, Y. Z., Pei, W., Chen, F., & Lu, G. (2026). Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients. https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients
MLA 9
Feng, Xin, et al. "Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients." https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients.
Chicago (author–date)
Feng, Xin, Jin Zhao, Yi-Zhen Zhang, Wenjie Pei, Fanglin Chen, and Guangming Lu. 2026. "Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients." https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients.
Harvard
Feng, X., Zhao, J., Zhang, Y. Z., Pei, W., Chen, F. and Lu, G. (2026) 'Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients', Available at: https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients.
Vancouver
Feng X, Zhao J, Zhang YZ, Pei W, Chen F, Lu G. Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients. https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients
IEEE
X. Feng, J. Zhao, Y. Z. Zhang, W. Pei, F. Chen, and G. Lu, "Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients," https://omanscience.com/en/articles/restoring-without-forgetting-filter-level-continual-image-restoration-via-parameter-space-integrated-gradients.