Abstract
The current deep learning training paradigm employs end-to-end backpropagation, regardless of the training stage, i.e. pre-training or fine-tuning. However, backpropagating through the entire model is neither biologically plausible nor memory efficient, since learning inside the brain is highly localized. Therefore, we propose LocalProp, a training procedure that locally updates the weights of a model. Our neuro-localized weight updates follow the "pre-training then fine-tuning" paradigm, where the pre-training is based on I-JEPA. After locally updating the weights, a pruning operation is performed, followed by a short final fine-tuning phase. Pruning helps by sending the learning signal from higher blocks to lower blocks. We perform experiments on several datasets, including large-scale benchmarks such as ImageNet, and empirically show that LocalProp reaches good performance at a fraction of GPU peak memory. By varying the number of jointly optimized blocks, we identify gradient-propagation span as a practical control over the accuracy-memory trade-off.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Grigore, D. N., Georgescu, I., & Ionescu, R. T. (2026). LocalProp: Neuro-Localized Memory-Efficient Backpropagation. https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation
MLA 9
Grigore, Diana-Nicoleta, et al. "LocalProp: Neuro-Localized Memory-Efficient Backpropagation." https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation.
Chicago (author–date)
Grigore, Diana-Nicoleta, Iuliana Georgescu, and Radu Tudor Ionescu. 2026. "LocalProp: Neuro-Localized Memory-Efficient Backpropagation." https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation.
Harvard
Grigore, D. N., Georgescu, I. and Ionescu, R. T. (2026) 'LocalProp: Neuro-Localized Memory-Efficient Backpropagation', Available at: https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation.
Vancouver
Grigore DN, Georgescu I, Ionescu RT. LocalProp: Neuro-Localized Memory-Efficient Backpropagation. https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation
IEEE
D. N. Grigore, I. Georgescu, and R. T. Ionescu, "LocalProp: Neuro-Localized Memory-Efficient Backpropagation," https://omanscience.com/en/articles/localprop-neuro-localized-memory-efficient-backpropagation.