Abstract
Spiking Neural Networks (SNNs) offer an energy-efficient alternative to conventional neural networks by exploiting sparse, binary spikes, and event-driven computation. However, the training of SNNs remains challenging, as spiking activations create a sharp loss landscape that hinders training, and diagonal-curvature optimizers such as the Adam family may fail to capture this geometry. The extension of curvature-based optimization methods to SNNs is further complicated by the sparse, discrete, and temporally recurrent nature of their underlying dynamics. To address these limitations, we propose SpiKFAX, a second-order optimization method that formulates a computationally tractable, Kronecker-factored approximation of the Fisher information matrix specifically adapted to the structure of SNNs. Empirical evaluation across five architectures and seven datasets demonstrates that SpiKFAX consistently yields improvements in test accuracy and training stability relative to other popular optimizers.
Keywords
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Doan, N. P., & Alouani, I. (2026). On the second-order optimization for spiking neural networks. https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks
MLA 9
Doan, Ngoc Phu, and Ihsen Alouani. "On the second-order optimization for spiking neural networks." https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks.
Chicago (author–date)
Doan, Ngoc Phu, and Ihsen Alouani. 2026. "On the second-order optimization for spiking neural networks." https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks.
Harvard
Doan, N. P. and Alouani, I. (2026) 'On the second-order optimization for spiking neural networks', Available at: https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks.
Vancouver
Doan NP, Alouani I. On the second-order optimization for spiking neural networks. https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks
IEEE
N. P. Doan, and I. Alouani, "On the second-order optimization for spiking neural networks," https://omanscience.com/en/articles/on-the-second-order-optimization-for-spiking-neural-networks.