With the growing interest in spiking neural networks (SNNs), the development of open-source neuromorphic training frameworks has also accelerated. However, there is currently a lack of systematic guidelines for selecting these frameworks. To address this issue, this paper proposes a benchmarking method for SNNs based on image classification tasks. This method designs a convolutional neural network and a fully connected deep neural networks to evaluate two SNN training approaches: direct training with surrogate gradient backpropagation and conversion from artificial neural networks (ANNs) to SNNs. Based on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark image datasets, the performance comparisons of various neuromorphic training frameworks are conducted using evaluation metrics such as training time and classification accuracy. Experimental results indicate that the neuromorphic training framework SpikingJelly outperforms others in terms of both training time and classification accuracy in direct SNN training, while the Lava framework achieves the highest classification accuracy in ANN-to-SNN conversion training.
HU Wangxin, CHENG Yingchao, HE Yulin, HUANG Zhexue, CAI Zhanchuan
. Benchmarking of Spiking Neural Networks and Performance Evaluation of Neuromorphic Training Frameworks[J]. Journal of Applied Sciences, 2025
, 43(1)
: 169
-182
.
DOI: 10.3969/j.issn.0255-8297.2025.01.012
[1] 郑南宁. 人工智能新时代[J]. 智能科学与技术学报, 2019, 1(1): 1-3. Zheng N N. The new era of artificial intelligence [J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 1-3. (in Chinese)
[2] 黄铁军, 余肇飞, 刘怡俊. 类脑机的思想与体系结构综述[J]. 计算机研究与发展, 2019, 56(6): 1135-1148. Huang T J, Yu Z F, Liu Y J. Brain-like machine: thought and architecture [J]. Journal of Computer Research and Development, 2019, 56(6): 1135-1148. (in Chinese)
[3] 张铁林, 徐波. 脉冲神经网络研究现状及展望[J]. 计算机学报, 2021, 44(9): 1767-1785. Zhang T L, Xu B. Research advances and perspectives on spiking neural networks [J]. Chinese Journal of Computers, 2021, 44(9): 1767-1785. (in Chinese)
[4] Roy K, Jaiswal A, Panda P. Towards spike-based machine intelligence with neuromorphic computing [J]. Nature, 2019, 575(7784): 607-617.
[5] Rathi N, Chakraborty I, Kosta A, et al. Exploring neuromorphic computing based on spiking neural networks: algorithms to hardware [J]. ACM Computing Surveys, 2023, 55(12): 1-49.
[6] Yang S M, Chen B D. Effective surrogate gradient learning with high-order information bottleneck for spike-based machine intelligence [J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(8): 1-15.
[7] Mostafa H. Supervised learning based on temporal coding in spiking neural networks [J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(7): 3227-3235.
[8] Bu T, Ding J, Yu Z, et al. Optimized potential initialization for low-latency spiking neural networks [C]//36th AAAI Conference on Artificial Intelligence, 2022: 11-20.
[9] Hao Z, Bu T, Ding J, et al. Reducing ANN-SNN conversion error through residual membrane potential [J]//37th AAAI Conference on Artificial Intelligence, 2023: 11-21.
[10] Baltes M, Abuhajar N, Yue Y, et al. Joint ANN-SNN co-training for object localization and image segmentation [C]//IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023: 1-5.
[11] Bu T, Fang W, Ding J, et al. Optimal ANN-SNN conversion for high-accuracy and ultra-lowlatency spiking neural networks [DB/OL]. 2023[2024-08-08]. https://arxiv.org/abs/2303.04347.
[12] Hu Y F, Tang H J, Pan G. Spiking deep residual networks [J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(8): 5200-5205.
[13] Zenke F, Ganguli S. SuperSpike: supervised learning in multilayer spiking neural networks [J]. Neural Computation, 2018, 30(6): 1514-1541.
[14] Javanshir A, Nguyen T, Mahmud M, et al. Advancements in algorithms and neuromorphic hardware for spiking neural networks [J]. Neural Computation, 2022, 34(6): 1289-1328.
[15] Wang Y, Shi K, Lu C, et al. Spatial-temporal self-attention for asynchronous spiking neural networks [C]//32nd International Joint Conference on Artificial Intelligence, 2023: 3085-3093.
[16] 张铁林, 李澄宇, 王刚, 等. 适合类脑脉冲神经网络的应用任务范式分析与展望[J]. 电子与信息学报, 2023, 45(8): 2675-2688. Zhang T L, Li C Y, Wang G, et al. Research advances and new paradigms for biologyinspired spiking neural networks [J]. Journal of Electronics & Information Technology, 2023, 45(8): 2675-2688. (in Chinese)
[17] Fang W, Chen Y, Ding J, et al. SpikingJelly: an open-source machine learning infrastructure platform for spike-based intelligence [J]. Science Advances, 2023, 9(40): eadi1480.
[18] Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[19] Xiao H, Rasul K, Vollgraf R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms [DB/OL]. 2017[2024-08-08]. https://arxiv.org/abs/1708.07747.
[20] Krizhevsky A. Learning multiple layers of features from tiny images [R]. Toronto: University of Toronto, 2009.
[21] 姜晓勇, 李忠义, 黄朗月, 等. 神经网络剪枝技术研究综述[J]. 应用科学学报, 2022, 40(5): 838-849. Jiang X Y, Li Z Y, Huang L Y, et al. Review of neural network pruning techniques [J]. Journal of Applied Sciences, 2022, 40(5): 838-849. (in Chinese)
[22] Lyu C Z, Xu J H, Zheng X Q. Spiking convolutional neural networks for text classification [C]//11th International Conference on Learning Representations, 2023: 1-17.
[23] Miikkulainen R, Liang J, Meyerson E, et al. Evolving deep neural networks [DB/OL]. 2017[2024-08-08]. https://arxiv.org/abs/1703.00548.
[24] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting [J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[25] Kinouchi O, Pazzini R, Copelli M. Mechanisms of self-organized quasicriticality in neuronal network models [J]. Frontiers in Physics, 2020, 8: 583213.
[26] Szeliski R. Computer vision: algorithms and applications [M]. New York: Springer-Verlag, 2010: 1-28.
[27] Viet N H C. Spike-based hybrid learning method for image recognition task [C]//Proceedings of the 20249th International Conference on Intelligent Information Technology, 2024: 124-133.
[28] 岳斌. 基于脉冲神经网络的手写数字识别系统的设计与实现[D]. 北京: 中国科学院大学, 2022.
