A network model SNSMRS (SMOTEENN-mixed residual networks-SVM network) based on hybrid sampling, squeeze and excitation (SE) module, improved deep residual network and support vector machines (SVM) is proposed to address the problem of uneven class distribution of unbalanced data sets in traditional structured multiclassification algorithms, which leads to increased classification difficulty. Firstly, the data distribution is improved by synthesizing minority oversampling and editing nearest neighbors technique. Then the features are extracted by combining SE module and a deep residual network, improved with batch normalization and group normalization. Finally, the network model uses support vector machine (SVM) to output the classification results. The SE module enhances the model’s feature differentiation ability and robustness. The improvements to the ResNet, through fusion normalization, mitigate issues such as gradient vanishing and accuracy degradation, and ensure stability and accuracy regardless of batch_size. Additionally, SVM enhances the classification accuracy by effectively utilizing feature vectors in space to classify and extract features. Comparison and ablation experiments are conducted on seven unbalanced public datasets of various sizes and domains. The experimental results show that the proposed model, SNSMRS, not only outperforms other deep learning models, but also increases the values of Macro-F1 and G-mean by approximately 3% and 4%, respectively, compared with the original ResNet. Macro-F1 and G-mean values of SNSMRS exceed 95% on four of the datasets, demonstrating its superior performance.
[1] Sun A X, Lim E P, Liu Y. On strategies for imbalanced text classification using SVM: a comparative study [J]. Decision Support Systems, 2009, 48(1): 191-201.
[2] Shilaskar S, Ghatol A. Diagnosis system for imbalanced multi-minority medical dataset [J]. Soft Computing, 2019, 23(13): 4789-4799.
[3] Benchaji I, Douzi S, Elouahidi B. Using genetic algorithm to improve classification of imbalanced datasets for credit card fraud detection [C]//2018 2nd Cyber Security in Networking Conference (CSNet), 2018: 1-5.
[4] Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: synthetic minority over-sampling technique [J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[5] Ha J, Lee J S. A new under-sampling method using genetic algorithm for imbalanced data classification [C]//Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication, 2016: 1-6.
[6] Zhang Y, Liu Q. On IoT intrusion detection based on data augmentation for enhancing learning on unbalanced samples [J]. Future Generation Computer Systems, 2022, 133: 213-227.
[7] Batista G E, Bazzan A L C, Monard M C. Balancing training data for automated annotation of keywords: a case study [C]//II Brazilian Workshop on Bioinformatics, 2003: 10-18.
[8] Li X, Zhang L. Unbalanced data processing using deep sparse learning technique [J]. Future Generation Computer Systems, 2021, 125: 480-484.
[9] Sun Y M, Kamel M S, Wong A K C, et al. Cost-sensitive boosting for classification of imbalanced data [J]. Pattern Recognition, 2007, 40(12): 3358-3378.
[10] Dufrenois F. A one-class kernel fisher criterion for outlier detection [J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(5): 982-994.
[11] Wang Y X. An ensemble learning imbalanced data classification method based on sample combination optimization [J]. Journal of Physics: Conference Series, 2019, 1284(1): 12035.
[12] Joshi M V, Kumar V, Agarwal R C. Evaluating boosting algorithms to classify rare classes: comparison and improvements [C]//2001 IEEE International Conference on Data Mining, 2001: 257-264.
[13] Scholkopf B, Mika S, Burges C J C, et al. Input space versus feature space in kernel-based methods [J]. IEEE Transactions on Neural Networks, 1999, 10(5): 1000-1017.
[14] Deng X H, Xu Y B, Chen L C, et al. Dynamic clustering method for imbalanced learning based on AdaBoost [J]. The Journal of Supercomputing, 2020, 76(12): 9716-9738.
[15] Wang S T, Liu S Y, Zhang J K, et al. A new method of diesel fuel brands identification: SMOTE oversampling combined with XGBoost ensemble learning [J]. Fuel, 2020, 282: 118848.
[16] Pandey M, Xu Z R, Sholle E, et al. Extraction of radiographic findings from unstructured thoracoabdominal computed tomography reports using convolutional neural network based natural language processing [J]. PLoS One, 2020, 15(7): e0236827.
[17] 葛晓伟, 李凯霞, 程铭. 基于CNN-SVM的护理不良事件文本分类研究[J]. 计算机工程与科学, 2020, 42(1): 161-166. Ge X W, Li K X, Cheng M. Text classification of nursing adverse events based on CNN-SVM [J]. Computer Engineering & Science, 2020, 42(1): 161-166. (in Chinese)
[18] Ahlawat S, Choudhary A. Hybrid CNN-SVM classifier for handwritten digit recognition [J]. Procedia Computer Science, 2020, 167: 2554-2560.
[19] Zhang L G, Sheng Z, Li Y B, et al. Retraction note: image object detection and semantic segmentation based on convolutional neural network [J]. Neural Computing and Applications, 2023, 35(4): 3579.
[20] 暴雨轩, 芦天亮, 杜彦辉, 等. 基于i_ResNet34模型和数据增强的深度伪造视频检测方法[J]. 计算机科学, 2021, 48(7): 77-85. Bao Y X, Lu T L, Du Y H, et al. Deepfake videos detection method based on i_ResNet34 model and data augmentation [J]. Computer Science, 2021, 48(7): 77-85. (in Chinese)
[21] 万鹏, 赵竣威, 朱明, 等. 基于改进ResNet50模型的大宗淡水鱼种类识别方法[J]. 农业工程学报, 2021, 37(12): 159-168. Wan P, Zhao J W, Zhu M, et al. Freshwater fish species identification method based on improved ResNet50 model [J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(12): 159-168. (in Chinese)
[22] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[23] Siddan G, Palraj P. Foetal neurodegenerative disease classification using improved deep ResNet classification based VGG-19 feature extraction network [J]. Multimedia Tools and Applications, 2022, 81(2): 2393-2408.
[24] 吕振虎, 许新征, 张芳艳. 基于挤压激励的轻量化注意力机制模块[J]. 计算机应用, 2022, 42(8): 2353-2360. Lyu Z H, Xu X Z, Zhang F Y. Lightweight attention mechanism module based on squeeze and excitation [J]. Journal of Computer Applications, 2022, 42(8): 2353-2360. (in Chinese)
[25] Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[26] Xu X L, Li W S, Duan Q L. Transfer learning and SE-ResNet152 networks-based for smallscale unbalanced fish species identification [J]. Computers and Electronics in Agriculture, 2021, 180: 105878.
[27] Batista G E A P A, Prati R C, Monard M C. A study of the behavior of several methods for balancing machine learning training data [J]. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 20-29.
[28] Hawla N V, Bowyer K W, Hall L O, et al. Smote: synthetic minority over-sampling technique [J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[29] Wu Y X, He K M. Group normalization [M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[30] Hendrycks D, Gimpel K. Gaussian error linear units (GELUs) [DB/OL]. 2016[2023-05-06]. http://arxiv.org/abs/1606.08415v5.
[31] Arik S ö, Pfister T. TabNet: attentive interpretable tabular learning [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(8): 6679-6687.
[32] Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks [J]. Information Processing & Management, 2009, 45(4): 427-437.
[33] Kubat M, Holte R, Matwin S. Learning when negative examples abound [C]//European Conference on Machine Learning. Springer, 1997: 146-153.
[34] 李蒙蒙, 刘艺, 李庚松, 等. 不平衡多分类算法综述[J]. 计算机应用, 2022, 42(11): 3307-3321. Li M M, Liu Y, Li G S, et al. Survey on imbalanced multi-class classification algorithms [J]. Journal of Computer Applications, 2022, 42(11): 3307-3321. (in Chinese)
