基于代价敏感卷积神经网络的集成分类算法

doi:10.3969/j.issn.0255-8297.2022.01.007

Abstract

Abstract: Aiming at the problem of low recognition rate of a few types of samples in unbalanced data sets, a classification algorithm based on cost sensitive convolutional neural network and AdaBoost (AdaBoost-CSCNN) was proposed. The cost sensitive convolutional neural network (CSCNN) is constructed by coordinating the cross entropy loss function of convolutional neural network (CNN) with a specific cost sensitive index. In training process, cost weighting mechanism is used to reduce the loss degree of key feature attributes of a few samples and realize the classification effect of a single CSCNN as a base classifier in AdaBoost. To verify the effectiveness of the algorithm, we carried out experiments on 9 data sets with different imbalance rates. Experimental performances, including Accuracy, Recall, F1-score and AUC, show that the AdaBoost-CSCNN algorithm has a good display for unbalanced data set classification.

Key words: cost sensitivity, convolutional neural network (CNN), AdaBoost, cost weighting mechanism, unbalanced data set

CLC Number:

TP391

ZHOU Chuanhua, XU Wenqian, ZHU Junjie. Ensemble Classification Algorithm Based on Cost Sensitive Convolutional Neural Networks[J]. Journal of Applied Sciences, 2022, 40(1): 69-79.

References

[1] Chawla N V, Japkowicz N, Kotcz A. Special issue on learning from imbalanced data sets[J]. ACM SIGKDD Explorations Newsletter, 2004, 6(1):1-6.
[2] Mazurowski M A, Habas P A, Zurada J M, et al. Training neural network classifiers for medical decision making:the effects of imbalanced datasets on classification performance[J]. Neural Networks, 2008, 21(2/3):427-436.
[3] 于巧, 姜淑娟, 张艳梅, 等. 分类不平衡对软件缺陷预测模型性能的影响研究[J]. 计算机学报, 2018, 41(4):809-824. Yu Q, Jiang S J, Zhang Y M, et al. The impact study of class imbalance on the performance of software defect prediction models[J]. Chinese Journal of Computers, 2018, 41(4):809-824. (in Chinese)
[4] Gamage S, Samarabandu J. Deep learning methods in network intrusion detection:a survey and an objective comparison[J]. Journal of Network and Computer Applications, 2020, 169(2):102767.
[5] 李艳霞, 柴毅, 胡友强, 等. 不平衡数据分类方法综述[J]. 控制与决策, 2019, 34(4):673-688. Li Y X, Chai Y, Hu Y Q, et al. Review of imbalanced data classification methods[J]. Control and Decision, 2019, 34(4):673-688. (in Chinese)
[6] Zheng W, Zhao H. Cost-sensitive hierarchical classification for imbalance classes[J]. Applied Intelligence, 2020, 50(1):1-11.
[7] Peng P, Zhang W J, Zhang Y, et al. Cost sensitive active learning using bidirectional gated recurrent neural networks for imbalanced fault diagnosis[J]. Neurocomputing, 2020, 407:232-245.
[8] 万建武, 杨明. 代价敏感学习方法综述[J]. 软件学报, 2020, 31(1):113-136. Wan J W, Yang M. Survey on cost-sensitive learning method[J]. Journal of Software, 2020, 31(1):113-136. (in Chinese)
[9] Yoav F, Robert E S. A decision-theoretic generalization of on-line learning and an application to Boosting[J]. Journal of Computer and System Sciences, 1997, 55(1):119-139.
[10] Chawla N V, Bowyer K W, Hall L O, et al. SMOTE:synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16:321-357.
[11] Seiffert C, Khoshgoftaar T M, Van H J, et al. RUSBoost:a hybrid approach to alleviating class imbalance[J]. IEEE Transactions on Systems, Man, and Cybernetics-Part A:Systems and Humans, 2009, 40(1):185-197.
[12] Fan W, Stolfo S J, Zhang J. AdaCost:misclassification cost-sensitive boosting[J]. Proc. International Conference on Machine Learning, 1999:97-105.
[13] 王忠震, 黄勃, 方志军, 等. 改进SMOTE的不平衡数据集成分类算法[J]. 计算机应用, 2019, 39(9):2591-2596. Wang Z Z, Huang B, Fang Z J, et al. Imporved SMOTE unbalanced data integration classification algorithm[J]. Journal of Computer Applications, 2019, 39(9):2591-2596. (in Chinese)
[14] Chen F, Cheng M, Tang B, et al. Pattern recognition of a sensitive feature set based on the orthogonal neighborhood preserving embedding and Adaboost_SVM algorithm for rolling bearing early fault diagnosis[J]. Measurement Science and Technology, 2020, 31(10):105007.
[15] 闻佳, 王宏君, 邓佳, 等. 基于深度学习的异常事件检测[J]. 电子学报, 2020, 48(2):308-313. Wen J, Wang H J, Deng J, et al. Abnormal event detection based on deep learning[J]. Acta Electronica Sinica, 2020, 48(2):308-313. (in Chinese)
[16] Frazao X, Alexandre L A. Weighted convolutional neural network ensemble[C]//Iberoamerican Congress on Pattern Recognition. Cham:Springer, 2014:674-681.
[17] Taherkhani A, Cosma G, Mcginnity T M. AdaBoost-CNN:an adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning[J]. Neurocomputing, 2020, 404:351-366.
[18] 董勋, 郭亮, 高宏力, 等. 代价敏感卷积神经网络:一种机械故障数据不平衡分类方法[J]. 仪器仪表学报, 2019, 40(12):205-213. Dong X, Guo L, Gao H L, et al. Cost sensitive convolutional neural network:a classification method for imbalanced data of mechanical fault[J]. Chinese Journal of Scientific Instrument, 2019, 40(12):205-213. (in Chinese)
[19] Donovan F, Talayeh R. A cost-sensitive convolution neural network learning for control chart pattern recognition[J]. Expert Systems with Applications, 2020, 150(C):113275.
[20] Tan C, Sun F, Kong T, et al. A survey on deep transfer learning[C]//International Conference on Artificial Neural Networks. Cham:Springer, 2018:270-279.

Ensemble Classification Algorithm Based on Cost Sensitive Convolutional Neural Networks

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	ZHU Li, YANG Qing, WU Tao, LI Chen, LI Ming. Emotional Analysis of Brain Waves Based on CNN and Bi-LSTM [J]. Journal of Applied Sciences, 2022, 40(1): 1-12.
[2]	ZHENG Changliang, PANG Ming. Spatial-Temporal Weight Attitude Motion Feature Extraction Algorithm Using Convolutional Neural Network [J]. Journal of Applied Sciences, 2021, 39(4): 594-604.
[3]	LI Wenju, HE Maoxian, ZHANG Yaoxing, CHEN Huiling, LI Peigang. Crack Detection of Track Slab Based on Convolutional Neural Network and Voting Mechanism [J]. Journal of Applied Sciences, 2021, 39(4): 627-640.
[4]	HAO Yan, SHI Huiyu, HUO Shoujun, HAN Dan, CAO Rui. Emotion Classification Based on EEG Deep Learning [J]. Journal of Applied Sciences, 2021, 39(3): 347-346.
[5]	CUI Pengtao, ZHANG Qian, LIU Jinghuai, ZHOU Chao, WANG Bin, SI Wen. FSCD-CNN Based Fast Mode Decision Algorithm for Intra-prediction in Depth Map Coding [J]. Journal of Applied Sciences, 2021, 39(3): 433-432.
[6]	LI Lei, ZHANG Qingmiao, ZHAO Junhui, NIE Yiwen. Short-Term Traffic Flow Prediction Method of Different Periods Based on Improved CNN-LSTM [J]. Journal of Applied Sciences, 2021, 39(2): 185-198.
[7]	LIU Wei, ZHANG Wanling, XIANG Shijun. Face Anti-spoofing Based on LBP, Multilayer DCT and CNN [J]. Journal of Applied Sciences, 2019, 37(5): 609-617.
[8]	WANG Canjun, LIAO Xin, CHEN Jiaxin, QIN Zheng, LIU Xuchong. Research on Facial Modification Detection Algorithm Based on Convolutional Neural Network [J]. Journal of Applied Sciences, 2019, 37(5): 618-630.
[9]	ZHAO Yunshan, DUAN Youxiang. Convolutional Neural Networks Text Classification Model Based on Attention Mechanism [J]. Journal of Applied Sciences, 2019, 37(4): 541-550.
[10]	DONG Wei, WANG Jian-jun. Contrast Enhancement Forensics Based on Modified Convolutional Neural Network [J]. Journal of Applied Sciences, 2017, 35(6): 745-753.
[11]	YAN Chao, WU Yue, YUE Xiao-dong. AdaBoost Algorithm with Classification Belief [J]. Journal of Applied Sciences, 2015, 33(2): 203-214.
[12]	HUANG Jie, NI Peng-yu. Pornographic-Image Filtering Based on Body Parts [J]. Journal of Applied Sciences, 2014, 32(4): 416-422.
[13]	XIAO Jian1;3, LIU Bo1, MEI Chen1, ZHU Min2, YANG Jun1, LIU Lei-bo2, WEI Shao-jun. Dynamical Coarse-Grained Partially Reconfigurable Face Detection [J]. Journal of Applied Sciences, 2012, 30(3): 299-305.
[14]	ZHANG Pin-zheng, WANG Jian-hong. Vehicle License Plate Location Based on Machine Learning [J]. Journal of Applied Sciences, 2011, 29(2): 147-152.
[15]	MIAO Zhi-min;HU Gu-yu;DING Li;ZHAO Lu-wen;PAN Zhi-song. Support Vector Date Description Implemented in Class-Imbalance Learning [J]. Journal of Applied Sciences, 2008, 26(1): 79-79 .