基于CNN和Bi-LSTM的脑电波情感分析

doi:10.3969/j.issn.0255-8297.2022.01.001

Abstract

Abstract: Aiming at the problem that most emotion recognition methods rely on manual feature extraction, a hybrid model based on convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) network is proposed. Firstly, one-dimensional data is converted into two-dimensional data, and spatial features are extracted by CNN. Then the one-dimensional data is input into Bi-LSTM to obtain temporal features. Finally, the fused spatial and temporal features are input into Softmax classifier to obtain final classification results. Experimental results on DEAP dataset show that CNN and BiLSTM hybrid model has good classification performance, and the accuracy in potency and arousal reaches 88.55% and 89.07%, respectively, proving the proposed model is a feasible and affective EEG emotion classification model.

Key words: electroencephalogram, emotion classification, convolutional neural network (CNN), bidirectional long short-term memory (Bi-LSTM) network, deep learning

CLC Number:

ZHU Li, YANG Qing, WU Tao, LI Chen, LI Ming. Emotional Analysis of Brain Waves Based on CNN and Bi-LSTM[J]. Journal of Applied Sciences, 2022, 40(1): 1-12.

References

[1] Ahmadlou M, Adeli H, Adeli A. Fractality analysis of frontal brain in major depressive disorder[J]. International Journal of Psychophysiology, 2012, 85(2):206-211.
[2] 李幼军, 黄佳进, 王海渊, 等. 基于SAE和LSTM RNN的多模态生理信号融合和情感识别研究[J]. 通信学报, 2017, 38(12):109-120. Li Y J, Huang J J, Wang H Y, et al. Study of emotion recognition based on fusion multimodal bio-signal with SAE and LSTM recurrent neural network[J]. Journal on Communications, 2017, 38(12):109-120. (in Chinese)
[3] Krisnandhika B, Faqih A, Pumamasari P D, et al. Emotion recognition system based on EEG signals using relative wavelet energy features and a modified radial basis function neural networks[C]//Proceedings of 2017 International Conference on Consumer Electronics and Devices (ICCED). London, UK:IEEE, 2017:50-54.
[4] Abdelaal M A, Alsway A A, Hefny H A. EEG-based emotion recognition using a wrapperbased feature selection method[C]//International Conference on Advanced Intelligent Systems and Informatics. Cairo, Egypt:AISI, 2018, 639:247-256.
[5] 李昕, 蔡二娟, 田彦秀, 等. 一种改进脑电特征提取算法及其在情感识别中的应用[J]. 生物医学工程学杂志, 2017(4):510-517. Li X, Cai E J, Tian Y X, et al. An improved electroencephalogram feature extraction algorithm and its application in emotion recognition[J]. Journal of Biomedical Engineering, 2017(4):510-517. (in Chinese)
[6] 孙颖, 马江河, 张雪英. 结合非线性全局特征和谱特征的脑电情感识别[J]. 计算机工程与应用, 2018, 54(17):116-121. Sun Y, Ma J H, Zhang X Y. EEG emotion recognition based on nonlinear global features and spectral feature[J]. Computer Engineering and Applications, 2018, 54(17):116-121. (in Chinese)
[7] 阚威, 李云. 基于LSTM的脑电情绪识别模型[J]. 南京大学学报自然科学, 2019, 55(1):110-116. Kan W, Li Y. Emotion recognition from EEG signals by using LSTM recurrent neural networks[J]. Journal of Nanjing University Natural Science, 2019, 55(1):110-116. (in Chinese)
[8] Tripathi S, Acharya S, Sharma R D, et al. Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset[C]//Twenty-Ninth IAAI Conference, 2017:4746-4752.
[9] Li X, Song D, Zhang P, et al. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network[C]//2016 IEEE International Conference on Bioinformatics and Biomedicine, Shenzhen, 2016:352-359.
[10] Salama E S, Elkhoribi R A, Shoman M, et al. EEG-based emotion recognition using 3D convolutional neural networks[J]. International Journal of Advanced Computer Science and Applications, 2018, 9(8):329.
[11] Song T, Zheng W, Song P, et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2020:11(3):532-541.
[12] 陈景霞, 王丽艳, 贾小云, 等. 基于深度卷积神经网络的脑电信号情感识别[J]. 计算机工程与应用, 2019, 55(18):103-110. Chen J X, Wang L Y, Jia X Y, et al. EEG-based emotion recognition using deep convolutional neural network[J]. Computer Engineering and Applications, 2019, 55(18):103-110. (in Chinese)
[13] 郝琰, 石慧宇, 霍首君, 等. 基于脑电信号深度学习的情感分类[J]. 应用科学学报, 2021, 39(3):347-346. Hao Y, Shi H Y, Huo S J, et al. Emotion classification based on EEG deep learning[J]. Journal of Applied Sciences, 2021, 39(3):347-356. (in Chinese)
[14] 柳素红, 孙晓, 李春彬. 基于位置信息重建和时频域信息融合的脑电信号情感识别[J/OL]. 计算机工程:1-9. (2021-02-01)[2021-06-27]. https://doi.org/10.19678/j.issn.1000-3428.0060195. Liu S H, Sun X, Li C B. Emotion recognition of EEG signals based on location information reconstruction and time-frequency information fusion[J/OL]. Computer Engineering:1-9. (2021-02-01)[2021-06-27]. https://doi.org/10.19678/j.issn.1000-3428.0060195. (in Chinese)
[15] 连卫芳, 晁浩, 刘永利. 基于SDAE和RELM的EEG情感识别方法[J/OL]. 计算机工程:2021, 47(9):75-83. Lian W F, Chao H, Liu Y L. EEG emotion recognition method based on stacked denoising autoencoder and regularized extreme learning machine[J/OL]. Computer Engineering:2021, 47(9):75-83. (in Chinese)
[16] 蔡冬丽, 钟清华, 朱永升, 等. 三维输入卷积神经网络脑电信号情感识别[J]. 计算机工程与应用, 2021, 57(5):161-167. Cai D L, Zhong Q H, Zhu Y S, et al. EEG emotion recognition using convolutional neural network with 3D input[J]. Computer Engineering and Applications, 2021, 57(5):161-167. (in Chinese)
[17] Yang Y L, Wu Q F, Qiu M, et al. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network[C]//Proceedings of 2018 International Joint Conference on Neural Networks. Rio de Janeiro:IEEE, 2018:1-7.
[18] Jasper H H. The ten-twenty electrode system of the international federation[J]. Electroencephalography and Clinical Neurophysiology, 1958, 10:371-375.
[19] Koelstra S, Muhl C, Soleymani M, et al. DEAP:a database for emotion analysis using physiological signals[J]. IEEE Transactions on Affective Computing, 2012, 3(1):18-31.
[20] Ma X Z, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//Proceedings of Association for Computational Linguistics, Sroudsburg, 2016:1064-1074.
[21] Wang X W, Nie D, Lu B L. Emotional state classification from EEG data using machine learning approach[J]. Neurocomputing, 2014, 129:94-106.
[22] 章敏敏, 徐和平, 王晓洁, 等. 谷歌TensorFlow机器学习框架及应用[J]. 微型机与应用, 2017, 36(10):58-60. Zhang M M, Xu H P, Wang X J, et al. Application of Google TensorFlow machine learning framework[J]. Microcomputer & Its Applications, 2017, 36(10):58-60. (in Chinese)

Emotional Analysis of Brain Waves Based on CNN and Bi-LSTM

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	ZHOU Chuanhua, XU Wenqian, ZHU Junjie. Ensemble Classification Algorithm Based on Cost Sensitive Convolutional Neural Networks [J]. Journal of Applied Sciences, 2022, 40(1): 69-79.
[2]	LEI Qianhui, PAN Lili, SHAO Weizhi, HU Haipeng, HUANG Yao. Segmentation Model of COVID-19 Lesions Based on Triple Attention Mechanism [J]. Journal of Applied Sciences, 2022, 40(1): 105-115.
[3]	KANG Huixian, YI Biao, WU Hanzhou. Recent Advances in Text Steganography and Steganalysis [J]. Journal of Applied Sciences, 2021, 39(6): 923-938.
[4]	OU Qiaofeng, XIAO Jiabing, XIE Qunqun, XIONG Bangshu. Multi-target Detection and Recognition for Vehicle Inspection Images Based on Deep Learning [J]. Journal of Applied Sciences, 2021, 39(6): 939-951.
[5]	ZHENG Changliang, PANG Ming. Spatial-Temporal Weight Attitude Motion Feature Extraction Algorithm Using Convolutional Neural Network [J]. Journal of Applied Sciences, 2021, 39(4): 594-604.
[6]	LI Wenju, HE Maoxian, ZHANG Yaoxing, CHEN Huiling, LI Peigang. Crack Detection of Track Slab Based on Convolutional Neural Network and Voting Mechanism [J]. Journal of Applied Sciences, 2021, 39(4): 627-640.
[7]	GUO Yubo, LU Jun, DUAN Pengqi. Automatic Classification of Bamboo Flute Playing Skills Based on Deep Learning [J]. Journal of Applied Sciences, 2021, 39(4): 685-694.
[8]	HAO Yan, SHI Huiyu, HUO Shoujun, HAN Dan, CAO Rui. Emotion Classification Based on EEG Deep Learning [J]. Journal of Applied Sciences, 2021, 39(3): 347-346.
[9]	DU Chengze, DUAN Youxiang, SUN Qifeng. Seismic Fault Identification Method Based on ResUNet and Dense CRF Model [J]. Journal of Applied Sciences, 2021, 39(3): 367-366.
[10]	CUI Pengtao, ZHANG Qian, LIU Jinghuai, ZHOU Chao, WANG Bin, SI Wen. FSCD-CNN Based Fast Mode Decision Algorithm for Intra-prediction in Depth Map Coding [J]. Journal of Applied Sciences, 2021, 39(3): 433-432.
[11]	LI Lei, ZHANG Qingmiao, ZHAO Junhui, NIE Yiwen. Short-Term Traffic Flow Prediction Method of Different Periods Based on Improved CNN-LSTM [J]. Journal of Applied Sciences, 2021, 39(2): 185-198.
[12]	WANG Wanguo, MU Shiyou, LIU Yue, LIU Guangxiu, LANG Fenling. Research on Insulator Self Exploding Detection in UAV Inspection Based on Deep Learning [J]. Journal of Applied Sciences, 2021, 39(2): 222-231.
[13]	LIU Zhiyu, ZHANG Shufen, LIU Yang, LUO Changyin, LI Min. Data Augmentation Method Based on Image Gradient [J]. Journal of Applied Sciences, 2021, 39(2): 302-311.
[14]	LIU Wei, ZHANG Wanling, XIANG Shijun. Face Anti-spoofing Based on LBP, Multilayer DCT and CNN [J]. Journal of Applied Sciences, 2019, 37(5): 609-617.
[15]	WANG Canjun, LIAO Xin, CHEN Jiaxin, QIN Zheng, LIU Xuchong. Research on Facial Modification Detection Algorithm Based on Convolutional Neural Network [J]. Journal of Applied Sciences, 2019, 37(5): 618-630.