基于子领域适应和时空学习的脑电情感识别

doi:10.3969/j.issn.0255-8297.2024.06.010

摘要/Abstract

摘要： 在跨受试情感识别中，不同受试者的脑电样本分布存在显著差异，常采用域适应方法减小脑电信号的个体差异。然而，全局域适应忽略了不同情感类别子领域的脑电分布差异，降低了情感特征的可区分性。此外，脑电信号包含众多的电极通道，并且受试者也只在部分刺激期间达到预期情绪，因此通道间的复杂空间信息和脑电信号的关键帧学习也是亟待解决的难题。为解决上述问题，提出了一种基于子领域适应和时空学习的脑电情感识别网络。首先，利用模型中的子领域适应模块，通过最小化类内差异和最大化类间差异以减小子领域的差异损失；再利用时空特征提取器捕获空间关联性及时间上下文信息，提取判别性情感特征。最后，在DEAP数据集上进行了受试者独立实验，实验结果验证了所提方法的性能，唤醒和效价的分类准确度分别达0.688 0和0.696 8。

关键词: 情感识别, 脑电, 子领域适应, 时空学习, 深度学习

Abstract: In cross-subject emotion recognition, there are significant differences in the distribution of electroencephalogram (EEG) samples among different subjects, and domain adaptation is commonly used to alleviate the differences. However, the differences in the EEG distribution across affective subdomains are ignored by global adaptation, which reduces the distinguishability of emotional features. Besides, EEG contains a number of electrodes, and subjects only reach the prospective emotion during part of stimuli. Learning the complex spatial information between channels and emphasizing critical EEG frames is essential. Hence, we propose a subdomain adaptation and spatial-temporal learning network for EEG-based emotion recognition. In the subdomain adaptation module, the difference loss in subdomains is reduced by minimizing intra-class differences and maximizing inter-class differences. A spatial-temporal feature extractor captures spatial correlations and temporal contexts, extracting discriminative emotional features. Subject-independent experiments conducted on the public DEAP dataset demonstrate the superior performance of the proposed method, achieving classification accuracies of 0.688 0 for arousal and 0.696 8 for valence, respectively.

Key words: emotion recognition, electroencephalogram (EEG), subdomain adaptation, spatial-temporal learning, deep learning

中图分类号:

唐意恒, 王永雄, 王哲, 张晓理. 基于子领域适应和时空学习的脑电情感识别[J]. 应用科学学报, 2024, 42(6): 1016-1026.

TANG Yiheng, WANG Yongxiong, WANG Zhe, ZHANG Xiaoli. Emotion Recognition of EEG Using Subdomain Adaptation and Spatial-Temporal Learning[J]. Journal of Applied Sciences, 2024, 42(6): 1016-1026.

参考文献

[1] 陈景霞, 郝为, 张鹏伟, 等. 基于混合神经网络的脑电时空特征情感分类[J]. 软件学报, 2021, 32(12): 3869-3883. Chen J X, Hao W, Zhang P W, et al. Emotion classification of spatiotemporal EEG features using hybrid neural networks [J]. Journal of Software, 2021, 32(12): 3869-3883. (in Chinese)
[2] Liu S Q, Wang X, Zhao L, et al. 3DCANN: a spatio-temporal convolution attention neural network for EEG emotion recognition [J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(11): 5321-5331.
[3] Wang Z, Wang Y X, Zhang J P, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition [J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 2507212.
[4] 蔡梓良, 郭苗苗, 杨新生, 等. 基于最大分类器差异域对抗方法的跨被试脑电情绪识别研究[J]. 生物医学工程学杂志, 2021, 38(3): 455-462. Cai Z L, Guo M M, Yang X S, et al. Cross-subject electroencephalogram emotion recognition based on maximum classifier discrepancy [J]. Journal of Biomedical Engineering, 2021, 38(3): 455-462. (in Chinese)
[5] Li Z Y, Zhao X W, Yang Y, et al. HVFM: an emotion classification model based on horizontal and vertical flow domain-adaptive [C]//2022 IEEE International Conference on Mechatronics and Automation (ICMA), 2022: 455-460.
[6] Tian Z K, Li D H, Yang Y, et al. A novel domain adversarial networks based on 3D-LSTM and local domain discriminator for hearing-impaired emotion recognition [J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(1): 363-373.
[7] Zhu Y C, Zhuang F Z, Wang J D, et al. Deep subdomain adaptation network for image classification [J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(4): 1713-1722.
[8] Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module [C]//European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[9] Koelstra S, Muhl C, Soleymani M, et al. DEAP: a database for emotion analysis; using physiological signals [J]. IEEE Transactions on Affective Computing, 2012, 3(1): 18-31.
[10] Jatupaiboon N, Pan-Ngum S, Israsena P. Real-time EEG-based happiness detection system [J]. The Scientific World Journal, 2013: 618649.
[11] Alhagry S, Aly A, Reda A. Emotion recognition based on EEG using LSTM recurrent neural network [J]. International Journal of Advanced Computer Science and Applications, 2017, 8(10): 355-358.
[12] Zheng W L, Zhu J Y, Peng Y, et al. EEG-based emotion classification using deep belief networks [C]//2014 IEEE International Conference on Multimedia and Expo (ICME), 2014: 6890166.
[13] Gao Z K, Yuan T, Zhou X J, et al. A deep learning method for improving the classification accuracy of SSMVEP-based BCI [J]. IEEE Transactions on Circuits and Systems II: Express Briefs, 2020, 67(12): 3447-3451.
[14] Gao Z K, Wang X M, Yang Y X, et al. A channel-fused dense convolutional network for EEGbased emotion recognition [J]. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(4): 945-954.
[15] Yin Z, Liu L, Chen J N, et al. Locally robust EEG feature selection for individual-independent emotion recognition [J]. Expert Systems with Applications, 2020, 162: 113768.
[16] Liang Z, Zhou R S, Zhang L, et al. EEGFuseNet: hybrid unsupervised deep feature characterization and fusion for high-dimensional EEG with an application to emotion recognition [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1913-1925.
[17] He Z P, Zhong Y S, Pan J H. Joint temporal convolutional networks and adversarial discriminative domain adaptation for EEG-based cross-subject emotion recognition [C]//2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022: 3214-3218.
[18] Wang Z, Wang Y X, Hu C F, et al. Transformers for EEG-based emotion recognition: a hierarchical spatial information learning model [J]. IEEE Sensors Journal, 2022, 22(5): 4359- 4368.
[19] Liu S Q, Wang Z Y, An Y L, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network [J]. Knowledge-Based Systems, 2023, 265: 110372.