In cross-subject emotion recognition, there are significant differences in the distribution of electroencephalogram (EEG) samples among different subjects, and domain adaptation is commonly used to alleviate the differences. However, the differences in the EEG distribution across affective subdomains are ignored by global adaptation, which reduces the distinguishability of emotional features. Besides, EEG contains a number of electrodes, and subjects only reach the prospective emotion during part of stimuli. Learning the complex spatial information between channels and emphasizing critical EEG frames is essential. Hence, we propose a subdomain adaptation and spatial-temporal learning network for EEG-based emotion recognition. In the subdomain adaptation module, the difference loss in subdomains is reduced by minimizing intra-class differences and maximizing inter-class differences. A spatial-temporal feature extractor captures spatial correlations and temporal contexts, extracting discriminative emotional features. Subject-independent experiments conducted on the public DEAP dataset demonstrate the superior performance of the proposed method, achieving classification accuracies of 0.688 0 for arousal and 0.696 8 for valence, respectively.
TANG Yiheng, WANG Yongxiong, WANG Zhe, ZHANG Xiaoli
. Emotion Recognition of EEG Using Subdomain Adaptation and Spatial-Temporal Learning[J]. Journal of Applied Sciences, 2024
, 42(6)
: 1016
-1026
.
DOI: 10.3969/j.issn.0255-8297.2024.06.010
[1] 陈景霞, 郝为, 张鹏伟, 等. 基于混合神经网络的脑电时空特征情感分类[J]. 软件学报, 2021, 32(12): 3869-3883. Chen J X, Hao W, Zhang P W, et al. Emotion classification of spatiotemporal EEG features using hybrid neural networks [J]. Journal of Software, 2021, 32(12): 3869-3883. (in Chinese)
[2] Liu S Q, Wang X, Zhao L, et al. 3DCANN: a spatio-temporal convolution attention neural network for EEG emotion recognition [J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(11): 5321-5331.
[3] Wang Z, Wang Y X, Zhang J P, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition [J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 2507212.
[4] 蔡梓良, 郭苗苗, 杨新生, 等. 基于最大分类器差异域对抗方法的跨被试脑电情绪识别研究[J]. 生物医学工程学杂志, 2021, 38(3): 455-462. Cai Z L, Guo M M, Yang X S, et al. Cross-subject electroencephalogram emotion recognition based on maximum classifier discrepancy [J]. Journal of Biomedical Engineering, 2021, 38(3): 455-462. (in Chinese)
[5] Li Z Y, Zhao X W, Yang Y, et al. HVFM: an emotion classification model based on horizontal and vertical flow domain-adaptive [C]//2022 IEEE International Conference on Mechatronics and Automation (ICMA), 2022: 455-460.
[6] Tian Z K, Li D H, Yang Y, et al. A novel domain adversarial networks based on 3D-LSTM and local domain discriminator for hearing-impaired emotion recognition [J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(1): 363-373.
[7] Zhu Y C, Zhuang F Z, Wang J D, et al. Deep subdomain adaptation network for image classification [J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(4): 1713-1722.
[8] Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module [C]//European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[9] Koelstra S, Muhl C, Soleymani M, et al. DEAP: a database for emotion analysis; using physiological signals [J]. IEEE Transactions on Affective Computing, 2012, 3(1): 18-31.
[10] Jatupaiboon N, Pan-Ngum S, Israsena P. Real-time EEG-based happiness detection system [J]. The Scientific World Journal, 2013: 618649.
[11] Alhagry S, Aly A, Reda A. Emotion recognition based on EEG using LSTM recurrent neural network [J]. International Journal of Advanced Computer Science and Applications, 2017, 8(10): 355-358.
[12] Zheng W L, Zhu J Y, Peng Y, et al. EEG-based emotion classification using deep belief networks [C]//2014 IEEE International Conference on Multimedia and Expo (ICME), 2014: 6890166.
[13] Gao Z K, Yuan T, Zhou X J, et al. A deep learning method for improving the classification accuracy of SSMVEP-based BCI [J]. IEEE Transactions on Circuits and Systems II: Express Briefs, 2020, 67(12): 3447-3451.
[14] Gao Z K, Wang X M, Yang Y X, et al. A channel-fused dense convolutional network for EEGbased emotion recognition [J]. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(4): 945-954.
[15] Yin Z, Liu L, Chen J N, et al. Locally robust EEG feature selection for individual-independent emotion recognition [J]. Expert Systems with Applications, 2020, 162: 113768.
[16] Liang Z, Zhou R S, Zhang L, et al. EEGFuseNet: hybrid unsupervised deep feature characterization and fusion for high-dimensional EEG with an application to emotion recognition [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1913-1925.
[17] He Z P, Zhong Y S, Pan J H. Joint temporal convolutional networks and adversarial discriminative domain adaptation for EEG-based cross-subject emotion recognition [C]//2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022: 3214-3218.
[18] Wang Z, Wang Y X, Hu C F, et al. Transformers for EEG-based emotion recognition: a hierarchical spatial information learning model [J]. IEEE Sensors Journal, 2022, 22(5): 4359- 4368.
[19] Liu S Q, Wang Z Y, An Y L, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network [J]. Knowledge-Based Systems, 2023, 265: 110372.
