基于长短时间记忆网络与集成学习的多通道脑电情感识别
Multi-Channel EEG Emotion Recognition Based on Long-Short-Term Memory Network and Ensemble Learning
DOI: 10.12677/CSA.2022.1210228, PDF,   
作者: 徐金阳, 陈 斌, 仇 苇：扬州大学信息工程学院，江苏 扬州
关键词: 情感识别；脑电信号；长短时记忆网络；集成学习；Emotion Recognition； EEG； LSTM； Ensemble Learning
摘要: 情感识别是人机交互中一个比较关键的问题，脑电作为人生理号中重要的组成部分，是识别人情感的关键因素。由于大脑中复杂的神经元活动会导致信号的非平稳行为，利用脑电进行情绪识别是一项具有挑战性的研究，并且多通道的分析是处理脑电信号时需要考虑的重要问题。本文提出了一种基于长短时间记忆网络与集成学习的多通道脑电情感识别模型，通过研究选取对情绪反馈响应较大的脑电通道，将不同通道的脑电数据分别放入长短时记忆网络中进行训练，再将各通道的训练模型通过集成学习策略进行优化，从而能整合各通道的信息进行情感分类。此研究在DEAP基准数据集上进行了情绪识别实验，我们在效价和唤醒两个情感维度的的情感识别结果均有明显的提升，准确率分别达到了87.6%，90.52%，验证了多通道集成方法的有效性。
Abstract: Emotion recognition is a key issue in human-computer interaction, and EEG, as an important part of human physiological signals, is also a key factor in recognizing human emotions. Emotion recognition using EEG is a challenging study due to the complex neuronal activity in the brain leading to non-stationary behavior of signals, and multi-channel analysis is an important issue to consider when processing EEG signals. This paper proposes a multi-channel EEG emotion recognition model based on long-short-term memory network and ensemble learning. Through research, the EEG channel that responds more to emotional feedback is selected, and the EEG data of different channels are put into the long-term memory network for training. Then, the training model of each channel is optimized through the integrated learning strategy, so that the information of each channel can be integrated for emotion classification. This study conducted emotion recognition experiments on the DEAP benchmark dataset. Our emotion recognition results in the two emotional dimensions of valence and arousal have been significantly improved, and the accuracy rates have reached 87.6% and 90.52%, which validates the effectiveness of the ensemble multi-channel approach.
文章引用：徐金阳, 陈斌, 仇苇. 基于长短时间记忆网络与集成学习的多通道脑电情感识别[J]. 计算机科学与应用, 2022, 12(10): 2237-2248. https://doi.org/10.12677/CSA.2022.1210228

参考文献

[1] Picard, R.W. (1995) Affective Computing. MIT Media Laboratory, Perceptual Computing, Cambridge.
[2] Zarshenas, H. and Auais, M.A. (2015) Application of BCI Systems in Neurorehabilitation: A Scoping Review Disability and Reha-bilitation. Assistive Technology, 10, 355-364. [Google Scholar] [CrossRef] [PubMed]
[3] Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., Lecompte, W.A., Pitcairn, T., Ricci-Bitti, P.E., et al. (1987) Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion. Journal of Personality and Social Psychology, 53, 712-717. [Google Scholar] [CrossRef] 
[4] Russell, J.A. (1980) A Circumplex Model of Affect. Journal of Personality and Social Psychology, 39, 1161-1178. [Google Scholar] [CrossRef] 
[5] Paltoglou, G. and Thelwall, M. (2012) Seeing Stars of Valence and Arousal in Blog Posts. IEEE Transactions on Affective Computing, 4, 116-123. [Google Scholar] [CrossRef] 
[6] Brittona, J.C., Phan, K.L., Taylor, S.F., Welsh, R.C., Berridge, K.C. and Liberzon, I. (2006) Neural Correlates of Social and Nonsocial Emotions: An fMRI Study. Neuroimage, 31, 397-409. [Google Scholar] [CrossRef] [PubMed]
[7] Lotfia, E. and Akbarzadeh-T, M.-R. (2014) Practical Emo-tional Neural Networks. Neural Networks, 59, 61-72. [Google Scholar] [CrossRef] [PubMed]
[8] Aboalayon, K.A.I., Faezipour, M., Almuhammadi, W.S. and Moslehpour, S. (2016) Sleep Stage Classification Using EEG Signal Analysis: A Comprehensive Survey and New In-vestigation. Entropy, 18, 272. [Google Scholar] [CrossRef] 
[9] Acharya, U., Sree, S.V., Swapna, G., Martis, R.J. and Suri, J.S. (2013) Automated EEG Analysis of Epilepsy: A Review. Knowledge-Based Systems, 45, 147-165. [Google Scholar] [CrossRef] 
[10] Engemann, D., Raimondo, F., King, J.-R., Rohaut, B., Louppe, G., Faugeras, F., Annen, J., Cassol, H., Gosseries, O., Slezak, D.F., et al. (2018) Robust EEG-Based Cross-Site and Cross-Protocol Classification of States of Consciousness. Brain, 141, 3179-3192. [Google Scholar] [CrossRef] [PubMed]
[11] Liu, N.-H., Chiang, C.-Y. and Chu, H.-C. (2013) Recognizing the De-gree of Human Attention Using EEG Signals from Mobile Sensors. Sensors, 13, 10273-10286. [Google Scholar] [CrossRef] [PubMed]
[12] So, W.K.Y., Wong, S., Mak, J.N. and Chan, R.H.M. (2017) An Evalua-tion of Mental Workload with Frontal EEG. PLOS ONE, 12, e0174949. [Google Scholar] [CrossRef] [PubMed]
[13] Thejaswini, S., Ravikumar, K.M., Jhenkar, L., Aditya, N. and Abhay, K.K. (2019) Analysis of EEG Based Emotion Detection of DEAP and SEED-IV Databases Using SVM. Inter-national Journal of Recent Technology and Engineering, 8, 207-211.
[14] Müller-Putz, G., Peicha, L. and Ofner, P. (2017) Movement Decoding from EEG: Target or Direction. Proceedings of the 7th Graz Brain-Computer Interface Conference, Graz, 18-22 September 2017. [Google Scholar] [CrossRef] 
[15] Jenke, R., Peer, A. and Buss, M. (2014) Feature Extraction and Selection for Emotion Recognition from EEG. IEEE Transactions on Affective Computing, 5, 327-339. [Google Scholar] [CrossRef] 
[16] Kim, M.-K., Kim, M., Oh, E. and Kim, S.-P. (2013) A Review on the Computational Methods for Emotional State Estimation from the Human EEG. Computational and Mathematical Methods in Medicine, 2013, Article ID: 573734. [Google Scholar] [CrossRef] [PubMed]
[17] Shi, L.-C., Jiao, Y.-Y. and Lu, B.-L. (2013) Differential Entropy Feature for EEG-Based Vigilance Estimation. The 2013 35th Annual International Conference of the IEEE Engineering in Medi-cine and Biology Society (EMBC), Osaka, 3-7 July 2013, 6627-6630.
[18] Duan, R.-N., Zhu, J.-Y. and Lu, B.-L. (2013) Differential Entropy Feature for EEG-Based Emotion Classification. The 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego, 6-8 November 2013, 81-84. [Google Scholar] [CrossRef] 
[19] Zhang, N., Zheng, W.L., Liu, W. and Lu, B.L. (2016) Continuous Vigilance Estimation Using LSTM Neural Networks. In: Hirose, A., et al., Eds., International Conference on Neural In-formation Processing, Springer, Berlin, 530-537. [Google Scholar] [CrossRef] 
[20] Hochreiter, S. and Schmidhuber, J. (1997) Long Short-Term Memory. Neural Computation, 9, 1735-1780. [Google Scholar] [CrossRef] [PubMed]
[21] Alhagry, S., Aly, A. and El-Khoribi, R. (2017) Emotion Recogni-tion Based on EEG Using LSTM Recurrent Neural Network. International Journal of Advanced Computer Science and Applications, 8, 8-11. [Google Scholar] [CrossRef] 
[22] Tang, H., Liu, W., Zheng, W.L. and Lu, B.L. (2017) Multi-modal Emotion Recognition Using Deep Neural Networks. In: Liu, D.R., et al., Eds., International Conference on Neu-ral Information Processing, Springer, Berlin, 811-819. [Google Scholar] [CrossRef] 
[23] Li, X., Song, D., Zhang, P., Yu, G.L., Hou, Y.X. and Hu, B. (2016) Emotion Recognition from Multi-Channel EEG Data through Convolutional Recurrent Neural Network. 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Shenzhen, 15-18 December 2016, 352-359. [Google Scholar] [CrossRef] 
[24] Chen, W., Chen, L.-L., Song, Z.-Z., Lou, X.-G. and Li, D.-D. (2020) EEG-Based Emotion Recognition Using Simple Recurrent Units Network and Ensemble Learning. Biomedical Signal Processing and Control, 58, Article ID: 101756. [Google Scholar] [CrossRef] 
[25] Salzman, C.D. and Fusi, S. (2010) Emotion, Cognition, and Men-tal State Representation in Amygdala and Prefrontal Cortex. Annual Review of Neuroscience, 33, 173-202. [Google Scholar] [CrossRef] [PubMed]
[26] Zhao, G., Zhang, Y. and Ge, Y. (2018) Frontal EEG Asymmetry and Middle Line Power Difference in Discrete Emotions. Frontiers in Behavioral Neuroscience, 12, Article No. 225. [Google Scholar] [CrossRef] [PubMed]
[27] Cimtay, Y. and Ekmekcioglu, E. (2020) Investigating the Use of Pretrained Convolutional Neural Network on Cross- Subject and Cross-Dataset EEG Emotion Recognition. Sen-sors (Basel), 20, Article No. 2034. [Google Scholar] [CrossRef] [PubMed]
[28] Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A. and Patras, I. (2011) DEAP: A Database for Emotion Analysis; Using Physiological Signals. IEEE Transactions on Affective Computing, 3, 18-31. [Google Scholar] [CrossRef] 
[29] Wang, F., Zhong, Sh., Peng, J., Jiang, J. and Liu, Y. (2018) Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks. In: Schoeffmann, K., et al., Eds., Multi-Media Modeling, MMM 2018, Lecture Notes in Computer Science, Vol. 10705, Springer, Cham, 82-93. [Google Scholar] [CrossRef] 
[30] Atkinson, J. and Campos, D. (2016) Improving BCI-Based Emotion Recognition by Combining EEG Feature Selection and Kernel Classifiers. Expert Systems with Applications, 47, 35-41. [Google Scholar] [CrossRef] 
[31] Yoon, H.J. and Chung, S.Y. (2013) EEG-Based Emotion Estimation Using Bayesian Weighted-Log-Posterior Function and Perceptron Convergence Algorithm. Computers in Bi-ology and Medicine, 43, 2230-2237. [Google Scholar] [CrossRef] [PubMed]
[32] Mert, A. and Akan, A. (2018) Emotion Recognition from EEG Signals by Using Multivariate Empirical Mode Decomposition. Pattern Analysis and Applications, 21, 81-89. [Google Scholar] [CrossRef]