基于多任务学习的时空混合注意力脑电信号情绪识别

Spatio-Temporal Hybrid Attention-Based EEG Emotion Recognition via Multi-Task Learning

摘要: 针对传统脑电情绪识别方法中存在的特征表征单维化、时序依赖建模不充分及任务间关联性被忽视等局限，本文提出一种面向多任务学习的时空混合注意力脑电情绪识别模型(MT-STCNN)。该方法首先在特征层面融合微分熵(DE)与功率谱密度(PSD)特征，从信号复杂度与频域能量分布两个维度全面表征情绪状态；在网络架构中引入轻量级Transformer编码器，强化对长时序脑电信号的全局依赖建模能力；进一步构建多任务协同学习框架，联合优化效价与唤醒度两项情感维度识别任务，通过特征共享机制与跨任务注意力模块实现任务间的知识迁移与互补，提升模型的识别效率。在DEAP数据集上的实验结果显示，本文方法在效价与唤醒度识别上的平均准确率分别达到98.26%和98.67%，性能显著优于当前主流模型，充分验证了所提方法在脑电情绪识别任务中的有效性与先进性。

Abstract: Aiming at the issues of single feature modality, inadequate temporal dependency modeling, and neglected task correlations in traditional EEG-based emotion recognition, this paper proposes a spatio-temporal hybrid attention EEG emotion recognition method based on multi-task learning (MT-STCNN). At the feature level, the method integrates Differential Entropy (DE) and Power Spectral Density (PSD) features to comprehensively characterize emotional states from both information complexity and energy distribution perspectives. In the network architecture, a lightweight Transformer encoder is introduced to enhance global modeling capabilities for long-term temporal dependencies. Furthermore, a multi-task learning framework is constructed to jointly optimize the recognition of valence and arousal dimensions. Through feature sharing and a cross-task attention mechanism, the model’s recognition efficiency is improved. Experiments on the DEAP dataset demonstrate that the proposed model achieves average recognition accuracies of 98.26% for valence and 98.67% for arousal, significantly outperforming existing mainstream methods, thereby validating the effectiveness and advancement of the proposed approach in EEG-based emotion recognition.