群体皮电信号辅助的音乐情绪流识别

Music Emotion Flow Recognition Assisted by Collective Electrodermal Activity

摘要: 动态音乐情绪识别旨在刻画音乐推进过程中连续变化的效价–唤醒度轨迹。现有方法主要依赖音频和群体平均主观标签，对音乐诱发的听者生理响应利用不足。皮电活动(Electrodermal Activity, EDA)反映汗腺活动引起的皮肤电学性质变化，可表征听者在音乐聆听过程中与交感神经活动相关的连续生理反应，为音乐情绪识别提供音频之外的生理证据。然而，将多名听者的EDA序列直接汇总，容易混合可重复的共同响应、听者差异与测量扰动。为此，本文提出群体皮电辅助音乐情绪流网络(Collective Electrodermal Activity-Assisted Music Emotion Flow Network, CEA-Flow)。该网络将同一音乐刺激下的多听者EDA建模为无序时序集合，通过不相交听者分组学习群体EDA共同响应，并以听者差异信息估计响应的可靠性，采用音频-EDA快、慢双时间尺度编码预测连续情绪轨迹。实验显示，采用CEA-Flow模型预测的效价和唤醒度一致性相关系数分别为0.201 ± 0.006和0.326 ± 0.005，平均一致性相关系数较仅音频特征模型预测提高0.048，且优于歌曲错配和时间错位EDA对照。结果表明，群体EDA共同响应能够为音频表征补充多听者在音乐刺激下的连续生理响应线索，听者差异信息则有助于估计并调节这些线索在不同时间位置的可靠性。

Abstract: Dynamic music emotion recognition aims to characterize the continuously changing valence-arousal trajectory as music unfolds. Existing methods rely mainly on audio features and group-averaged subjective annotations, with limited use of the physiological responses induced by music. Electrodermal activity (EDA) reflects changes in the electrical properties of the skin caused by sweat gland activity. It can characterize continuous physiological responses associated with sympathetic activity during music listening and provide evidence beyond audio for music emotion recognition. However, directly aggregating EDA sequences from multiple listeners may mix repeatable common responses, listener differences, and measurement noise. To address this problem, we propose the Collective Electrodermal Activity-Assisted Music Emotion Flow Network (CEA-Flow). CEA-Flow models multi-listener EDA recorded under the same musical stimulus as an unordered set of temporal sequences. It learns the group EDA common response through disjoint listener groups, estimates response reliability from listener difference information, and predicts continuous emotion trajectories using Fast and Slow temporal encoding of audio and EDA. Experiments show that CEA-Flow achieves concordance correlation coefficients of 0.201 ± 0.006 for valence and 0.326 ± 0.005 for arousal. Its mean concordance correlation coefficient is 0.048 higher than that of the model using only audio features, and it outperforms the song-mismatched and time-shifted EDA controls. These results indicate that the group EDA common response can supplement audio representations with continuous physiological response cues shared by multiple listeners during music stimulation, while listener difference information helps estimate and regulate the reliability of these cues at different time positions.