基于改进3D-ResNet34模型的视频人员情绪识别研究

Research on Video Personnel Emotion Recognition Based on Improved 3D-ResNet34 Model

摘要: 由于人员情绪的多变性，面部表情会随着时间的推移与情绪的变化而变化，这一点在视频数据中表现尤为显著，此时使用只对单帧图像进行训练的网络模型来判别视频数据中的人员心情显然不可行。因此为解决上述问题，提出一种结合3D-ResNet34、ConvLSTM以及Transformer的网络模型用于人员情绪识别的方法。首先，对视频数据集进行预处理，将完整的视频数据划分成若干个连续片段，使用3D-ResNet34网络对视频片段进行空间特征提取。其次，在网络模型中设计添加ConvLSTM模块用于从空间特征中提取更深层次的时间维度特征。然后，通过使用自设计的Transformer模块对每个视频片段生成一个注意力分布，用于加权融合各帧特征，得到一个表示整个视频片段的注意力特征。最后对提取完成的时间维度特征与注意力特征进行特征融合，形成一个新的特征表示，并送入Softmax层进行分类识别。实验表明，本文设计的方法取得了较好的识别准确率。

Abstract: Due to the variability of people’s emotions, facial expressions will change with the passage of time and the changes of emotions, which is particularly significant in video data. At this time, it is obviously not feasible to use the network model trained only on a single frame image to identify people’s moods in video data. Therefore, in order to solve the above problems, a method combining 3D-ResNet34, ConvLSTM and Transformer network model is proposed for human emotion recognition. Firstly, the video data set is preprocessed, the complete video data is divided into several continuous fragments, and the spatial features of the video fragments are extracted using 3D-ResNet34 network. Secondly, ConvLSTM modules are added to the network model to extract deeper temporal features from spatial features. Then, a self-designed Transformer module can be used to generate an attention distribution for each video clip, which can be used to weight and fuse each frame feature to obtain an attention feature representing the entire video clip. Finally, the extracted time dimension features and attention features are fused to form a new feature representation, which is sent to the Softmax layer for classification and recognition. Experiments show that the method designed in this paper achieves a good recognition accuracy.