基于深度自编码器对网络的跨域行为识别研究

Cross Domain Action Recognition Based on Deep Dual Auto-Encoder Network

摘要: 为了解决行为识别模型训练阶段需要大量视频行为数据的问题，考虑利用行为图像减少行为视频的训练数量，提高视频行为识别性能。提出深度自编码器对网络算法DDAEN (Deep Dual Auto-Encoder Networks)。首先，提取出视频的关键帧，作为将图像迁移到视频的中介。然后，根据图像、关键帧和视频的行为类别，定义跨模态相似度矩阵和自模态相似度矩阵，并借助深度网络减小图像和视频间模态差异，得到图像、关键帧和视频的域不变特征。最后，将域不变特征和语义特征作为自编码器对的输入，中间隐层得到对齐特征，并将对齐特征进行串联式融合，训练SVM分类器，实现视频行为识别。在ASD→UCF101和Stanford40→UCF101数据集上进行实验，结果表明，DDAEN在视频标记样本较少情况下，比未利用图像的视频行为识别iDTs+SVM方法在识别率上相对提高了25.44%和20.82%。证实了DDAEN可以利用图像使视频行为识别准确率提高。

Abstract: In order to solve the problem that a large amount of video action data is needed in the training stage of the action recognition model, it is considered to use images to reduce the number of videos and improve the performance of video action recognition. Deep Dual Auto-Encoder Network algorithm (DDAEN) is proposed. Firstly, the keyframe of the video is extracted as the medium to transfer the image to the video. Then, according to the action categories of images, keyframes and videos, the cross-modal similarity matrix and self-modal similarity matrix are defined, the modal differences between images and videos are reduced by a deep network, and the domain invariant features of images, keyframes and videos are obtained. Finally, the domain invariant feature and semantic feature are used as the input of the dual autoencoders, and the aligned feature is obtained by the middle hidden layer, letting the aligned feature fused in series to train the SVM classifier to realize video action recognition. Experiments on ASD→UCF101 and Stanford40→UCF101 show that the proposed method can improve the action recognition rate by 25.44% and 20.82% compared with the recognition iDTs+SVM method without image in the case of fewer video tag samples, confirming that the method in this paper can use images to make video action recognition accuracy increased.