基于LSTM多特征融合人体动作识别算法

Human Action Recognition Algorithm Based on LSTM with Multi-Feature Fusion

摘要: 为解决现有人体动作识别方法依赖单一模态数据、复杂场景下识别精度不足的问题，提出一种基于LSTM的多特征融合人体动作识别算法。首先，通过Kinect深度相机获取人体20个关键关节点的3D坐标信息，提取关节点的位置、运动速度及加速度等多维度特征；其次，通过关键帧定位与降采样处理优化特征数据，降低计算复杂度；最后，将整合后的多模态特征输入LSTM模型，利用其时空建模能力实现动作分类。为验证算法性能，在Kinetics-Skeleton、NTU-RGB (X-Sub)及NTU-RGB + D (X-View)三个公开数据集上进行实验，结果表明：算法对前20类动作的识别准确率均达到98%以上，其中NTU-RGB + D (X-View)数据集上部分动作类别准确率超99%；混淆矩阵分析显示模型分类一致性良好，具备较强的抗干扰能力与场景适应性。该算法通过多特征融合策略充分挖掘人体动作的时空关联信息，为复杂环境下的高精度动作识别提供了有效解决方案。

Abstract: To address the issues of existing human action recognition methods relying on single-modal data and insufficient recognition accuracy in complex scenarios, a human action recognition algorithm based on LSTM with multi-feature fusion is proposed. Firstly, 3D coordinate information of 20 key human joint points is acquired using a Kinect depth camera, and multi-dimensional features such as the position, motion velocity, and acceleration of the joint points are extracted. Secondly, key frame localization and downsampling are applied to optimize the feature data and reduce computational complexity. Finally, the integrated multi-modal features are input into the LSTM model, which leverages its spatiotemporal modeling capability to achieve action classification. To verify the algorithm’s performance, experiments are conducted on three public datasets: Kinetics-Skeleton, NTU-RGB (X-Sub), and NTU-RGB + D (X-View). The results demonstrate that the algorithm achieves an accuracy of over 98% for the top 20 action categories on all three datasets, with some categories exceeding 99% accuracy on the NTU-RGB + D (X-View) dataset. Confusion matrix analysis shows that the model exhibits good classification consistency, along with strong anti-interference ability and scene adaptability. By fully exploring the spatiotemporal correlation information of human actions through a multi-feature fusion strategy, the algorithm provides an effective solution for high-precision action recognition in complex environments.