基于深度学习与Mediapipe的列车司机手比行为检测方法研究

Research on Train Driver Hand Gesture Behavior Detection Method Based on Deep Learning and Mediapipe

摘要: 列车司机的行为监测在智能交通系统中对于提高安全性和减少交通事故至关重要。针对列车司机在驾驶过程中手比行为的识别，本研究提出了一种基于深度学习与Mediapipe技术相结合的手比行为检测方法。研究重点在于提升手比行为的检测精度与实时性，尤其是在复杂环境下的应用。研究首先使用ResNet50卷积神经网络(CNN)对列车驾驶舱图像数据集进行训练，完成对掌手比与指手比的分类任务。通过对不同手势类型的数据进行训练，模型成功实现了超过85%的准确率，验证了深度学习在此类行为识别中的有效性。此外，研究采用了Mediapipe框架，通过实时的手部关键点检测与姿态估计，基于动态视频数据对智轨司机的手比行为进行了分析。该方法结合关键点之间的几何关系，准确率达到90%，能够在动态驾驶环境中实现高效的行为识别。本研究的创新性在于，结合深度学习的特征提取能力与Mediapipe的实时骨架点检测，优化了手比行为的检测精度和环境适应性。通过实验验证，提出的检测方法能够在复杂环境下稳定运行，具有显著的实时性和鲁棒性。这为智能交通系统中的司机行为监控提供了新的技术路径，尤其在提升智能驾驶舱安全性和交互效率方面具有重要应用价值。

Abstract: The monitoring of train driver behavior is crucial for enhancing safety and reducing traffic accidents in intelligent transportation systems. This study proposes a hand gesture behavior detection method for train drivers during operation, which combines deep learning with Mediapipe technology. The focus of the research is to improve the detection accuracy and real-time performance of hand gestures, especially in complex environments. The study first uses the ResNet50 convolutional neural network (CNN) to train a dataset of train cockpit images, completing the classification task of palm gestures and finger gestures. By training on different gesture types, the model successfully achieved an accuracy rate exceeding 85%, validating the effectiveness of deep learning in such behavior recognition tasks. Additionally, the research employs the Mediapipe framework for real-time hand keypoint detection and posture estimation, analyzing the hand gesture behaviors of smart track drivers based on dynamic video data. The method, which incorporates the geometric relationships between keypoints, achieved an accuracy rate of 90%, enabling efficient behavior recognition in dynamic driving environments. The novelty of this study lies in the integration of deep learning’s feature extraction capabilities with Mediapipe’s real-time skeletal point detection, optimizing the detection accuracy and environmental adaptability of hand gestures. Experimental validation shows that the proposed detection method can operate stably in complex environments, demonstrating significant real-time performance and robustness. This provides a new technical pathway for driver behavior monitoring in intelligent transportation systems, with substantial application value, particularly in enhancing the safety and interaction efficiency of intelligent cockpits.