面向未来状态预警的多模态疲劳识别与连续预测研究

Research on Multimodal Fatigue Recognition and Continuous Prediction for Future State Early Warning

摘要: 为解决传统疲劳检测方法主观性强、实时性不足的问题，本研究提出了一种面向未来状态预警的多模态疲劳识别与连续预测框架。该框架融合了基于光电容积描记(PPG)信号的生理指标与基于面部图像的行为特征，旨在实现对个体疲劳状态的精准动态监测与预测。研究首先利用集成学习构建了PPG信号三分类(清醒、中度疲劳、高度疲劳)模型，准确率达到98%。同时，设计了结合时序卷积(TimeDistributed CNN)与长短期记忆网络(LSTM)的视觉模型，以捕捉动态面部特征。在此基础上，本文提出一种基于标签对齐的后期融合策略，通过逻辑回归模型自适应加权各模态贡献，有效提升了分类鲁棒性。为实现状态预警，进一步构建了多变量LSTM联合预测模型，以预测关键生理指标的未来趋势。实验结果表明，该多模态框架能够准确识别不同疲劳等级，并为实现从“事后识别”到“事前干预”的智能疲劳管理提供了有效的方法论支持。

Abstract: To address the subjectivity and real-time limitations of traditional fatigue detection methods, this study proposes a multimodal fatigue recognition and continuous prediction framework for future state early warning. The framework integrates physiological indicators from photoplethysmography (PPG) signals with behavioral features from facial images, aiming to enable accurate dynamic monitoring and prediction of individual fatigue states. Initially, an ensemble learning model was developed to classify PPG signals into three states (awake, moderate fatigue, and severe fatigue), achieving 98% accuracy. Concurrently, a visual model combining a TimeDistributed Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) network was designed to capture dynamic facial features. Building on this, a late fusion strategy based on label alignment is introduced, which uses a logistic regression model to adaptively weight the contributions of each modality, effectively enhancing classification robustness. To enable early warning, a multivariate LSTM model was further constructed to forecast the future trends of key physiological indicators. Experimental results demonstrate that the proposed multimodal framework can accurately identify different levels of fatigue and provides effective methodological support for transitioning from post-hoc recognition to proactive intervention in intelligent fatigue management.