时频双域注意力融合的YOLOv12课堂微表情识别研究

Research on Time-Frequency Dual-Domain Attention Fusion in YOLOv12 for Classroom Micro-Expression Recognition

摘要: 在课堂教学场景中，基于视觉的学生课堂表情识别研究对于学习状态评估和教学效果优化具有重要指导价值。针对传统YOLOv12算法在课堂表情识别任务中存在的小目标检测精度不足、易受环境干扰(低照度、复杂背景噪声)等技术瓶颈，本研究提出多维特征增强的改进架构：在backbone网络设计中，引入了MCAM-A2C2f复合模块，通过通道–高度–宽度三维注意力机制与动态特征聚合的协同优化，实现了跨维度特征交互与关键信息强化；在检测头模块，采用DASM-C3K2混合架构，通过空间语义注意力(SSA)机制有效融合局部微表情特征与全局上下文信息，结合频域注意力(FSA)模块捕捉高频表情细节，构建时频双域特征表达体系。特别地，引入自适应阈值焦点损失函数(ATFL)替代传统交叉熵损失，通过动态调整难易样本权重系数，显著提升模型的环境鲁棒性。在自建的课堂表情数据集上，改进后的YOLOv12m_MCAM实现0.4%的mAP提升，YOLOv12m_DSAM取得1.5%的检测精度增益，而YOLOv12m_ATFL更展现出6.2%的显著性能提升，YOLOv12m_(MCAM & DSAM)提升了1.3%，YOLO_(ATFL + MCAM)提升了1.7%，YOLO_(ATFL + DSAM)提升了0.1%，YOLOv12m_(ATFL + MCAM & DSAM)提升了0.5%。

Abstract: In the context of classroom teaching scenarios, research on student in-class expression recognition based on visual analysis holds significant guiding value for evaluating learning states and optimizing teaching effectiveness. Addressing the technical limitations of the traditional YOLOv12 algorithm in classroom expression recognition tasks—such as inadequate detection accuracy for small targets and vulnerability to environmental interferences (low illumination, complex background noise)—this study proposes an enhanced architecture with multi-dimensional feature augmentation. In the backbone network design, an MCAM-A2C2f composite module is introduced. This module synergistically optimizes a three-dimensional attention mechanism across channel, height, and width dimensions, combined with dynamic feature aggregation, thereby enabling cross-dimensional feature interaction and reinforcement of critical information. For the detection head module, a DASM-C3K2 hybrid architecture is adopted. By integrating spatial semantic attention (SSA) mechanisms to effectively fuse local micro-expression features with global contextual information, and incorporating a frequency-domain sensitivity attention (FSA) module to capture high-frequency expression details, this architecture constructs a time-frequency dual-domain feature representation framework. Notably, an adaptive threshold focal loss (ATFL) function is proposed to replace conventional cross-entropy loss. Through dynamic adjustment of weight coefficients for samples of varying difficulty levels, this loss function significantly enhances the model’s environmental robustness. Experimental results on our self-constructed classroom expression dataset demonstrate that the improved YOLOv12m_MCAM achieves a 0.4% increase in mAP, while YOLOv12m_DSAM gains a 1.5% improvement in detection accuracy. YOLOv12m_ATFL shows a remarkable 6.2% performance boost. Further combinations yield: YOLOv12m-MCAM & DSAM improves by 1.3%, YOLO-ATFL + MCAM by 1.7%, YOLO-ATFL + DSAM by 0.1%, and YOLOv12m-ATFL + MCAM & DSAM by 0.5%.