基于人物交互的学生课堂行为识别研究

Research on Students’ Classroom Behavior Recognition Based on Human-Object Interaction

摘要: 学生课堂行为分析是评估课堂教学效果的有效方法，现有的学生行为识别研究仅针对学生自身进行识别，而对学生与周围物品的交互关注不够。基于此，本文提出了一种基于人物交互的学生课堂行为识别研究方法，通过分析教室监控视频，检测出学生和物品目标，并基于人与物的交互关系来识别其课堂行为。首先，考虑到笔和手机等小目标物品所占像素和可提取的有效特征较少，提出了一种改进YOLOv5s的目标检测方法，解决随着网络层数的叠加，小目标的特征信息逐渐消失导致漏检的问题。然后，为解决教室环境中目标数量较多，目标与目标间存在遮挡等因素导致网络难以提取特征的问题，引入Triplet注意力机制，增强网络提取特征的能力。接下来，采用VSGNet网络识别人与物的交互关系以确定行为类别。最后，在自制教室数据集和公开数据集进行了多组对比实验，实验结果表明，与原YOLOv5s网络相比，改进后的网络在自制和公开数据集上mAP分别提升了3.06%和3.2%，召回率分别提升了3.1%和4.2%，验证了改进方法的有效性。

Abstract: Student classroom behavior analysis is an effective way to assess the effectiveness of classroom instruction. Existing research on student behavior identification focuses only on the identification of students themselves, and not enough attention is paid to students’ interactions with surrounding objects. Based on this, this paper proposes a research method for recognizing students’ classroom behaviors based on Human-Object interaction, by analyzing classroom surveillance videos, detecting students and object targets, and recognizing their classroom behaviors based on person-object interaction. Firstly, considering that small target items such as pens and cell phones occupy fewer pixels and can extract fewer effective features, a target detection method is proposed to improve YOLOv5s to solve the problem of missing detection due to the gradual disappearance of feature information of small targets as the layers of the network are superimposed. Then, in order to solve the problem that it is difficult for the network to extract features due to a large number of targets in the classroom environment and the occlusion between targets and targets, the Triplet attention mechanism is introduced to enhance the ability of the network to extract features. Next, the VSGNet network is used to identify human-object interactions to determine behavior categories. Finally, multiple sets of comparison experiments were conducted on the self-made classroom dataset and the public dataset. The experimental results showed that the improved network improved the mAP by 3.06% and 3.2% on the self-made and public datasets, respectively, and improved the recall by 3.1% and 4.2%, respectively, compared with the original YOLOv5s network, which verified the effectiveness of the improved method.