跨模态时空交叉注意力下机器人抓取滑动检测

Robot Grasping Slip Detection Based on Cross-Modal Spatiotemporal Cross-Attention Mechanism

摘要: 在机器人领域，滑动检测是一个关键的任务。机器人需要利用多模态信息进行特征提取、信息融合交互与灵巧操作。为此，提出一个基于跨模态时空交叉注意力机制的多模态融合模型，用于滑动检测。该模型利用时空注意力学习多模态传感器反馈的物理特征，将学习到的视触觉时空特征通过跨模态交叉注意力进行交互融合。最后，通过多层感知机(MLP)预测滑动检测结果。使用7自由度XArm机械臂、D455摄像头和XELA触觉传感器进行数据采集、模型训练和验证。结果表明，该模型的滑动检测准确率高达97.8%，所提出的模型在可靠、顺利执行机器人抓取任务方面具有较高的研究和应用价值。

Abstract: In the field of robotics, slip detection is a crucial task. Robots need to utilize multimodal information for feature extraction, information fusion interaction, and dexterous manipulation. For this, a multimodal fusion model based on cross-modal spatiotemporal attention mechanism is proposed for slip detection. The model uses spatiotemporal attention to learn the physical features reflected by multimodal sensor feedback, and the learned visuotactile spatiotemporal features are interactively fused through cross-modal attention. Finally, slip detection results are predicted using a Multi-layer perceptron (MLP). Data collection, model training, and validation are carried out using a 7-DOF XArm robotic arm, a D455 camera, and XELA tactile sensors. The results indicate that the slip detection accuracy of this model reaches up to 97.8%, demonstrating the high research and practical value of the proposed model in ensuring reliable and smooth execution of robotic grasping tasks.