基于轻量孪生网络的无人机RGB-T目标跟踪算法

RGB-T Tracking Algorithm for Unmanned Aerial Vehicles Based on Lightweight Siamese Network

摘要: 随着无人机在目标跟踪领域的广泛应用，尤其是在复杂环境(如低光照、恶劣天气)下的追踪效果难以保障，可见光与热红外(RGB-T)双模态数据融合成为提升跟踪性能的关键手段。然而，这种融合面临异构特征高效交互、视角差异及计算资源受限等挑战。本文提出一种基于孪生网络的轻量化目标跟踪算法SiamTSA (Siamese Network with Temporal and Spatial Attention)。首先，采用改进的MobileNetV3-small作为主干网络，降低计算开销并适配无人机平台；其次，设计跨模态时空交互注意力模块，通过时间注意力建模视觉风格差异和空间注意力对齐视角差异，抑制冗余噪声并增强跨模态一致性特征表达；进一步提出双模态自适应惩罚选择模块，通过分析预测框的尺度与宽高比变化筛选更优输出框，提升了跟踪框的稳定性。在GTOT、RGBT234及VTUAV数据集上的实验表明，SiamTSA在跟踪成功率(VTUAV: 67.5%)与实时性(56.3 FPS)方面均优于主流算法，兼顾精度与效率。本文方法为复杂场景下的无人机多模态目标跟踪提供了轻量化解决方案。

Abstract: With the widespread application of unmanned aerial vehicles (UAVs) in object tracking, especially the increasing demand for robust performance in complex environments (e.g., low-light conditions, adverse weather), the fusion of visible and thermal infrared (RGB-T) multimodal data has become a critical approach to enhance tracking accuracy. However, this fusion faces challenges such as efficient interaction of heterogeneous features, perspective differences, and limited computational resources. This paper proposes a lightweight object tracking algorithm named SiamTSA (Siamese Network with Temporal and Spatial Attention). First, an improved MobileNetV3-small is adopted as the backbone to reduce computational costs and adapt to UAV platforms. Second, a cross-modal temporal spatial interaction attention module is designed to model visual style differences via temporal attention and align spatial discrepancies via spatial attention, thereby suppressing redundant noise and enhancing cross-modal consistent feature representation. Furthermore, a dual-modal adaptive penalty selection module enhances tracking stability by selecting optimal bounding boxes through analysis of scale and aspect ratio variations. Experiments on GTOT, RGBT234, and VTUAV datasets demonstrate that SiamTSA outperforms state-of-the-art methods in tracking success rate (VTUAV: 67.5%) and real-time performance (56.3 FPS), balancing accuracy and efficiency. The proposed method provides a lightweight solution for UAV-based multimodal object tracking in complex scenarios.