FM-VXNet：一种基于MVX-Net改进的多模态3D目标检测算法的研究

FM-VXNet: A Study on an Improved Multimodal 3D Object Detection Algorithm Based on MVX-Net

摘要: 多模态3D目标检测通过融合不同模态信息，有效克服了单一模态的局限性，在自动驾驶和机器人导航等领域展现出重要价值。然而，现有方法仍存在图像分支对全局语义的建模能力有限、跨模态融合多依赖简单拼接，未能充分挖掘模态间的互补潜力、点云体素特征在密度分布不均时易受噪声或冗余信息干扰等不足。针对上述问题，本文提出频域多模态体素网络(Frequency-domain Multimodal Voxel Network, FM-VXNet)模型。该模型是基于多模态体素网络(Multimodal Voxel Network, MVX-Net)设计，它包含三个核心模块：(1) 在图像分支中引入频域–空间域融合模块(Frequency and Spatial Fusion Module, FFCM)，借助快速傅里叶变换增强全局语义感知能力；(2) 提出双向跨模态门控注意力模块(Bidirectional Cross-Modal Gated Attention, Bi-CMGA)，实现图像与点云特征间的双向交互融合，并引入通道级门控机制以抑制噪声干扰，提升融合特征的判别力；(3) 在体素特征编码阶段设计双模态密度感知注意力模块(Bimodal Density-aware Attention, BiDA)，通过密度感知与通道重标定机制，有效缓解稀疏体素中的噪声问题和密集体素中的冗余现象。改进后的FM-VXNet算法在KITTI数据集上的实验表明，FM-VXNet在鸟瞰图(BEV)检测任务中，全类平均精度(mean Average Precision, mAP)在简单、中等和困难场景下分别达到96.3%、95.2%和92.9%；在3D检测任务中，mAP分别达到96.2%、88.9%和87.7%，相较BEVFusion、MVX-Net等主流算法平均提升5.7%~8.2%。本研究创新性地引入频域分析、双向门控注意力与密度感知机制，为多模态3D目标检测提供了新的研究思路。

Abstract: Multimodal 3D object detection, by fusing data from different modalities, effectively overcomes the limitations of single-modal approaches and has demonstrated significant value in fields such as autonomous driving and robot navigation. However, current methods still face several shortcomings: the image branch has a limited capacity for global semantic modeling; cross-modal fusion often relies on simple feature concatenation, failing to fully exploit the complementary potential between modalities; and point cloud voxel features are susceptible to noise or redundant information when the density distribution is uneven. To address these issues, this paper proposes the Frequency-domain Multimodal Voxel Network (FM-VXNet). Designed based on the Multimodal Voxel Network (MVX-Net), the model incorporates three core modules: (1) the Frequency and Spatial Fusion Module (FFCM), which leverages the Fast Fourier Transform (FFT) to enhance global semantic perception in the image branch; (2) the Bidirectional Cross-Modal Gated Attention (Bi-CMGA) module, which enables bidirectional interactive fusion between image and point cloud features and introduces a channel-wise gating mechanism to suppress noise and improve the discriminative power of the fused features; (3) the Bimodal Density-aware Attention (BiDA) module, which operates during the voxel feature encoding stage and effectively mitigates noise in sparse voxels and redundancy in dense voxels through density-aware and channel re-calibration mechanisms. Experiments on the KITTI dataset show that the enhanced FM-VXNet algorithm achieves mean Average Precision (mAP) scores of 96.3%, 95.2%, and 92.9% for the Bird’s Eye View (BEV) detection task under easy, moderate, and hard settings, respectively. For the 3D detection task, it achieves mAP scores of 96.2%, 88.9%, and 87.7% across the respective difficulty levels, outperforming state-of-the-art methods like BEV Fusion and MVX-Net by an average of 5.7% to 8.2%. This research innovatively introduces frequency-domain analysis, bidirectional gated attention, and density-aware mechanisms, offering a new direction for multimodal 3D object detection research.