MS3D-Net：一种端到端的多传感器融合3D检测网络

MS3D-Net: An End-to-End Multi-Sensor Fusion 3D Detection Network

摘要: 随着自动驾驶技术的发展，对车辆环境的3D感知要求越来越高，而多传感器融合可以很好的满足这一要求。针对目前融合技术中存在的网络设计不系统、信息丢失过大和融合策略粗糙问题，本文设计了一种端到端的多传感器融合3D检测网络——MS3D-Net。为秉承系统设计理念找到最优的多模态融合层级，先提出了新的融合层次划分法，再基于Faster-Rcnn源架构的检测模型中通过控制变量法，找到了最适合的特征融合层级；为降低跨模态数据融合过程中的信息损失，设计了新的高维表示，并提出与之对应的融合方法3D-T；为提高融合策略的精细度，提高融合检测精度，受Long Short-Term Memory (LSTM)机制启发拓展设计了中晚期门控递归融合单元，同时为提升图像特征的提取效率，提出了CP卷积。最后在KITTI数据集上进行训练与验证，本文方法在提高检测精度的同时又保证了检测速度。

Abstract: With the development of autonomous driving technology, 3D perception of the vehicle environment is becoming more and more demanding, and multi-sensor fusion can meet this requirement very well. To address the problems of unsystematic network design, excessive information loss and rough fusion strategies in current fusion technologies, this paper designs an end-to-end multi-sensor fusion sensing network—MS3D-Net. In order to find the optimal multi-modal fusion hierarchy in adherence to the system design concept, a new fusion hierarchy division method is first proposed, and then the most suitable feature fusion level was found by the control variable method in the detection model based on the Faster-Rcnn architecture. In order to reduce the information loss during cross-modal data fusion, a new high-dimensional representation is designed, and proposes the corresponding fusion method 3D-T. In order to improve the fineness of the fusion strategy and increase the fusion detection accuracy, a mediumlate gated recursive fusion unit is extended and designed inspired by the long short-term memory (LSTM) mechanism. At the same time, in order to improve the efficiency of image feature extraction, CP convolution is proposed. Finally, the method is trained and validated on the KITTI dataset, and the detection speed is guaranteed while improving the detection accuracy.