多重注意力非局部特征融合的序列影像–激光点云深度补全
Depth Completion between Single Image and Sparse LiDAR Data with Multiple Attention and Nonlocal Feature
DOI: 10.12677/GST.2022.102011, PDF,    国家自然科学基金支持
作者: 张 晨*, 陈 驰, 杨必胜：武汉大学测绘遥感信息工程国家重点实验室，湖北 武汉；张 飞：上海维智卓新信息科技有限公司，上海
关键词: 深度补全；注意力机制；自校准卷积；非局部特征；点云；Depth Completion； Attention Model； Self-Calibrated Convolutions； Nonlocal Feature； Point Cloud
摘要: 有效且准确的深度信息能够精确感知场景的几何结构。目前主要采用的是激光雷达传感器。但由于其扫描线数有限，深度信息的稠密度非常低。由此提出深度补全任务(Depth Completion)，基于给定的稀疏深度值来估计稠密深度信息。本文针对先前深度补全存在的边界模糊和混合深度的问题，提出一种多重注意力非局部特征融合的序列影像–激光点云深度补全模型。该模型通过融合人序列影像和激光点云，实现了多模态数据的优势互补，通过网络学习到更丰富的特征。实验表明该方法可以有效降低误差，提高深度补全的效果，极大地提高例如目标识别、目标跟踪、路径规划等任务的精度。
Abstract: Effective and accurate depth information can accurately perceive the geometry of the scene. At present, LiDAR sensors are mainly used. But due to the limited number of scanning lines, the density of depth information is very low. Therefore, a depth completion task is proposed to estimate the dense depth information based on the given sparse depth map. In order to solve the problems of boundary blur and mixed depth in previous depth completion tasks, this paper proposes a depth completion network between single image and sparse LiDAR data with multiple attention and non-local feature. The fusion of image and LiDAR data realizes the complementary advantages of multi-modal data, and richer features can be learned through the network. Experiments show that this network can effectively reduce the error, improving the effect of depth completion, and greatly improve the accuracy of tasks such as target recognition, target tracking, path planning and so on.
文章引用：张晨, 张飞, 陈驰, 杨必胜. 多重注意力非局部特征融合的序列影像–激光点云深度补全[J]. 测绘科学技术, 2022, 10(2): 111-120. https://doi.org/10.12677/GST.2022.102011

参考文献

[1] Cui, Z., Heng, L., Yeo, Y.C., Geiger, A., Pollefeys, M. and Sattler, T. (2019) Real-Time Dense Mapping for Self-Driving Vehicles Using Fisheye Cameras. International Conference on Robotics and Automation (ICRA), Montreal, 20-24 May 2019, 6087-6093. [Google Scholar] [CrossRef] 
[2] Song, X., Dai, Y., Zhou, D., Liu, L., Li, W., Li, H. and Yang, R. (2020) Channel Attention Based Iterative Residual Learning for Depth Map Super-Resolution. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 5630-5639. [Google Scholar] [CrossRef] 
[3] Armbrüster, C., Wolter, M., Kuhlen, T., Spijkers, W. and Fimm, B. (2008) Depth Perception in Virtual Reality: Distance Estimations in Peri- and Extrapersonal Space. Cyberpsychology Behavior, 11, 9-15. [Google Scholar] [CrossRef] [PubMed]
[4] Park, J., Joo, K., Hu, Z., Liu, C.-K. and So Kweon, I. (2020) Non-Local Spatial Propagation Network for Depth Completion. European Conference on Computer Vision (ECCV), 120-136. [Google Scholar] [CrossRef] 
[5] Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T. and Geiger, A. (2017) Sparsity Invariant CNNs. International Conference on 3D Vision (3DV), Qingdao, 10-12 October 2017, 11-20. [Google Scholar] [CrossRef] 
[6] Chodosh, N., Wang, C. and Lucey, S. (2018) Deep Convolutional Compressed Sensing for Lidar Depth Completion. Asian Conference on Computer Vision, 499-513. [Google Scholar] [CrossRef] 
[7] Ma, F. and Karaman, S. (2018) Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image. International Conference on Robotics and Automation (ICRA), Brisbane, 21-25 May 2018, 4796-4803. [Google Scholar] [CrossRef] 
[8] Cheng, X., Wang, P. and Yang, R. (2018) Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network. European Conference on Computer Vision (ECCV), 103-119. [Google Scholar] [CrossRef] 
[9] Liu, S., De Mello, S., Gu, J., Zhong, G., Yang, M.-H. and Kautz, J. (2017) Learning Affinity via Spatial Propagation Networks. Advances in Neural Information Processing Systems, 30.
[10] Liu, J.-J., Hou, Q., Cheng, M.-M., Wang, C. and Feng, J. (2020) Improving Convolutional Networks with Self-Calibrated Convolutions. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 10093-10102. [Google Scholar] [CrossRef] 
[11] Woo, S., Park, J., Lee, J.-Y. and Kweon, I.S. (2018) CBAM: Convolutional Block Attention Module. European Conference on Computer Vision (ECCV), 3-19. [Google Scholar] [CrossRef] 
[12] Qiu, J., Cui, Z., Zhang, Y., Zhang, X., Liu, S., Zeng, B. and Pollefeys, M. (2019) DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse Lidar Data and Single Color Image. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 3313-3322. [Google Scholar] [CrossRef] 
[13] Xu, Y., Zhu, X., Shi, J., Zhang, G., Bao, H. and Li, H. (2019) Depth Completion from Sparse Lidar Data with Depth-Normal Constraints. IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 2811-2820. [Google Scholar] [CrossRef] 

为你推荐