点云上采样技术研究
Research on Point Cloud Upsampling Technologies
DOI: 10.12677/JISP.2024.131002, PDF,    国家自然科学基金支持
作者: 徐 天, 张长伦*：北京建筑大学理学院，北京
关键词: 深度学习；点云上采样；特征扩展；Deep Learning； Point Cloud Upsampling； Feature Expansion
摘要: 点云上采样旨在将稀疏、嘈杂、不完整的点云转换为密集、干净、完整的点云，这样有利于提高下游任务的性能。当前的点云上采样方法主要分为基于优化和基于学习的方法，本文针对基于深度学习的点云上采样算法进行了综述。本文从点云上采样的开山之作PU-Net引入，阐述了上采样算法的发展，并总结了插值算法在点云上采样中的应用，然后对不同特征扩展方法进行了比较，介绍了上采样算法的评价指标和常用数据集，最后对点云上采样的发展前景进行了展望。
Abstract: Point cloud upsampling aims to convert sparse, noisy, and incomplete point clouds into dense, clean, and complete point clouds, which is conducive to improving the performance of downstream tasks. Current point cloud upsampling algorithms are mainly classified into optimization-based and learning-based methods, and this paper provides a review of deep learning-based point cloud up-sampling algorithms. This paper introduces PU-Net, the pioneer of point cloud upsampling. Then it describes the development of upsampling algorithms and summarizes the application of interpolation algorithms in point cloud upsampling. Then it compares the different feature expansion meth-ods, introduces the evaluation indexes of upsampling algorithms and the commonly used datasets. Finally, it looks forward to the development prospects of point cloud upsampling.
文章引用：徐天, 张长伦. 点云上采样技术研究[J]. 图像与信号处理, 2024, 13(1): 10-20. https://doi.org/10.12677/JISP.2024.131002

参考文献

[1] Cai, Q., Pan, Y., Yao, T. and Mei, T. (2022) 3D Cascade RCNN: High Quality Object Detection in Point Clouds. IEEE Transactions on Image Processing, 31, 5706-5719. [Google Scholar] [CrossRef] 
[2] An, P., Liang, J., Ma, T., Chen, Y., Wang, L. and Ma, J. (2023) ProUDA: Progressive Unsupervised Data Augmentation for Semi-Supervised 3D Object Detection on Point Cloud. Pattern Recognition Letters, 170, 64-69. [Google Scholar] [CrossRef] 
[3] Xie, G., Li, Y., Wang, Y., Li, Z. and Qu, H. (2023) 3D Point Cloud Object Detection Algorithm Based on Temporal Information Fusion and Uncertainty Estimation. Remote Sensing, 15, Article 2986. [Google Scholar] [CrossRef] 
[4] Newcombe, R.A., Izadi, S., Hilliges, O., et al. (2011) KinectFusion: Real-Time Dense Surface Mapping and Tracking. IEEE International Symposium on Mixed and Aug-mented Reality (ISMAR), Basel, 26-29 October 2011, 127-136. [Google Scholar] [CrossRef] 
[5] Riegler, G., Ulusoy, A.O., Bischof, H. and Geiger, A. (2017) Octnetfusion: Learning Depth Fusion from Data. IEEE In-ternational Conference on 3D Vision (3DV), Qingdao, 10-12 October 2017, 57-66. [Google Scholar] [CrossRef] 
[6] Charles, R.Q., Su, H., Kaichun, M. and Guibas, L.J. (2017) Point-Net: Deep Learning on Point Sets for 3D Classification and Segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 77-85. [Google Scholar] [CrossRef] 
[7] Qi, C.R., Yi, L., Su, H. and Guibas, L.J. (2017) PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. International Conference on Neural Information Pro-cessing Systems (NIPS), Long Beach, 4-9 December 2017, 5099-5108.
[8] Yu, L., Li, X., Fu, C.W., Cohen-Or, D. and Heng, P.A. (2018) PU-Net: Point Cloud Upsampling Network. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 2790-2799. [Google Scholar] [CrossRef] 
[9] Wang, YF., Wu, S., Huang, H., Cohen-Or, D. and Sorkine-Hornung, O. (2019) Patch-Based Progressive 3D Point Set Upsampling. 2019 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 5951-5960. [Google Scholar] [CrossRef] 
[10] Li, R., Li, X., Heng, P.A. and Fu, C.W. (2021) Point Cloud Up-sampling via Disentangled Refinement. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 344-353. [Google Scholar] [CrossRef] 
[11] Zhang, P., Wang, X., Ma, L., Wang, S., Kwong, S. and Jiang, J. (2021) Progressive Point Cloud Upsampling via Differentiable Rendering. IEEE Transactions on Circuits and Systems for Video Technology, 31, 4673-4685. [Google Scholar] [CrossRef] 
[12] Bai, Y., Wang, X., Ang Jr., M.H. and Rus, D. (2022) BIMS-PU: Bi-Directional and Multi-Scale Point Cloud Upsampling. IEEE Robotics and Automation Letters, 7, 7447-7454. [Google Scholar] [CrossRef] 
[13] Du, H., Yan, X., Wang, J., Xie, D. and Pu, S. (2022) Point Cloud Upsampling via Cascaded Refinement Network. In: Wang, L., Gall, J., Chin, T.J., Sato, I. and Chellappa, R., Eds., Computer Vision—ACCV 2022, Springer, Cham, 106-122.
[14] Qian, G., Abualshour, A., Li, G., Thabet, A. and Ghanem, B. (2021) PU-GCN: Point Cloud Upsampling Using Graph Convolutional Networks. 2021 IEEE/CVF Con-ference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 11678-11687. [Google Scholar] [CrossRef] 
[15] Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A. (2017) Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proceedings of the AAAI Confer-ence on Artificial Intelligence, 31, 4278-4284. [Google Scholar] [CrossRef] 
[16] Szegedy, C., et al. (2015) Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 1-9. [Google Scholar] [CrossRef] 
[17] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z. (2016) Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pat-tern Recognition (CVPR), Las Vegas, 27-30 June 2016, 2818-2826. [Google Scholar] [CrossRef] 
[18] Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M. and Sol-omon, J.M. (2019) Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics, 38, 1-12. [Google Scholar] [CrossRef] 
[19] Gu, F., Zhang, C., Wang, H., He, Q. and Huo, L. (2022) PU-WGCN: Point Cloud Upsampling Using Weighted Graph Convolutional Networks. Remote Sensing, 14, Article 5356. [Google Scholar] [CrossRef] 
[20] Li, R., Li, X., Fu, C.W., Cohen-Or, D. and Heng, P.A. (2019) PU-GAN: A Point Cloud Upsampling Adversarial Network. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 7202-7211. [Google Scholar] [CrossRef] 
[21] Liu, H., Yuan, H., Hamzaoui, R., Gao, W. and Li, S. (2022) PU-Refiner: A Geometry Refiner with Adversarial Learning for Point Cloud Upsampling. ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 23-27 May 2022, 2270-2274. [Google Scholar] [CrossRef] 
[22] Zhou, K., Dong, M. and Arslanturk, S. (2022) “Ze-ro-Shot” Point Cloud Upsampling. 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, 18-22 July 2022, 1-6. [Google Scholar] [CrossRef] 
[23] Liu, H., Yuan, H., Hou, J., Hamzaoui, R. and Gao, W. (2022) PUFA-GAN: A Frequency-Aware Generative Adversarial Network for 3D Point Cloud Upsampling. IEEE Transactions on Image Processing, 31, 7389-7402. [Google Scholar] [CrossRef] 
[24] Mao, A., Du, Z., Hou, J., Duan, Y., Liu, Y.J. and He, Y. (2022) PU-Flow: A Point Cloud Upsampling Network with Normalizing Flows. IEEE Transactions on Visualization and Computer Graphics, 29, 4964-4977. [Google Scholar] [CrossRef] 
[25] Hu, X., Wei, X. and Sun, J. (2023) A Noising-Denoising Framework for Point Cloud Upsampling via Normalizing Flows. Pattern Recognition, 140, Article ID: 109569. [Google Scholar] [CrossRef] 
[26] Liu, X.H., Han, Z.Z., Wen, X., Liu, Y.S. and Zwicker, M. (2019) L2G Auto-Encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention. Proceedings of the 27th ACM International Conference on Multimedia, Nice France, 21-25 October 2019, 989-997. [Google Scholar] [CrossRef] 
[27] Zhao, Y.F., Hui, L. and Xie, J. (2021) SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering. Proceedings of the 29th ACM International Conference on Mul-timedia, 20-24 October 2021, 2214-2223. [Google Scholar] [CrossRef] 
[28] Liu, X., Liu, X., Liu, Y.S. and Han, Z. (2022) SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization. IEEE Transactions on Image Processing, 31, 4213-4226. [Google Scholar] [CrossRef] 
[29] Feng, W., Li, J., Cai, H., Luo, X. and Zhang, J. (2022) Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling. 2022 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 18612-18621. [Google Scholar] [CrossRef] 
[30] Zhao, W., et al. (2022) Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation. 2022 IEEE/CVF Conference on Computer Vision and Pat-tern Recognition (CVPR), New Orleans, 18-24 June 2022, 1989-1997. [Google Scholar] [CrossRef] 
[31] Yang, Y., Feng, C., Shen, Y. and Tian, D. (2018) Folding-Net: Point Cloud Auto-Encoder via Deep Grid Deformation. 2018 IEEE/CVF Conference on Computer Vision and Pat-tern Recognition, Salt Lake City, 18-23 June 2018, 206-215. [Google Scholar] [CrossRef] 
[32] Han, B., Zhang, X. and Ren, S. (2022). PU-GACNet: Graph At-tention Convolution Network for Point Cloud Upsampling. Image and Vision Computing, 118, Article ID: 104371.[CrossRef] 
[33] Zhao, T., Li, L., Tian, T., Ma, J. and Tian, J. (2023) APUNet: Attention-Guided Upsampling Network for Sparse and Non-Uniform Point Cloud. Pattern Recognition, 143, Article ID: 109796. [Google Scholar] [CrossRef] 
[34] Mao, A., Duan, Y., Wen, Y.H., Du, Z., Cai, H. and Liu, Y.J. (2023) Invertible Residual Neural Networks with Conditional Injector and Interpolator for Point Cloud Upsampling. Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23), Macao, 19-25 August 2023, 1267-1275.
[35] Nguyen, T., Pham, Q.H., Le, T., Pham, T., Ho, N. and Hua, B.S. (2021) Point-Set Dis-tances for Learning Representations of 3D Point Clouds. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 10458-10467. [Google Scholar] [CrossRef] 
[36] Jesorsky, O., Kirchberg, K.J. and Frischholz, R. (2001) Ro-bust Face Detection Using the Hausdorff Distance. In: Bigun, J. and Smeraldi, F., Eds., Audio- and Video-Based Bio-metric Person Authentication, Springer, Berlin, 90-95. [Google Scholar] [CrossRef] 
[37] Fan, H., Su, H. and Guibas, L. (2017) A Point Set Generation Network for 3D Object Reconstruction from a Single Image. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 2463-2471. [Google Scholar] [CrossRef] 
[38] Chang, A.X., Funkhouser, T., Guibas, L.J., et al. (2015) ShapeNet: An Information-Rich 3D Model Repository. arXiv: 1512.03012.
[39] Li, Z., Li, G., Li, T.H., Liu, S. and Gao, W. (2023) Semantic Point Cloud Upsampling. IEEE Transactions on Multimedia, 25, 3432-3442. [Google Scholar] [CrossRef] 
[40] He, Y., Tang, D., Zhang, Y., Xue, X. and Fu, Y. (2023) Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Van-couver, 17-24 June 2023, 5354-5363. [Google Scholar] [CrossRef]