高效采样的神经隐式表面重建研究
Research on Efficient Sampling of Neural Implicit Surface Reconstruction
摘要: 近几年，神经隐式表面在三维表达上展现出了巨大的潜力。许多先前工作扩展神经辐射场(Neural Radiance Field, NeRF)的训练管线，通过在光线上采样点，然后建模出一种概率分布与隐式曲面建立联系，从而可以使得隐式曲面拟合。然而，这些工作要么需要过长的训练时间，要么需要在特定的加速结构下处理较多的采样点。本文工作，提出联合占据栅格(Occupancy Grid)和一种非线性函数来指导采样的策略，使得可以实现在合理的时间内，处理较少的采样点并达到较好的物体表面重建效果。同时，这里引入了渐进式训练方式使得模型拟合更加鲁棒。通过在真实数据集和合成数据集的实验表明，我们的方法在物体表面恢复任务上能取得较好的效果，并且在新视角合成任务上也能取得出色的表现。
Abstract: In recent years, neural implicit surfaces have demonstrated significant potential in 3D representations. Many previous works extend the training pipeline of Neural Radiance Field (NeRF) by modeling a probability distribution to establish a connection with implicit surface using sampledpoints along rays, thus enabling the good fitting of implicit surface. However, these works either require excessively long training times or deal with a large number of sampled points under specific acceleration structures. In this work, we propose a strategy that combines Occupancy Grid and a nonlinear function to guide the sampling process, allowing for the achievement of better object surface reconstruction with fewer sampled points in a reasonable amount of time. Additionally, we introduce a progressive training approach to enhance the robustness of the model fitting. Experiments results on real world and synthetic datasets demonstrate the effectiveness of our method in object surface reconstruction tasks and outstanding performance in novel view synthesis tasks.
文章引用：何思源, 刘兴林. 高效采样的神经隐式表面重建研究[J]. 计算机科学与应用, 2024, 14(3): 146-158. https://doi.org/10.12677/csa.2024.143065

参考文献

[1] Mildenhall, B., Srinivasan, P.P., Tancik, M., et al. (2021) NeRf: Representing Scenes as Neural Radiance Fields for View Synthesis. Communications of the ACM, 65, 99-106. [Google Scholar] [CrossRef] 
[2] Wang, P., Liu, L., Liu, Y., et al. (2021) Neus: Learning Neural Implicit Surfaces by Volume Rendering for Multi-View Reconstruction. arXiv: 2106.10689.
[3] Yariv, L., Gu, J., Kasten, Y., et al. (2021) Volume Rendering of Neural Implicit Surfaces. Advances in Neural Information Processing Systems, 34, 4805-4815.
[4] Oechsle, M., Peng, S. and Geiger, A. (2021) UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, 10-17 October 2021, 5569-5579. [Google Scholar] [CrossRef] 
[5] Müller, T., Evans, A., Schied, C. and Keller, A. (2022) Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Transactions on Graphics, 41, 1-15. [Google Scholar] [CrossRef] 
[6] Park, J.J., Florence, P., Straub, J., et al. (2019) DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 165-174. [Google Scholar] [CrossRef] 
[7] Mescheder, L., Oechsle, M., Niemeyer, M., et al. (2019) Occupancy Networks: Learning 3D Reconstruction in Function Space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 4455-4465. [Google Scholar] [CrossRef] 
[8] Niemeyer, M., Mescheder, L., Oechsle, M., et al. (2020) Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 3501-3512. [Google Scholar] [CrossRef] 
[9] Yariv, L., Kasten, Y., Moran, D., et al. (2020) Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. Advances in Neural Information Processing Systems, 33, 2492-2502.
[10] Wang, Y., Skorokhodov, I. and Wonka, P. (2023) PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 17-24 June 2023, 12598-12607. [Google Scholar] [CrossRef] 
[11] Wang, Y., Skorokhodov, I. and Wonka, P. (2022) Hf-Neus: Improved Surface Reconstruction Using High-Frequency Details. Advances in Neural Information Processing Systems, 35, 1966-1978.
[12] Wang, Y., Han, Q., Habermann, M., et al. (2023) NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-View Reconstruction. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, 1-6 October 2023, 3272-3283. [Google Scholar] [CrossRef] 
[13] Wu, T., Wang, J., Pan, X., et al. (2022) Voxurf: Voxel-Based Efficient and Accurate Neural Surface Reconstruction. arXiv: 2208.12697.
[14] Li, Z., Müller, T., Evans, A., et al. (2023) Neuralangelo: High-Fidelity Neural Surface Reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 17-24 June 2023, 8456-8465. [Google Scholar] [CrossRef] 
[15] Neff, T., Stadlbauer, P., Parger, M., et al. (2021) DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields Using Depth Oracle Networks. Computer Graphics Forum, 40, 45-59. [Google Scholar] [CrossRef] 
[16] Prinzler, M., Hilliges, O. and Thies, J. (2023) DINER: Depth-Aware Image-Based NEural Radiance Fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 17-24 June 2023, 12449-12459. [Google Scholar] [CrossRef] 
[17] Roessle, B., Barron, J.T., Mildenhall, B., et al. (2022) Dense Depth Priors for Neural Radiance Fields from Sparse Input Views. 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 18-24 June 2022, 12882-12891. [Google Scholar] [CrossRef] 
[18] Lorensen, W.E. and Cline, H.E. (1998) Marching Cubes: A High Resolution 3D Surface Construction Algorithm. Seminal Graphics: Pioneering Efforts That Shaped the Field, 1, 347-353. [Google Scholar] [CrossRef] 
[19] Li, L. and Zhang, J. (2023) L0-Sampler: An L0 Model Guided Volume Sampling for NeRF. arXiv: 2311.07044.
[20] Yu, Z., Chen, A., Antic, B., et al. (2022) Sdfstudio: A Unified Framework for Surface Reconstruction.
https://github.com/autonomousvision/sdfstudio
[21] Barron, J.T., Mildenhall, B., Verbin, D., et al. (2022) Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 18-24 June 2022, 5460-5469. [Google Scholar] [CrossRef]