基于多尺度细节的高保真三维人脸重建
High-Fidelity 3D Facial Reconstruction Based on Multi-Scale Details
摘要: 三维重建算法在人脸识别、影视娱乐、医疗美容等领域有广泛应用，但在细节特征恢复方面仍存在局限。为有效提高人脸几何细节重建恢复模型的建模性能，提出了一种基于多尺度细节模型的高保真三维人脸细节重建修复算法。首先，设计了A2D-ResNet50网络，并在其中集成了残差可变核卷积模块，通过动态调整卷积核采样位置以增强细节特征提取。其次，引入了相对总变分(RTV)细节损失函数以增强局部几何细节恢复。此外，对特征点损失函数进行优化，并结合拉普拉斯变形约束，确保局部细节精确对齐和全局形状保持。实验结果显示，该方法在重建精度方面较现有技术有所提高，具有更高的鲁棒性和个性化细节恢复能力。
Abstract: 3D reconstruction algorithms are widely used in fields such as face recognition, film and television entertainment, and medical aesthetics, but they have limitations in the recovery of detailed features. To balance global shape fidelity and local geometric detail recovery, a high-fidelity 3D facial reconstruction algorithm based on multi-scale details is proposed. The algorithm first designs an A2D-ResNet50 network and integrates a Residual Alterable Kernel Convolution Block within it, which enhances the extraction of detailed features by dynamically adjusting the sampling position of the convolutional kernel. Secondly, a Relative Total Variation (RTV) detail loss function is introduced to enhance the recovery of local geometric details. In addition, the feature point loss function is optimized and combined with Laplacian deformation constraints to ensure precise alignment of local details and preservation of global shape. Experimental results show that the method improves in detail accuracy compared to existing technologies and has higher robustness and the ability to recover personalized details.
文章引用：彭辰. 基于多尺度细节的高保真三维人脸重建[J]. 建模与仿真, 2025, 14(2): 796-807. https://doi.org/10.12677/mos.2025.142196

参考文献

[1] Paysan, P., Knothe, R., Amberg, B., et al. (2009) A 3D Face Model for Pose and Illumination Invariant Face Recognition. Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, Genova, 2-4 September 2009, 296-301.
[2] 高翔, 黄法秀, 刘春平, 等. 3DMM与GAN结合的实时人脸表情迁移方法[J]. 计算机应用与软件, 2020, 37(4): 119-126.
[3] Blanz, V. and Vetter, T. (2003) Face Recognition Based on Fitting a 3D Morphable Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 1063-1074. [Google Scholar] [CrossRef] 
[4] Blanz, V., Romdhani, S. and Vetter, T. (2002) Face Identification across Different Poses and Illuminations with a 3D Morphable Model. Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition, Washington DC, 21 May 2002, 202-207. [Google Scholar] [CrossRef] 
[5] Blanz, V., Scherbaum, K. and Seidel, H. (2007) Fitting a Morphable Model to 3D Scans of Faces. 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, 14-21 October 2007, 1-8. [Google Scholar] [CrossRef] 
[6] Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A. and Dunaway, D. (2016) A 3D Morphable Model Learnt from 10,000 Faces. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 5543-5552. [Google Scholar] [CrossRef] 
[7] Tran, A.T., Hassner, T., Masi, I. and Medioni, G. (2017) Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 1493-1502. [Google Scholar] [CrossRef] 
[8] Jiang, L., Zhang, J., Deng, B., Li, H. and Liu, L. (2018) 3D Face Reconstruction with Geometry Details from a Single Image. IEEE Transactions on Image Processing, 27, 4756-4770. [Google Scholar] [CrossRef] [PubMed]
[9] Tewari, A., Zollhofer, M., Bernard, F., Garrido, P., Kim, H., Perez, P., et al. (2020) High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 357-370. [Google Scholar] [CrossRef] [PubMed]
[10] Richardson, E., Sela, M., Or-El, R. and Kimmel, R. (2017) Learning Detailed Face Reconstruction from a Single Image. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 5553-5562. [Google Scholar] [CrossRef] 
[11] Zeng, X., Peng, X. and Qiao, Y. (2019) DF2Net: A Dense-Fine-Finer Network for Detailed 3D Face Reconstruction. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 2315-2324. [Google Scholar] [CrossRef] 
[12] Khan, A., Hayat, S., Ahmad, M., Cao, J., Tahir, M.F., Ullah, A., et al. (2020) Learning-Detailed 3D Face Reconstruction Based on Convolutional Neural Networks from a Single Image. Neural Computing and Applications, 33, 5951-5964. [Google Scholar] [CrossRef] 
[13] Zhang, X., Song, Y., Song, T., et al. (2023) AKConv: Convolutional Kernel with Arbitrary Sampled Shapes and Arbitrary Number of Parameters. arXiv: 2311.11587.
[14] Xu, L., Yan, Q., Xia, Y. and Jia, J. (2012) Structure Extraction from Texture via Relative Total Variation. ACM Transactions on Graphics, 31, 1-10. [Google Scholar] [CrossRef] 
[15] He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. [Google Scholar] [CrossRef] 
[16] Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C. and Seidel, H. (2004) Laplacian surface editing. Proceedings of the 2004 Eurographics/ACM Siggraph Symposium on Geometry Processing, Nice, 8-10 July 2004, 175-184. [Google Scholar] [CrossRef] 
[17] Schroff, F., Kalenichenko, D. and Philbin, J. (2015) FaceNet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 815-823. [Google Scholar] [CrossRef] 
[18] Deng, Y., Yang, J., Xu, S., Chen, D., Jia, Y. and Tong, X. (2019) Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, 16-17 June 2019, 285-295. [Google Scholar] [CrossRef] 
[19] Sanyal, S., Bolkart, T., Feng, H. and Black, M.J. (2019) Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 7755-7764. [Google Scholar] [CrossRef] 
[20] Li, X., Chen, C., Zhou, S., Lin, X., Zuo, W. and Zhang, L. (2020) Blind Face Restoration via Deep Multi-Scale Component Dictionaries. In: Vedaldi, A., Bischof, H., Brox, T. and Frahm, J.M., Eds., Computer Vision—ECCV 2020, Springer, 399-415. [Google Scholar] [CrossRef] 
[21] Wang, M., Deng, W., Hu, J., Tao, X. and Huang, Y. (2019) Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 692-702. [Google Scholar] [CrossRef] 
[22] Tran, A.T., Hassner, T., Masi, I., Paz, E., Nirkin, Y. and Medioni, G. (2018) Extreme 3D Face Reconstruction: Seeing through Occlusions. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 3935-3944. [Google Scholar] [CrossRef] 
[23] Yang, H., Zhu, H., Wang, Y., Huang, M., Shen, Q., Yang, R., et al. (2020) FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 598-607. [Google Scholar] [CrossRef] 
[24] Abrevaya, V.F., Boukhayma, A., Torr, P.H.S., et al. (2020) Cross-Modal Deep Face Normals with Deactivable Skip Connections. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 4979-4989.
[25] Feng, Y., Feng, H., Black, M.J. and Bolkart, T. (2021) Learning an Animatable Detailed 3D Face Model from In-The-Wild Images. ACM Transactions on Graphics, 40, 1-13. [Google Scholar] [CrossRef] 
[26] Feng, Y., Wu, F., Shao, X., Wang, Y. and Zhou, X. (2018) Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network. In: Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y., Eds., Computer Vision—ECCV 2018, Springer, 557-574. [Google Scholar] [CrossRef] 
[27] Chen, A., Chen, Z., Zhang, G., Mitchell, K. and Yu, J. (2019) Photo-Realistic Facial Details Synthesis from Single Image. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 9428-9438. [Google Scholar] [CrossRef] 

为你推荐