基于DECA的单目人脸三维重建研究
Research on Three-Dimensional Reconstruction of Monocular Face Based on DECA
DOI: 10.12677/CSA.2023.1312248, PDF,    科研立项经费支持
作者: 王承伟, 王振凯, 李昊渊, 张一帆：河北地质大学信息工程学院，河北 石家庄；张翠军*：河北地质大学信息工程学院，河北 石家庄;河北地质大学人工智能与机器学习研究室，河北 石家庄
关键词: 人脸重建；深度学习；Vision Transformer；DropKey；Face Reconstruction； Deep Learning； Vision Transformer； DropKey
摘要: 针对基于单目图像的DECA模型在人脸三维重建时精度不高，且容易出现过拟合的问题，提出用Vision Transformer (ViT)改进DECA模型的特征提取器部分，增强模型的局部和全局理解能力，提取更高维的特征，以提高人脸特征点的检测精度和人脸重建的精确性。进一步，引入DropKey策略，将ViT中的Key作为Drop对象，惩罚注意力峰值，以改善训练过程中的过拟合问题。实验结果表明，在引入ViT和DropKey策略后，人脸三维重建的效果有明显的提升。
Abstract: In view of the low accuracy of the monocular image-based DECA model in face 3D reconstruction and the problem of overfitting, Vision Transformer (ViT) is proposed to replace the feature extractor part of the DECA model to enhance the local and global understanding ability of the model and extract higher-dimensional features. To improve the accuracy of face feature point detection and face reconstruction. Further, the DropKey strategy is introduced, and the Key in ViT is used as Drop object to punish the attention peak, so as to improve the overfitting problem in the training process. The experimental results show that after the introduction of ViT and DropKey strategies, the effect of face 3D reconstruction has been significantly improved.
文章引用：王承伟, 张翠军, 王振凯, 李昊渊, 张一帆. 基于DECA的单目人脸三维重建研究[J]. 计算机科学与应用, 2023, 13(12): 2500-2508. https://doi.org/10.12677/CSA.2023.1312248

参考文献

[1] 林琴, 李卫军, 董肖莉, 宁欣, 陈鹏. 基于双目视觉的人脸三维重建[J]. 智能系统学报, 2018, 13(4): 534-542.
[2] Castelán, M. and Hancock, E.R. (2004) Acquiring Height Maps of Faces from a Single Image. Proceedings 2nd International Symposium on 3D Data Processing, Visualization and Transmission, Thessaloniki, 6-9 September 2004, 183-190.
[3] 叶于平. 高真实感人脸表情动态三维重建及迁移方法研究[D]: [博士学位论文]. 深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2022.
[4] Blanz, V. and Vetter, T. (1999) A Morphable Model for the Synthesis of 3D Faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, 8-13 August 1999, 187-194. [Google Scholar] [CrossRef] 
[5] Cao, C., Weng, Y., Zhou, S., et al. (2013) FaceWarehouse: A 3D Facial Expression Database for Visual Computing. IEEE Transactions on Visualization and Computer Graphics, 20, 413-425. [Google Scholar] [CrossRef] 
[6] Tuan Tran, A., Hassner, T., Masi, I., et al. (2017) Regressing Robust and Discrim-inative 3D Morphable Models with a Very Deep Neural Network. Proceedings of the IEEE Conference on Computer Vision and Pat-tern Recognition, Honolulu, 21-26 July 2017, 5163-5172. [Google Scholar] [CrossRef] 
[7] He, K.M., et al. (2016) Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. [Google Scholar] [CrossRef] 
[8] Zhu, X.Y., et al. (2016) Face Alignment across Large Poses: A 3D Solution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 146-155. [Google Scholar] [CrossRef] 
[9] Feng, Y., Wu, F., Shao, X., et al. (2018) Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Net-Work. Proceedings of the European Conference on Computer Vision (ECCV), Munich, 8-14 September 2018, 534-551. [Google Scholar] [CrossRef] 
[10] Guo, J.Z., et al. (2020) Towards Fast, Ac-curate and Stable 3D Dense Face Alignment. European Conference on Computer Vision, Glasgow, 23-28 August 2020, 152-168. [Google Scholar] [CrossRef] 
[11] Li, T., Bolkart, T., Black, M.J., et al. (2017) Learning a Model of Facial Shape and Expression from 4D Scans. ACM Transactions on Graphics, 36, Article No. 194. [Google Scholar] [CrossRef] 
[12] Feng, Y., et al. (2021) Learning an Animatable Detailed 3D Face Model from In-the-Wild Images. ACM Transactions on Graphics, 40, Article No. 88. [Google Scholar] [CrossRef] 
[13] Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2010) An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale.
[14] Li, B., Hu, Y., Nie, X., et al. (2023) DropKey for Vision Trans-former. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 18-22 June 2023, 22700-22709. [Google Scholar] [CrossRef] 
[15] NoW Challenge (2019)
https://now.is.tue.mpg.de/
[16] Cao, Q., Shen, L., et al. (2018) Vggface2: A Dataset for Recognizing Faces across Pose and Age. 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, 15-19 May 2018, 67-74. [Google Scholar] [CrossRef] 
[17] Wang, M., Deng, W., Hu, J., et al. (2019) Racial Faces in the Wild: Reducing racial Bias by Information Maximization Adaptation Network. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, 27 October-2 November 2019, 692-702. [Google Scholar] [CrossRef] 
[18] Karras, T., Laine, S., Aittala, M., et al. (2020) Analyzing and Improving the Image Quality of StyleGAN. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 8110-8119. [Google Scholar] [CrossRef] 
[19] Bulat, A. and Tzimiropoulos, G. (2017) How Far Are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks). Proceedings of the IEEE International Con-ference on Computer Vision, Venice, 22-29 October 2017, 1021-1030. [Google Scholar] [CrossRef] 
[20] Nirkin, Y., Masi, I., Tuan, A.T., et al. (2018) On Face Segmentation, Face Swapping, and Face Perception. 2018 13th IEEE International Confer-ence on Automatic Face & Gesture Recognition (FG 2018), Xi’an, 15-19 May 2018, 98-105. [Google Scholar] [CrossRef]