基于三维的人脸合成与替换系统实现
Realization of 3D-Based Face Synthesis and Replacement System
DOI: 10.12677/SEA.2022.113056, PDF,  被引量   
作者: 钟卓岑, 史泽奇：浙江理工大学信息学院，浙江 杭州
关键词: 人脸替换；人脸合成；Face Replacement； Face Synthesis
摘要: 在保持人物身份的前提下对人脸合成，也称为人脸替换，目的将源图像中的人脸无缝且逼真地融合到目标图像。本文提出了一种自动化的人脸合成与替换系统，利用带有标签的图像合成用户感兴趣的全新形象，实现如虚拟试发、虚拟试衣的应用。系统的输入为单张人脸图像和查询文本，其中查询文本用于搜索相关图像，构建待替换的目标图像集合。为提高人脸合成效果，本文首先通过人脸图像重建三维人脸模型，在三维空间完成了人脸的姿态与表情迁移，然后通过人脸图像分解估计环境光照，实现了人脸图像的重光照，最后通过图像融合技术实现人脸的图像融合。实验证明我们的方法可以合成较为真实的人脸图像。
Abstract: Face synthesis, also known as face replacement, while maintaining the identity of the person, aims to seamlessly and realistically fuse the face in the source image to the target image. This paper proposes an automated face synthesis and replacement system, which uses labeled images to synthesize new images that users are interested in, and realizes applications such as virtual hair test and virtual fitting. The input of the system is a single face image and query text, where the query text is used to search for relevant images and construct a set of target images to be replaced. In order to improve the effect of face synthesis, this paper firstly reconstructs a 3D face model through face images, completes the face pose and expression migration in 3D space, and then estimates the ambient lighting through face image decomposition to realize the re-illumination of face images, and finally realize the image fusion of the face through the image fusion technology. Experiments show that our method can synthesize more realistic face images.
文章引用：钟卓岑, 史泽奇. 基于三维的人脸合成与替换系统实现[J]. 软件工程与应用, 2022, 11(3): 538-548. https://doi.org/10.12677/SEA.2022.113056

参考文献

[1] Blanz, V. and Vetter, T. (1999) A Morphable Model for the Synthesis of 3d Faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, 8-13 August 1999, 187-194. [Google Scholar] [CrossRef] 
[2] Zhang, K., Zhang, Z., Li, Z., et al. (2016) Joint Face Detection and Alignment Using Multitask Cascaded Convolu-Tional Networks. IEEE Signal Processing Letters, 23, 1499-1503. [Google Scholar] [CrossRef] 
[3] Kaehler, A. and Bradski, G. (2016) Learning OpenCV 3: Computer Vision in C++ with the OpenCV library. O’Reilly Media, Inc., Newton.
[4] Lee, C.H., Liu, Z., Wu, L., et al. (2020) Maskgan: Towards Diverse and Interactive Facial Image Manipulation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 5549-5558. [Google Scholar] [CrossRef] 
[5] Yu, C., Wang, J., Peng, C., et al. (2018) Bisenet: Bilateral Segmentation Network for Real-Time Semantic Segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y., (Eds.), Computer Vision-ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, Heidelberg, Cham, 325-341. [Google Scholar] [CrossRef] 
[6] Fan, M., Lai, S., Huang, J., et al. (2021) Rethinking Bisenet for Real-Time Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 9716-9725. [Google Scholar] [CrossRef] 
[7] Zhu, X., Liu, X., Lei, Z., et al. (2017) Face Alignment in Full Pose Range: A 3d Total Solution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 78-92. [Google Scholar] [CrossRef] 
[8] Frolova, D., Simakov, D. and Basri, R. (2004) Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting. In: Pajdla, T. and Matas, J., (Eds.), Computer Vision-ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, Springer, Heidelberg, 574-587. [Google Scholar] [CrossRef] 
[9] Sengupta, S., Kanazawa, A., Castillo, C.D., et al. (2018) Sfsnet: Learning Shape, Reflectance and Illuminance of Facesin the Wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 6296-6305. [Google Scholar] [CrossRef] 
[10] Willman, J.M. (2022) Creating Custom Widgets. Beginning PyQt. Apress, Berkeley, 341-358. [Google Scholar] [CrossRef] 

为你推荐