基于StyleGAN的草图到服装图像的生成方法
A Sketch-to-Clothing Image Generation Method Based on StyleGAN
DOI: 10.12677/CSA.2022.1210246, PDF,   
作者: 胡新荣, 陈 雷, 罗瑞奇*, 刘军平, 梁金星, 彭 涛, 李 丽, 罗 航：武汉纺织大学计算机与人工智能学院，湖北 武汉
关键词: 草图；服装；图像生成；特征融合；风格向量；Sketching； Clothing； Image Generation； Feature Fusion； Style Vectors
摘要: 草图生成服装图像被广泛应用在服装设计过程中，可直接展示服装的设计效果，节约大量成本。但在当前，普遍利用条件变量实现服装草图的图形化渲染或通过学习图像与草图的公共属性向量去映射图像等方法，生成的服装图案受草图约束严重，生成的图像深度信息缺失。因此本文针对上述问题提出了一种基于StyleGAN的从草图到服装图像的图像生成方法。首先，利用VGG网络，对多种被变换的草图图像，进行内容特征提取。有针对性的提取目标特征，加深其内容特征，降低其他特征图信息对生成图像的约束；其次，利用特征金字塔的多尺度的特征融合方法，通过小型映射网络生成中间风格向量，利用StyleGAN生成器，丰富图像深度信息。实验结果表明我们的方法在草图生成服装图像方面具有一定的优越性，可以更完整地保留草图的细节，生成图像的质量更佳。
Abstract: Sketch-generated garment images are widely used in the garment design process, which can directly show the design effect of garments and save a lot of costs. At present, it is common to use conditional variables to achieve graphical rendering of garment sketches or to map images by learning the common attribute vectors of images and sketches, etc. The generated garment patterns are severely constrained by sketches and the depth information of the generated images is missing. Therefore, this paper proposes a StyleGAN-based image generation method from sketches to garment images to address the above problems. First, using the VGG network, a variety of sketch images has been transformed, and the content features are extracted. Targeted extraction of target features deepens their content features and reduces the constraint of other feature map information on the generated images. Second, a multi-scale feature fusion method using a feature pyramid generates intermediate style vectors through a small mapping network and enriches the image depth information using a StyleGAN generator. The experimental results show the superiority of our method in the sketch generation of garment images, which can retain the details of sketches more completely and generate images with better quality.
文章引用：胡新荣, 陈雷, 罗瑞奇, 刘军平, 梁金星, 彭涛, 李丽, 罗航. 基于StyleGAN的草图到服装图像的生成方法[J]. 计算机科学与应用, 2022, 12(10): 2405-2415. https://doi.org/10.12677/CSA.2022.1210246

参考文献

[1] Chen, W.L. and Hays, J. (2018) Sketchygan: Towards Diverse and Realistic Sketch to Image Synthesis. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-23 June 2018, 9416-9425. [Google Scholar] [CrossRef] 
[2] Lu, Y.Y., Wu, S.Z., Tai, Y., et al. (2018) Image Generation from Sketch Constraint Using Contextual Gan. In: Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y., Eds., Proceedings of the 2018 European Conference on Computer Vision, Springer Press, Berlin, 205-220. [Google Scholar] [CrossRef] 
[3] Huang, X., Liu, M.Y., Belongie, S. and Kautz, J. (2018) Mul-timodal Unsupervised Image-to-Image Translation. In: Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y., Eds., Computer Vision—ECCV 2018, Springer, Cham. [Google Scholar] [CrossRef] 
[4] Gao, C., Liu, Q., Xu, Q., Wang, L., Liu, J. and Zou, C. (2020) SketchyCOCO: Image Generation from Freehand Scene Sketches. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13-19 June 2020, 5173-5182. [Google Scholar] [CrossRef] 
[5] Richardson, E., et al. (2021) Encoding in Style: A Style-GAN Encoder for Image-to-Image Translation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogni-tion (CVPR), Nashville, TN, USA, 20-25 June 2021, 2287-2296. [Google Scholar] [CrossRef] 
[6] Karras, T., Samuli, L., Aittala, M., Hellsten, J., Lehtinen, J. and Timo, A. (2020) Analyzing and Improving the Image Quality of StyleGAN. 2020 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13-19 June 2020, 8107-8116. [Google Scholar] [CrossRef] 
[7] Simonyan, K. and Zisserman, A. (2014) Very Deep Convo-lutional Networks for Large-Scale Image Recognition.
https://arxiv.org/abs/1409.1556
[8] Mirza, M. and Osindero, S. (2014) Conditional Generative Adversarial Nets.
https://arxiv.org/abs/1411.1784
[9] Zhu, J.Y., Park, T., Isola, P., et al. (2017) Unpaired Image-to-Image Transla-tion Using Cycle-Consistent Adversarial Networks. 2017 IEEE International Conference on Computer Vision, Venice, Italy, 22-29 October 2017, 2242-2251. [Google Scholar] [CrossRef] 
[10] Cui, Y.R., Liu, Q., Gao, C. and Su, Z. (2018). FashionGAN: Display Your Fashion Design Using Conditional Generative Adversarial Nets. Computer Graphics Forum, 37, 109-119.[CrossRef] 
[11] Liu, B., Zhu, Y., Song, K. and Elgammal, A. (2021) Self-Supervised Sketch-to-Image Synthesis. Proceedings of the AAAI Conference on Artificial Intelligence, 35, 2073-2081. [Google Scholar] [CrossRef] 
[12] Kingma, D.P. and Welling, M. (2014) Auto-Encoding Variational Bayes.
https://arxiv.org/abs/1312.6114
[13] Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B. and Be-longie, S. (2017) Feature Pyramid Networks for Object Detection. 2017 IEEE Conference on Computer Vision and Pat-tern Recognition (CVPR), Honolulu, HI, USA, 21-26 July 2017, 936-944. [Google Scholar] [CrossRef] 
[14] Zhang, R., Isola, P., Efros, A., et al. (2018) The Unreasonable Effec-tiveness of Deep Features as a Perceptual Metric. 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-23 June 2018, 586-595. [Google Scholar] [CrossRef] 
[15] Johnson, J., Alahi, A. and Li, F.-F. (2016) Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In Leibe, B., Matas, J., Sebe, N. and Welling, M., Eds., 2016 European Conference on Computer Vision, Springer, Cham, 694-711. [Google Scholar] [CrossRef] 
[16] Guan, S.Y., Tai, Y., Ni, B.B., Zhu, F.D., Huang, F.Y. and Yang, X.K. (2007) Collaborative Learning for Faster StyleGAN Embedding.
https://arxiv.org/abs/2007.01758
[17] Heusel, M., Ramsauer, H., Unterthiner, T., et al. (2017) GANs Trained by a Two Timescale Update Rule Converge to a Local Nash Equilibrium. Proceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 4-9 December 2017, 6626-6637.
https://dl.acm.org/doi/10.5555/3295222.3295408
[18] 王建欣, 史英杰, 刘昊, 黄海峤, 杜方. 基于GAN的手绘草图图像翻译研究综述[J/OL]. 计算机应用研究: 1-10.
https://kns.cnki.net/kcms/detail/detail.aspx?doi=10.19734/j.issn.1001-3695.2022.01.0027, 2022-06-27.
[19] Chen, S.-Y., Su, W.C., Gao, L., Xia, S.H. and Fu, H.B. (2020) DeepFaceDrawing: Deep Generation of Face Images from Sketches. ACM Transactions on Graphics, 39, Article No. 72. [Google Scholar] [CrossRef]