基于GAN的超分辨率重建算法研究
A Study of Super Resolution Algorithm Basing on Generative Adversarial Network
DOI: 10.12677/CSA.2020.1010201, PDF,    科研立项经费支持
作者: 吴熹鹏*, 黄 溪, 王先兵#：武汉大学国家网络安全学院，湖北 武汉；何 涛：中国科学院计算技术研究所，北京；吴中鼎, 柴婉秋：贵阳铝镁设计研究院有限公司，贵州 贵阳
关键词: 图像超分辨率；生成对抗模型；密集卷积网络；深度学习；Super-Resolution； Generative Adversarial Network； DenseNet； Deep Learning
摘要: 图像超分辨率重建是图像复原的重要分支之一，算法任务是进行图像或者视频的修复或重建，目标是提高原始图像或视频帧的分辨率，目前很多超分辨率重建算法使用生成对抗模型来提升分辨率。本文拟采用密集型卷积网络对生成对抗模型的生成器部分进行改进，并提供一个将高层次特征与低层次特征联合起来的方法，将每一层的特征图传播到后续卷积层，从而在模型不同卷积层之间建立一条捷径，使得梯度信息在层次较深的网络模块中实现回传。然后在此基础上采用生成对抗网络生成高频信息，用来补充图像或视频帧细节，获得更加真实的高分辨率图像，达到提升重建结果的目标。
Abstract: Image super-resolution is one of the most important branches of image restoration, which is used for restoring or reconstructing images or videos, thus enhancing the resolution of the original images or frames of video. At present, most image super-resolution algorithms are using the generative adversarial network. In this paper, DenseNet is used to improve the generator part of the GAN model, and a method of combining high-level features and low-level features is provided to spread the feature map of each layer to the subsequent convolution layer, so as to establish a shortcut be-tween different convolution layers of the model, making gradient information in the deeper net-work module propagate back. On this basis, the high-frequency information is generated by the GAN, which is used to supplement the image or video frame details, and obtain more real high-resolution images, thus improving the reconstruction results.
文章引用：吴熹鹏, 黄溪, 王先兵, 何涛, 吴中鼎, 柴婉秋. 基于GAN的超分辨率重建算法研究[J]. 计算机科学与应用, 2020, 10(10): 1908-1920. https://doi.org/10.12677/CSA.2020.1010201

参考文献

[1] Zhou, F., Yang, W.M. and Liao, Q.M. (2012) Interpolation-Based Image Super-Resolution Using Multi-Surface Fitting. IEEE Transactions on Image Processing, 21, 3312-3318. [Google Scholar] [CrossRef] 
[2] Lin, Z.C. and Shum, H.Y. (2004) Fundamental Limits of Reconstruction Based Super Resolution Algorithms under Local Transla-tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 83-97. [Google Scholar] [CrossRef] 
[3] Pan, Z.-X., Yu, J., Hu, S.-X. and Sun, W.-D. (2014) Single Image Super Resolution Based on Multi-Scale Structural Self-Similarity. Acta Automatica Sinica, 40, 594-603.
[4] Lian, Q.-S., Shi, B.-S. and Chen, S.Z. (2015) Research Advances on Dictionary Learning Models, Algorithms and Applications. Acta Automatica Sinica, 41, 240-260.
[5] Ledig, C., Theis, L., Huszar, F., et al. (2016) Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 4680-4690. [Google Scholar] [CrossRef] 
[6] Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al. (2014) Generative Adversarial Nets. Advances in Neural Information Processing Systems, Montreal, 8-13 December 2014, 2672-2680.
[7] Dong, C., Loy, C.C., He, K., et al. (2014) Learning a Deep Convolutional Network for Image Su-per-Resolution. In: European Conference on Computer Vision, Springer, Cham, 184-199. [Google Scholar] [CrossRef] 
[8] Huang, G., Liu, Z., Weinberger, K.Q., et al. (2017) Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, Vol. 1, 3. [Google Scholar] [CrossRef] 
[9] Zhang, Y., Tian, Y., Kong, Y., et al. (2018) Residual Dense Network for Image Super-Resolution. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 2472-2481. [Google Scholar] [CrossRef] 
[10] Deng, J., Dong, W., Socher, R., et al. (2009) Imagenet: A Large-Scale Hierarchical Image Database. IEEE Conference on Computer Vision and Pattern Recognition, Miami, 20-25 June 2009, 248-255. [Google Scholar] [CrossRef] 
[11] Agustsson, E. and Timofte, R. (2017) Ntire 2017 Challenge on Single Image Super-Resolution: Dataset and Study. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, 21-26 July 2017, Vol. 3, 2. [Google Scholar] [CrossRef] 
[12] Kingma, D.P. and Ba, J. (2014) Adam: A Method for Stochastic Optimization.
[13] Bevilacqua, M., Roumy, A., Guillemot, C., et al. (2012) Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neighbor Embedding. Proceedings British Machine Vision Conference 2012, Surrey, 3-7 September 2012, 135.1-135.10. [Google Scholar] [CrossRef] 
[14] Zeyde, R., Elad, M. and Protter, M. (2010) On Single Image Scale-Up Using Sparse-Representations. In: International Conference on Curves and Surfaces, Springer, Berlin, Heidelberg, 711-730. [Google Scholar] [CrossRef] 
[15] Martin, D., Fowlkes, C., Tal, D., et al. (2001) A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. Eighth IEEE International Conference on Computer Vision, Vol. 2, 416-423.
[16] Huang, J.B., Singh, A. and Ahuja, N. (2015) Single Image Super-Resolution from Transformed Self-Exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 5197-5206. [Google Scholar] [CrossRef] 
[17] De Boor, C. (1962) Bicubic Spline Interpolation. Journal of Mathematics & Physics, 41, 212-218. [Google Scholar] [CrossRef] 
[18] Dong, C., Chen, C.L., He, K., et al. (2016) Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 38, 295-307. [Google Scholar] [CrossRef] 
[19] Yang, J.C., Wright, J., Huang, T.S. and Ma, Y. (2010) Image Superresolution via Sparse Representation. IEEE Transactions on Image Processing, 19, 2861-2873. [Google Scholar] [CrossRef]