基于深度学习的单图超分辨率研究
Deep Learning-Based Single Image Super-Resolution Research
DOI: 10.12677/jisp.2024.134038, PDF,    科研立项经费支持
作者: 廖可心, 陆利坤, 曾庆涛, 李超超, 王 彤：北京印刷学院学院信息工程学院，北京
关键词: 图像超分辨率；深度学习；评价指标；Image Super-Resolution； Deep Learning； Evaluation Metrics
摘要: 图像超分辨率重建(Image Super-Resolution)是一种计算机视觉技术，其目标是将一张低分辨率图像(LR)恢复成高分辨率图像(SR)，以达到提高图像质量、改善人眼视觉效果的目的。随着科学技术的发展，深度学习与图像超分辨率重建结合极大地提升了图像处理的能力和效果。其中，单图像超分辨率(Single Image Super-Resolution, SISR)是指从单张低分辨率图像生成高分辨率图像的技术，单图超分辨率旨在从一张低分辨率的图像生成对应的高分辨率图像，而不依赖于额外的信息或多张图像。本文介绍了图像超分辨率的背景以及发展过程，对近年单图超分辨率重建方法进行了比较，并讨论了单图超分辨率面临的挑战。
Abstract: Image Super-Resolution (SR) is a computer vision technique aimed at reconstructing a high-resolution image (HR) from a low-resolution image (LR), with the goal of improving image quality and enhancing visual experience. With advancements in science and technology, the combination of deep learning with image super-resolution has significantly enhanced image processing capabilities and results. Single Image Super-Resolution (SISR) refers to the technique of generating a high-resolution image from a single low-resolution image, focusing on recovering the corresponding high-resolution image without relying on additional information or multiple images. This paper provides an overview of the background and development of image super-resolution, compares recent methods for single image super-resolution reconstruction, and discusses the challenges faced by single image super-resolution.
文章引用：廖可心, 陆利坤, 曾庆涛, 李超超, 王彤. 基于深度学习的单图超分辨率研究[J]. 图像与信号处理, 2024, 13(4): 440-456. https://doi.org/10.12677/jisp.2024.134038

参考文献

[1] Yuan, H.L., Zhang, B.Y., Zhu, M.Y., Liu, L.G. and Wang, J. (2023) High-Quality Supersampling via Mask-Reinforced Deep Learning for Real-Time Rendering.
[2] Ulyanov, D., Vedaldi, A. and Lempitsky, V. (2018) Deep Image Prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 18-23 June 2018, 9446-9454.
[3] de Leeuw den Bouter, M.L., Ippolito, G., O’Reilly, T.P.A., Remis, R.F., van Gijzen, M.B. and Webb, A.G. (2022) Deep Learning-Based Single Image Super-Resolution for Low-Field MR Brain Images. Scientific Reports, 12, Article No. 6362. [Google Scholar] [CrossRef] [PubMed]
[4] Lu, T., Wang, J., Zhang, Y., Wang, Z. and Jiang, J. (2019) Satellite Image Super-Resolution via Multi-Scale Residual Deep Neural Network. Remote Sensing, 11, Article No. 1588. [Google Scholar] [CrossRef] 
[5] Cao, J., Jia, Y., Yan, M. and Tian, X. (2021) Superresolution Reconstruction Method for Ancient Murals Based on the Stable Enhanced Generative Adversarial Network. EURASIP Journal on Image and Video Processing, 2021, Article No. 28. [Google Scholar] [CrossRef] 
[6] Dong, C., Loy, C.C., He, K. and Tang, X. (2016) Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307. [Google Scholar] [CrossRef] [PubMed]
[7] Xin Li, and Orchard, M.T. (2001) New Edge-Directed Interpolation. IEEE Transactions on Image Processing, 10, 1521-1527. [Google Scholar] [CrossRef] [PubMed]
[8] Chen, M., Huang, C. and Lee, W. (2005) A Fast Edge-Oriented Algorithm for Image Interpolation. Image and Vision Computing, 23, 791-798. [Google Scholar] [CrossRef] 
[9] Bresenham, J.E. (1998) Algorithm for Computer Control of a Digital Plotter. In: Seminal Graphics: Pioneering Efforts That Shaped the Field, Association for Computing Machinery, 1-6. [Google Scholar] [CrossRef] 
[10] Keys, R. (1981) Cubic Convolution Interpolation for Digital Image Processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 29, 1153-1160. [Google Scholar] [CrossRef] 
[11] Tsai, R.Y. and Huang, T.S. (1984) Multiframe Image Restoration and Registration. Advances in Computer Vision and Image Processing, 1, 317-339.
[12] Dai, S., Han, M., Wu, Y. and Gong, Y. (2007). Bilateral Back-Projection for Single Image Super Resolution. 2007 IEEE International Conference on Multimedia and Expo, Beijing, 2-5 July 2007, 1039-1042.[CrossRef] 
[13] Zhang, H., Zhang, Y., Li, H. and Huang, T.S. (2012) Generative Bayesian Image Super Resolution with Natural Image Prior. IEEE Transactions on Image Processing, 21, 4054-4067. [Google Scholar] [CrossRef] [PubMed]
[14] Chang, H., Yeung, D.Y. and Xiong, Y. (2004) Super-Resolution through Neighbor Embedding. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington DC, 27 June-2 July 2004, 1.
[15] Yang, J.C., Wright, J., Huang, T.S. and Ma, Y. (2010) Image Super-Resolution via Sparse Representation. IEEE Transactions on Image Processing, 19, 2861-2873. [Google Scholar] [CrossRef] [PubMed]
[16] Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A.P., Bishop, R., et al. (2016) Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 1874-1883. [Google Scholar] [CrossRef] 
[17] Kim, J., Lee, J.K. and Lee, K.M. (2016) Accurate Image Super-Resolution Using Very Deep Convolutional Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 1646-1654. [Google Scholar] [CrossRef] 
[18] Bengio, Y., Simard, P. and Frasconi, P. (1994) Learning Long-Term Dependencies with Gradient Descent Is Difficult. IEEE Transactions on Neural Networks, 5, 157-166. [Google Scholar] [CrossRef] [PubMed]
[19] Lai, W., Huang, J., Ahuja, N. and Yang, M. (2017) Deep Laplacian Pyramid Networks for Fast and Accurate Super-resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 624-632. [Google Scholar] [CrossRef] 
[20] Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017) Enhanced Deep Residual Networks for Single Image Super-resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 21-26 July 2017, 136-144. [Google Scholar] [CrossRef] 
[21] Nah, S., Kim, T.H. and Lee, K.M. (2017) Deep Multi-Scale Convolutional Neural Network for Dynamic Scene Deblurring. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 3883-3891. [Google Scholar] [CrossRef] 
[22] Ahn, N., Kang, B. and Sohn, K. (2018) Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network. In: Ferrari, V., et al., Eds., Computer Vision—ECCV 2018, Springer International Publishing, 256-272. [Google Scholar] [CrossRef] 
[23] Howard, A.G. (2017) Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications.
[24] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B. and Fu, Y. (2018) Image Super-Resolution Using Very Deep Residual Channel Attention Networks. In: Ferrari, V., et al., Eds., Computer Vision—ECCV 2018, Springer International Publishing, 294-310. [Google Scholar] [CrossRef] 
[25] Tong, T., Li, G., Liu, X. and Gao, Q. (2017) Image Super-Resolution Using Dense Skip Connections. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 22-29 October 2017, 4799-4807. [Google Scholar] [CrossRef] 
[26] Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., et al. (2017) Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 4681-4690. [Google Scholar] [CrossRef] 
[27] Zhou, S., Zhang, J., Zuo, W., et al. (2020) Cross-Scale Internal Graph Neural Network for Image Super-Resolution. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6-12 December 2020, 3499-3509.
[28] Gu, S., Zhang, L., Zuo, W. and Feng, X. (2014) Weighted Nuclear Norm Minimization with Application to Image Denoising. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 2862-2869. [Google Scholar] [CrossRef] 
[29] Mairal, J., Bach, F., Ponce, J., Sapiro, G. and Zisserman, A. (2009) Non-Local Sparse Models for Image Restoration. 2009 IEEE 12th International Conference on Computer Vision, Kyoto, 29 September-2 October 2009, 2272-2279. [Google Scholar] [CrossRef] 
[30] Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. and Hovy, E. (2016) Hierarchical Attention Networks for Document Classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, June 2016, 1480-1489. [Google Scholar] [CrossRef] 
[31] Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W. and Hu, Q. (2020). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13-19 June 2020.[CrossRef] 
[32] Hu, J., Shen, L. and Sun, G. (2018) Squeeze-and-Excitation Networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 7132-7141. [Google Scholar] [CrossRef] 
[33] Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L. and Timofte, R. (2021) Swinir: Image Restoration Using Swin Transformer. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 11-17 October 2021, 1833-1844. [Google Scholar] [CrossRef] 
[34] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., et al. (2021) Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 10012-10022. [Google Scholar] [CrossRef] 
[35] Chen, X., Wang, X., Zhang, W., et al. (2023) Hat: Hybrid Attention Transformer for Image Restoration.
[36] Chen, X., Wang, X., Zhou, J., Qiao, Y. and Dong, C. (2023) Activating More Pixels in Image Super-Resolution Transformer. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, 18-22 June 2023, 22367-22377. [Google Scholar] [CrossRef] 
[37] Gu, J. and Dong, C. (2021) Interpreting Super-Resolution Networks with Local Attribution Maps. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 9199-9208. [Google Scholar] [CrossRef] 
[38] Conde, M.V., Choi, U., Burchi, M. and Timofte, R. (2023) Swin2sr: Swinv2 Transformer for Compressed Image Super-Resolution and Restoration. Computer Vision—ECCV 2022 Workshops, Tel Aviv, 23-27 October 2022, 669-687. [Google Scholar] [CrossRef] 
[39] Li, K., Wang, Y., Zhang, J., Gao, P., Song, G., Liu, Y., et al. (2023) Uniformer: Unifying Convolution and Self-Attention for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 12581-12600. [Google Scholar] [CrossRef] [PubMed]
[40] Agustsson, E. and Timofte, R. (2017) NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 21-26 July 2017, 126-135. [Google Scholar] [CrossRef] 
[41] 方玉明, 眭相杰, 鄢杰斌, 等. 无参考图像质量评价研究进展[J]. 中国图象图形学报, 2021, 26(2): 265-286.