基于CNN和Transformer的轻量级超分辨率重建网络研究

Research on Lightweight Super-Resolution Network Based on CNN and Transformer

摘要: 随着深度学习的发展，单图像超分辨率技术取得了长足的进步。然而，现有的大多数研究都专注于卷积神经网络来构建具有大量层数的更深层次的网络模型。这些方法难以应用于现实场景，因为它们不可避免的伴随着复杂操作所带来的计算和内存成本问题。为此，我们提出了一种用于单图像超分辨率重建的轻量级混合模型——轻量级融合CNN-Swin Transformer网络。具体来说，我们使用带有移动窗口的Swin Transformer块充分学习图像的长期依赖性，并构建了一个基于CNN的局部特征提取块来有效地提取图像的局部特征细节。同时，设计了一个多路径动态卷积块来学习图像的边缘特征。实验结果表明，与基于Transformer的单图像超分辨率模型相比，本文提出的模型取得了更好的结果。

Abstract: With the development of deep learning, single image super-resolution technology has made great progress. However, most of the existing research focuses on convolutional neural networks to construct deeper network models with a large number of layers. These methods are difficult to apply to real-world scenarios because they inevitably come with computational and memory costs associated with complex operations. Therefore, we propose a lightweight hybrid model for super-resolution reconstruction of single image—lightweight fusion CNN-Swin Transformer network. Specifically, we use Swin Transformer block with shifted windows to fully learn the long-term dependence of the image, and build a CNN-based local feature extraction block to effectively extract the local feature details of the image. Meanwhile, a multipath dynamic convolution block is designed to learn the edge features of the image. Experimental results show that compared with the single image super-resolution model based on Transformer, the proposed model achieves better results.