基于图像融合和超分辨率重建的近红外图像彩色化
Colorization of Near-infrared Images Based on Image Fusion and Super-Resolution Reconstruction
DOI: 10.12677/SEA.2023.126085, PDF,    科研立项经费支持
作者: 杨 薪, 王文举：上海理工大学出版印刷与艺术学院，上海；孔玲君*：上海出版印刷高等专科学校印刷包装工程系，上海
关键词: 近红外图像；彩色化；融合；超分辨率；深度学习；Near-Infrared Images； Colorization； Fusion； Super-Resolution； Deep Learning
摘要: 为了提升近红外图像彩色化的图像质量和分辨率，提出通过图像融合和超分辨率重建相结合的方式实现近红外图像彩色化。首先，将近红外图像和RGB彩色图像输入到带有通道注意力机制模块的图像融合网络中得到融合后的彩色化图像，然后再通过亮度增强模块提升图像亮度，最后通过图像超分辨率重建提升图像分辨率。通过实验，将本文算法与其他算法进行对比，结果表明主观视觉上本文彩色化结果的图像质量和分辨率都更高，客观评价上，基于KAIST数据集的对比实验表明，本文算法的PSNR值为20.617，SSIM值为0.882，与其他算法相比，PSNR值和SSIM值分别提升了137.91%和87.66%；在IVRL RGB-NIR数据集上的PSNR值为18.295，SSIM值为0.796，与其他算法相比也都有提升。本文算法能够有效地提升近红外图像彩色化后的图像质量及分辨率，具有一定的实际应用意义。
Abstract: In order to improve the image quality and resolution of near-infrared image colorization, a combination of image fusion and super-resolution reconstruction is proposed to achieve near-infrared image colorization. Firstly, near-infrared images and RGB color images are input into an image fusion network with channel attention mechanism modules to obtain fused colored images. Then, the brightness of the image is enhanced through a brightness enhancement module, and finally, the image resolution is improved through image super-resolution reconstruction. Through experiments, the algorithm proposed in this paper was compared with other algorithms, and the results showed that the image quality and resolution of the colorization results in this paper were higher subjectively. Objectively, based on the KAIST dataset, comparative experiments showed that the PSNR value and SSIM value of the algorithm proposed in this paper were 20.617 and 0.882, respectively. Compared with other algorithms, the PSNR value and SSIM value increased by 137.91% and 87.66%, respectively; The PSNR value on the IVRL RGB-NIR dataset is 18.295, and the SSIM value is 0.796, which is also improved compared to other algorithms. The algorithm proposed in this paper can effectively improve the image quality and resolution of near-infrared images after colorization, and has certain practical application significance.
文章引用：杨薪, 孔玲君, 王文举. 基于图像融合和超分辨率重建的近红外图像彩色化[J]. 软件工程与应用, 2023, 12(6): 873-882. https://doi.org/10.12677/SEA.2023.126085

参考文献

[1] 胡建文, 汪泽平, 胡佩. 基于深度学习的空谱遥感图像融合综述[J]. 自然资源遥感, 2023, 35(1): 1-14.
[2] Zhang, H. and Ma, J. (2021) GTP-PNet: A Residual Learning Network Based on Gradient Transformation Prior for Pansharpening. ISPRS Journal of Photogrammetry and Remote Sensing, 172, 223-239. [Google Scholar] [CrossRef] 
[3] Jiang, M., Shen, H., Li, J., et al. (2020) A Differential Information Residual Convolutional Neural Network for Pansharpening. ISPRS Journal of Photogrammetry and Remote Sensing, 163, 257-271. [Google Scholar] [CrossRef] 
[4] He, L., Xi, D., Li, J., et al. (2020) A Spectral-Aware Convolutional Neural Network for Pansharpening. Applied Sciences, 10, Article No. 5809. [Google Scholar] [CrossRef] 
[5] Hu, J., Hu, P., Kang, X., et al. (2020) Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network. IEEE Transactions on Geoscience and Remote Sensing, 59, 2231-2244. [Google Scholar] [CrossRef] 
[6] Vitale, S. and Scarpa, G. (2020) A Detail-Preserving Cross-Scale Learning Strategy for CNN-Based Pansharpening. Remote Sensing, 12, Article No. 348. [Google Scholar] [CrossRef] 
[7] 黄珊珊, 江倩, 金鑫, 等. 结合双胞胎结构与生成对抗网络的半监督遥感图像融合[J]. 计算机辅助设计与图形学学报, 2021, 33(1): 92-105.
[8] Arjovsky, M., Chintala, S. and Bottou, L. (2017) Wasserstein Generative Adversarial Networks. International Conference on Machine Learning, Sydney, 6-11 August 2017, 214-223.
[9] Mirza, M. and Osindero, S. (2014) Conditional Generative Adversarial Nets.
[10] Zhou, C., Zhang, J., Liu, J., et al. (2020) PercepPan: Towards Unsupervised Pan-Sharpening Based on Perceptual Loss. Remote Sensing, 12, Article No. 2318. [Google Scholar] [CrossRef] 
[11] Luo, S., Zhou, S., Feng, Y., et al. (2020) Pansharpening via Unsupervised Convolutional Neural Networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 4295-4310. [Google Scholar] [CrossRef] 
[12] Li, H. and Wu, X.J. (2018) DenseFuse: A Fusion Approach to Infrared and Visible Images. IEEE Transactions on Image Processing, 28, 2614-2623. [Google Scholar] [CrossRef] 
[13] Lim, B., Son, S., Kim, H., et al. (2017) Enhanced Deep Residual Networks for Single Image Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 21-26 July 2017, 1132-1140. [Google Scholar] [CrossRef] 
[14] Ramachandran, P., Zoph, B. and Le, Q.V. (2017) Swish: A Self-Gated Activation Function.
[15] Hwang, S., Park, J., Kim, N., et al. (2015) Multispectral Pedestrian Detection: Benchmark Dataset and Baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 1037-1045. [Google Scholar] [CrossRef] 
[16] Brown, M. and Süsstrunk, S. (2011) Multi-Spectral SIFT for Scene Category Recognition. CVPR IEEE 2011, Colorado Springs, 20-25 June 2011, 177-184. [Google Scholar] [CrossRef] 
[17] Huynh-Thu, Q. and Ghanbari, M. (2008) Scope of Validity of PSNR in Image/Video Quality Assessment. Electronics Letters, 44, 800-801. [Google Scholar] [CrossRef] 
[18] Wang, Z., Bovik, A.C., Sheikh, H.R., et al. (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13, 600-612. [Google Scholar] [CrossRef] 
[19] Ronneberger, O., Fischer, P. and Brox, T. (2015) U-net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, 5-9 October 2015, 234-241. [Google Scholar] [CrossRef] 
[20] Limmer, M. and Lensch, H.P.A. (2016) Infrared Colorization Using Deep Convolutional Neural Networks. 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, 18-20 December 2016, 61-68. [Google Scholar] [CrossRef] 
[21] Sekiguchi, S. and Yamamoto, M. (2020) Near-Infrared Image Colorization by Convolutional Neural Network with Perceptual Loss. 2020 IEEE 9th Global Conference on Consumer Electronics (GCCE), Kobe, 13-16 October 2020, 88-89. [Google Scholar] [CrossRef] 
[22] Valsesia, D., Fracastoro, G. and Magli, E. (2020) NIR Image Colorization with Graph-Convolutional Neural Networks. 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), Macau, 1-4 December 2020, 451-454. [Google Scholar] [CrossRef] 

为你推荐