基于AutoFusion的高光谱图像循环反馈融合网络
Hyperspectral Image Cyclic Feedback Fusion Network Based on AutoFusion
DOI: 10.12677/mos.2025.146478, PDF,   
作者: 梅 景：上海理工大学光电信息与计算机工程学院，上海
关键词: 高光谱图像融合；深度学习；AutoFusion；循环反馈；GRU；Hyperspectral Image Fusion； Deep Learning； AutoFusion； Recurrent Feedback； GRU
摘要: 针对高光谱图像(HSI)与多光谱图像(MSI)融合中光谱信息整合与超分辨率性能的挑战，本文提出了一种基于AutoFusion和循环反馈的高光谱图像融合网络。该网络首先通过光谱分组对低分辨率HSI进行超分辨率重建，利用循环反馈机制迭代更新初始超分辨率图像，结合自适应融合模块(AutoFusion)动态融合多源信息，生成每组的高质量超分辨率特征。其次，通过特征提取网络分别处理融合特征和MSI特征，结合上采样和点卷积操作进一步优化特征表示。最后，将每组特征输入GRU模块生成注意力权重，用于加权调整各组超分辨率结果，重构最终的高分辨率高光谱图像(HR-HSI)。实验在CAVE和Harvard数据集上验证了Net模型，与七种主流融合算法相比，该网络在PSNR、SSIM和SAM等指标上表现优异，证明了其在高光谱图像融合领域的先进性。
Abstract: Aiming at the challenges of spectral information integration and super-resolution performance in the fusion of hyperspectral images (HSI) and multispectral images (MSI), this paper proposes a hyperspectral image fusion network based on AutoFusion and cyclic feedback. This network first performs super-resolution reconstruction of the low-resolution HSI through spectral grouping, iteratively updates the initial super-resolution image using the cyclic feedback mechanism, and dynamically fuses multi-source information in combination with the adaptive fusion module (AutoFusion) to generate high-quality super-resolution features for each group. Secondly, the fused features and MSI features are processed respectively through the feature extraction network, and the feature representation is further optimized by combining upsampling and dot convolution operations. Finally, each group of features is input into the GRU module to generate attention weights, which are used to weighted adjust the super-resolution results of each group and reconstruct the final high-resolution hyperspectral image (HR-HSI). The experiments verified the Net model on the CAVE and Harvard datasets. Compared with seven mainstream fusion algorithms, this network performed outstandingly in indicators such as PSNR, SSIM and SAM, proving its advancement in the field of hyperspectral image fusion.
文章引用：梅景. 基于AutoFusion的高光谱图像循环反馈融合网络[J]. 建模与仿真, 2025, 14(6): 84-98. https://doi.org/10.12677/mos.2025.146478

参考文献

[1] Ghamisi, P., Yokoya, N., Li, J., Liao, W., Liu, S., Plaza, J., et al. (2017) Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the Art. IEEE Geoscience and Remote Sensing Magazine, 5, 37-78. [Google Scholar] [CrossRef] 
[2] Lu, B., Dao, P., Liu, J., He, Y. and Shang, J. (2020) Recent Advances of Hyperspectral Imaging Technology and Applications in Agriculture. Remote Sensing, 12, Article 2659. [Google Scholar] [CrossRef] 
[3] Li, W., Wu, G., Zhang, F. and Du, Q. (2018) Target Detection in Hyperspectral Images Using Deep Convolutional Networks. IEEE Transactions on Geoscience and Remote Sensing, 56, 3410-3420.
[4] Loncan, L., de Almeida, L.B., Bioucas-Dias, J.M., Briottet, X., Chanussot, J., Dobigeon, N., et al. (2015) Hyperspectral Pansharpening: A Review. IEEE Geoscience and Remote Sensing Magazine, 3, 27-46. [Google Scholar] [CrossRef] 
[5] Wang, X., Hu, Q., Cheng, Y. and Ma, J. (2023) Hyperspectral Image Super-Resolution Meets Deep Learning: A Survey and Perspective. IEEE/CAA Journal of Automatica Sinica, 10, 1668-1691. [Google Scholar] [CrossRef] 
[6] Lanaras, C., Bioucas-Dias, J., Galliani, S., Baltsavias, E. and Schindler, K. (2023) A Review of Hyperspectral Image Su-per-Resolution Based on Deep Learning. Remote Sensing, 15, Article 2853.
[7] Yokoya, N., Yairi, T. and Iwasaki, A. (2012) Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion. IEEE Transactions on Geoscience and Remote Sensing, 50, 528-537. [Google Scholar] [CrossRef] 
[8] Dian, R., Li, S., Guo, A. and Fang, L. (2018) Deep Hyperspectral Image Fusion Using Convolutional Neural Networks. IEEE Transactions on Geoscience and Remote Sensing, 56, 7177-7188.
[9] Simoes, M., Bioucas-Dias, J., Almeida, L.B. and Chanussot, J. (2015) A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization. IEEE Transactions on Geoscience and Remote Sensing, 53, 3373-3388. [Google Scholar] [CrossRef] 
[10] Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S. and Du, Q. (2017) Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network. Remote Sensing, 9, Article 1139. [Google Scholar] [CrossRef] 
[11] Chen, Y., He, W., Yokoya, N. and Huang, T. (2020) Hyperspectral Image Restoration Using Weighted Group Sparsity-Regularized Low-Rank Tensor Decomposition. IEEE Transactions on Cybernetics, 50, 3556-3570. [Google Scholar] [CrossRef] [PubMed]
[12] Xing, C., Cong, Y., Wang, Z. and Wang, M. (2022) Fusion of Hyperspectral and Multispectral Images by Convolutional Sparse Representation. IEEE Geoscience and Remote Sensing Letters, 19, 1-5. [Google Scholar] [CrossRef] 
[13] Li, Y., Wu, X., Chu, J. and Wang, X. (2018) Hyperspectral and Multispectral Image Fusion Using Deep Convolutional Neural Network. IEEE Geoscience and Remote Sensing Letters, 15, 415-419.
[14] Yang, J., Zhao, Y. and Chan, J.C. (2018) Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network. Remote Sensing, 10, 800. [Google Scholar] [CrossRef] 
[15] He, X., Yao, J., Zhang, B., Li, M. and Pan, C. (2021) RAFNet: Recurrent Attention Fusion Network for Hyperspectral and Multispectral Image Fusion. IEEE Transactions on Geoscience and Remote Sensing, 59, 6747-6760.
[16] Zhang, L., Nie, J., Wei, W., Li, Y. and Zhang, Y. (2023) Reciprocal Transformer for Hyperspectral and Multispectral Image Fusion. Information Fusion, 93, 344-356.
[17] Sahu, G. and Vechtomova, O. (2021) Adaptive Fusion Techniques for Multimodal Data. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online, 19-23 April 2021, 3156-3166. [Google Scholar] [CrossRef] 
[18] Huynh-Thu, Q. and Ghanbari, M. (2008) Scope of Validity of PSNR in Image/Video Quality Assessment. Electronics Letters, 44, 800-801. [Google Scholar] [CrossRef] 
[19] Wang, Z., Bovik, A.C., Sheikh, H.R. and Simoncelli, E.P. (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13, 600-612. [Google Scholar] [CrossRef] [PubMed]
[20] Yuhas, R.H., Goetz, A.F.H. and Boardman, J.W. (1992) Discrimination among Semi-Arid Landscape Endmembers Using the Spectral Angle Mapper (SAM) Algorithm. Summaries of the Third Annual JPL Airborne Geoscience Workshop, Pasadena, 1-5 June 1992, 147-149.
[21] Willmott, C. and Matsuura, K. (2005) Advantages of the Mean Absolute Error (MAE) over the Root Mean Square Error (RMSE) in Assessing Average Model Performance. Climate Research, 30, 79-82. [Google Scholar] [CrossRef] 
[22] Wald, L. (1999) Some Terms of Reference in Data Fusion. IEEE Transactions on Geoscience and Remote Sensing, 37, 1190-1193. [Google Scholar] [CrossRef] 
[23] Lee Rodgers, J. and Nicewander, W.A. (1988) Thirteen Ways to Look at the Correlation Coefficient. The American Statistician, 42, 59-66. [Google Scholar] [CrossRef] 
[24] Uezato, T., Hong, D., Yokoya, N. and He, W. (2020) Guided Deep Decoder: Unsupervised Image Pair Fusion. In: Vedaldi, A., Bischof, H., Brox, T. and Frahm, J.M., Eds., Lecture Notes in Computer Science, Springer International Publishing, 87-102. [Google Scholar] [CrossRef] 
[25] Zhang, X., Huang, W., Wang, Q. and Li, X. (2021) SSR-NET: Spatial-Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion. IEEE Transactions on Geoscience and Remote Sensing, 59, 5953-5965. [Google Scholar] [CrossRef] 
[26] Zhang, L., Nie, J., Wei, W., Zhang, Y., Liao, S. and Shao, L. (2020) Unsupervised Adaptation Learning for Hyperspectral Imagery Super-Resolution. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 3070-3079. [Google Scholar] [CrossRef] 
[27] Liu, X., Liu, Q. and Wang, Y. (2020) Remote Sensing Image Fusion Based on Two-Stream Fusion Network. Information Fusion, 55, 1-15. [Google Scholar] [CrossRef] 
[28] Qu, J., Liu, X., Dong, W., Liu, Y., Zhang, T., Xu, Y., et al. (2024) Progressive Multi-Iteration Registration-Fusion Co-Optimization Network for Unregistered Hyperspectral Image Super-Resolution. IEEE Transactions on Geoscience and Remote Sensing, 62, 1-14. [Google Scholar] [CrossRef] 
[29] Yang, J., Xiao, L., Zhao, Y. and Chan, J.C. (2024) Unsupervised Deep Tensor Network for Hyperspectral-Multispectral Image Fusion. IEEE Transactions on Neural Networks and Learning Systems, 35, 13017-13031. [Google Scholar] [CrossRef] [PubMed]

为你推荐