一种基于无监督学习的两阶段解耦图像去雾算法研究
A Two-Stage Disentanglement Image Dehazing Algorithm Based on Unsupervised Learning
DOI: 10.12677/csa.2024.144087, PDF,    国家自然科学基金支持
作者: 林 盛, 黎 敏, 胡 杰, 赵 丽*：温州大学计算机与人工智能学院，浙江省安全应急智能信息技术重点实验室，浙江 温州
关键词: 图像去雾；注意力机制；Transformer；无监督学习；Image Dehazing； Attention Mechanism； Transformer； Unsupervised Learning
摘要: 利用卷积神经网络进行监督学习是解决图像去雾问题的一种常用的解决方法。然而，现有的方法大多主要使用成对的合成雾霾数据集，这可能不能准确地代表真实雾霾天气的场景。针对这一问题，本文提出了一种基于无监督学习的两阶段解耦去雾网络。该网络由三个子网络组成，它们分别将观测到的雾霾图像分解为无雾霾图像层、透射图层和大气光层。同时，该网络分为恢复无雾图像和透射图两个阶段。在第一阶段，利用嵌入的暗通道先验来获得无雾霾像和透射图的粗略估计。在第二阶段，通过两个子网络对第一阶段的结果进行细化，以产生更精确的无雾霾图像和透射图，而大气光则由另一个子网络直接估计。此外，本文还设计了一种新的多尺度注意力模块，作为细化无雾图像的子网。多尺度注意力模块在自注意力中执行多尺度的标记聚合，以捕获不同尺度的特征。实验结果表明，本文提出的网络获得了有效的雾性能和令人满意的视觉效果，且在PSNR、SSIM和主观视觉效果方面均优于现有的无监督去雾方法。
Abstract: The use of convolutional neural networks for supervised learning is a commonly used method to solve the problem of image dehazing. However, most existing methods mainly use paired synthetic haze datasets, which may not accurately represent real-world haze weather scenarios. This article proposes a two-stage decoupling dehazing network based on unsupervised learning to address this issue. This network consists of three sub networks, which respectively decompose the observed haze images into a haze free image layer, a transmission layer, and an atmospheric light layer. Meanwhile, the network is divided into two stages: restoring fog free images and transmitting images. In the first stage, rough estimates of haze free images and transmission images are obtained using embedded dark channel priors. In the second stage, the results of the first stage are refined through two sub networks to produce more accurate haze free images and transmission maps, while atmospheric light is directly estimated by another sub network. In addition, this article also designs a new multi-scale attention module as a subnet for refining haze free images. The multi-scale attention module performs multi-scale label aggregation in self attention to capture features of different scales. The experimental results show that the proposed network achieves effective defogging performance and satisfactory visual effects, and is superior to existing unsupervised defogging methods in terms of PSNR, SSIM, and subjective visual effects.
文章引用：林盛, 黎敏, 胡杰, 赵丽. 一种基于无监督学习的两阶段解耦图像去雾算法研究[J]. 计算机科学与应用, 2024, 14(4): 163-176. https://doi.org/10.12677/csa.2024.144087

参考文献

[1] Huang, P., Zhao, L., Jiang, R., et al. (2021) Self-Filtering Image Dehazing with Self-Supporting Module. Neurocomputting, 432, 57-69. [Google Scholar] [CrossRef] 
[2] Li, X., Yu, H., Zhao, C., et al. (2023) DADRnet: Cross-Domain Image Dehazing via Domain Adaptation and Disentangled Representation. Neurocomputing, 544, Article 126242. [Google Scholar] [CrossRef] 
[3] Mccartney, E.J. and Hall, F.F. (1977) Optics of the Atmosphere: Scattering by Molecules and Particles. Physics Today, 30, 76-77. [Google Scholar] [CrossRef] 
[4] Nayar, S. and Narasimhan, S. (1999) Vision in Bad Weather. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, 20-27 September 1999, 820-827. [Google Scholar] [CrossRef] 
[5] Narasimhan, S.G. and Nayar, S.K. (2002) Vision and the Atmosphere. International Journal of Computer Vision, 48, 233-254. [Google Scholar] [CrossRef] 
[6] He, K., Sun, J. and Tang, X. (2009) Single Image Haze Removal Using Dark Channel Prior. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, 20-25 June 2009, 1956-1963. [Google Scholar] [CrossRef] 
[7] Zhu, Q., Mai, J. and Shao, L. (2015) A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior. IEEE Transactions on Image Processing, 24, 3522-3533. [Google Scholar] [CrossRef] 
[8] Cai, B., Xu, X., Jia, K., et al. (2016) DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Transactions on Image Processing, 25, 5187-5198. [Google Scholar] [CrossRef] 
[9] Zhang, H. and Patel, V.M. (2018) Densely Connected Pyramid Dehazing Network. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 3194-3203. [Google Scholar] [CrossRef] 
[10] Li, B., Peng, X., Wang, Z., et al. (2017) AOD-Net: All-in-one Dehazing Network. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 22-29 October 2017, 4780-4788. [Google Scholar] [CrossRef] 
[11] Song, Y., He, Z., Qian, H., et al. (2023) Vision Transformers for Single Image Dehazing. IEEE Transactions on Image Processing, 32, 1927-1941. [Google Scholar] [CrossRef] 
[12] Qin, X., Wang, Z., Bai, Y., et al. (2020) FFA-Net: Feature Fusion Attention Network for Single Image Dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 11908-11915. [Google Scholar] [CrossRef] 
[13] Song, Y., Zhou, Y., Qian, H., et al. (2022) Rethinking Performance Gains in Image Dehazing Networks. arXiv 2209.11448.
[14] Ren, W., Liu, S., Zhang, H., et al. (2016) Single Image Dehazing via Multi-Scale Convolutional Neural Networks. Computer Vision–ECCV 2016, Amsterdam, 11-14 October 2016, 154-169. [Google Scholar] [CrossRef] 
[15] Wang, Z., Cun, X., Bao, J., et al. (2022) Uformer: A General U-Shaped Transformer for Image Restoration. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 17662-17672. [Google Scholar] [CrossRef] 
[16] Fattal, R. (2015) Dehazing Using Color-Lines. ACM Transaction on Graphics, 34, Article No. 13. [Google Scholar] [CrossRef] 
[17] Berman, D., Treibitz, T. and Avidan, S. (2016) Non-Local Image Dehazing. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 1674-1682. [Google Scholar] [CrossRef] 
[18] Chen, D., He, M., Fan, Q., et al. (2019) Gated Context Aggregation Network for Image Dehazing and Deraining. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, 7-11 January 2019, 1375-1383. [Google Scholar] [CrossRef] 
[19] Li, R., Pan, J., Li, Z., et al. (2018) Single Image Dehazing via Conditional Generative Adversarial Network. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 8202-8211. [Google Scholar] [CrossRef] 
[20] Engin, D., Genc, A. and Kemal Ekenel, H. (2018) Cycle-Dehaze: Enhanced Cyclegan for Single Image Dehazing. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, 18-22 June 2018, 938-9388. [Google Scholar] [CrossRef] 
[21] Qu, Y., Chen, Y., Huang, J., et al. (2019) Enhanced Pix2pix Dehazing Network. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 8152-8160. [Google Scholar] [CrossRef] 
[22] Bai, H., Pan, J., Xiang, X., et al. (2022) Self-Guided Image Dehazing Using Progressive Feature Fusion. IEEE Transactions on Image Processing, 31, 1217-1229. [Google Scholar] [CrossRef] 
[23] Wu, H., Qu, Y., Lin, S., et al. (2021) Contrastive Learning for Compact Single Image Dehazing. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 10546-10555. [Google Scholar] [CrossRef] 
[24] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010.
[25] Zhao, D., Li, J., Li, H., et al. (2022) Complementary Feature Enhanced Network with Vision Transformer for Image Dehazing. arXiv 2109.07100.
[26] Liu, Z., Lin, Y., Cao, Y., et al. (2021) Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 10012-10022. [Google Scholar] [CrossRef] 
[27] Guo, C., Yan, Q., Anwar, S., et al. (2022) Image Dehazing Transformer with Transmission Aware 3D Position Embedding. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 5802-5810. [Google Scholar] [CrossRef] 
[28] Lempitsky, V., Vedaldi, A., Ulyanov, D. (2018) Deep Image Prior. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 9446-9454. [Google Scholar] [CrossRef] 
[29] Gandelsman, Y., Shocher, A., Irani, M. (2019) “Double-DIP”: Unsupervised Image Decomposition via Coupled Deep-Image-Priors. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 11018-11027. [Google Scholar] [CrossRef] 
[30] Li, B., Gou, Y., Gu, S., et al. (2020) You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network. International Journal of Computer Vision, 129, 1754-1767. [Google Scholar] [CrossRef] 
[31] Li, B., Gou, Y., Liu, J.Z., et al. (2020) Zero-Shot Image Dehazing. IEEE Transactions on Image Processing, 29, 8457-8466. [Google Scholar] [CrossRef] 
[32] Kar, A., Dhara, S.K., Sen, D., et al. (2021) Zero-Shot Single Image Restoration through Controlled Perturbation of Koschmieder’s Model. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 16200-16210. [Google Scholar] [CrossRef] 
[33] Ioffe, S. and Szegedy, C. (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, 6-11 July 2015, 448-456.
[34] Maas, A.L. (2013) Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30th International Conference on Machine Learning,
[35] Nair, V. and Hinton, G.E. (2010) Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, 21-24 June 2010, 807-814.
[36] Kingma, D. and Welling, M. (2013) Auto-Encoding Variational Bayes. International Conference on Learning Representations, Canada, 20 December 2013, 1-14.
[37] Ba, J., Kiros, J.R. and Hinton, G.E. (2016) Layer Normalization. ArXiv abs/1607.06450.
[38] Tarel, J.P., Hautière, N. (2009) Fast Visibility Restoration from a Single Color or Gray Level Image. 2009 IEEE 12th International Conference on Computer Vision, Kyoto, 29 September-2 October 2009, 2201-2208. [Google Scholar] [CrossRef] 
[39] Li, B., Ren, W., Fu, D., et al. (2019) Benchmarking Single-Image Dehazing and Beyond. IEEE Transactions on Image Processing, 28, 492-505. [Google Scholar] [CrossRef] 
[40] Wang, Z., Bovik, A., Sheikh, H., et al. (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13, 600-612. [Google Scholar] [CrossRef] 
[41] Kingma, D.P. and Ba, J. (2015) Adam: A Method for Stochastic Optimization. International Conference on Learning Representations, San Diego, 7-9 May 2015, 1-15.