图像语义分割方法研究进展
Recent Progress in Semantic Image Segmentation
DOI: 10.12677/JISP.2023.121007, PDF,  被引量    国家自然科学基金支持
作者: 汤婧婧, 章 盛, 徐立中*：河海大学计算机与信息学院，江苏 南京
关键词: 机器学习；计算机视觉；深度学习；语义分割；Machine Learning； Computer Vision； Deep Learning； Semantic Segmentation
摘要: 图像语义分割任务是将图像的每个像素分类到每一个实例中，每个实例对应一个类。该任务是场景理解概念的一部分，由于深度学习的图像语义分割方法能更好地解释图像的全局上下文，越来越受到计算机视觉和机器学习研究者的关注，并广泛应用于室内导航、自动驾驶，甚至虚拟或增强现实系统等领域。本文介绍图像语义分割的术语的概念，回顾传统和现有的深度学习方法，强调了它们在该领域的贡献和意义，以及语义分割算法的评价指标与常用数据集，最后，我们对当前语义图像分割任务中存在的一些问题进行讨论，并提出相关解决方法和研究展望。
Abstract: The semantic image segmentation task consists of classifying each pixel of an image into an instance, where each instance corresponds to a class. This task is a part of the concept of scene understanding or better explaining the global context of an image. Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: indoor navigation, autonomous driving, and even virtual or augmented reality systems to name a few. This paper provides the concept of image semantic segmentation terms, and reviews traditional and existing deep learning methods with emphasizing their contributions and significance in this field, as well as the evaluation indicators of semantic segmentation algorithms and commonly used datasets. At last, we discuss some problems in current semantic image segmentation tasks, and propose relevant solutions and research prospects.
文章引用：汤婧婧, 章盛, 徐立中. 图像语义分割方法研究进展[J]. 图像与信号处理, 2023, 12(1): 61-75. https://doi.org/10.12677/JISP.2023.121007

参考文献

[1] Lu, X., Wang, W., Shen, J., et al. (2022) Zero-Shot Video Object Segmentation with Co-Attention Siamese Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 2228-2242.
[2] Batra, A., Singh, S., Pang, G., et al. (2019) Improved Road Connectivity by Joint Learning of Orientation and Segmentation. 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 16-20 June 2019, 10377- 10385.
[Google Scholar] [CrossRef] 
[3] Sehar, U. and Naseem, M.L. (2022) How Deep Learning Is Empowering Semantic Segmentation: Traditional and Deep Learning Techniques for Semantic Segmentation: A Comparison. Multimedia Tools and Applications, 81, 30519-30544.
[Google Scholar] [CrossRef] [PubMed]
[4] Sharif Razavian, A., Azizpour, H., Sullivan, J., et al. (2014) CNN Features Off-the-Shelf: An Astounding Baseline for Recognition. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 806-813.
[Google Scholar] [CrossRef] 
[5] Long, J., Shelhamer, E. and Darrell, T. (2015) Fully Convolutional Net-Works for Semantic Segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 3431-3440.
[Google Scholar] [CrossRef] 
[6] Friedman, N., Geiger, D. and Goldszmidt, M. (1997) Bayesian Network Classifiers. Machine Learning, 29, 131-163.
[Google Scholar] [CrossRef] 
[7] Liu, W., Rabinovich, A. and Berg, A.C. (2015) ParseNet: Looking Wider to See Better.
[8] Chen, L.-C., Papandreou, G., et al. (2014) Semantic Image Segmentation with Deep Convolutionalnets and Fully Connected CRFs.
[9] Schwing, A.G. and Urtasun, R. (2015) Fully Connected Deep Structured Networks.
[10] Lin, G., Shen, C., et al. (2016) Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 3194-3203.
[Google Scholar] [CrossRef] 
[11] Noh, H., Hong, S. and Han, B. (2015) Learning Deconvolution Network for Semantic Segmentation. 2015 IEEE International Conference on Computer Vision, Santiago, 7-13 December 2015, 1520-1528.
[Google Scholar] [CrossRef] 
[12] Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 2481-2495.
[Google Scholar] [CrossRef] 
[13] Fu, J., Liu, J., et al. (2019) Stacked Deconvolutional Network for Semantic Segmentation. IEEE Transactions on Image Processing.
[Google Scholar] [CrossRef] 
[14] Ronneberger, O., Fischer, P. and Brox, T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Berlin, 234- 241.
[Google Scholar] [CrossRef] 
[15] Milletari, F., Navab, N. and Ahmadi, S.-A. (2016) V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. International Conference on 3D Vision IEEE, Stanford, 25-28 October 2016, 565-571.
[Google Scholar] [CrossRef] 
[16] Lin, T.-Y., Dollar, P., et al. (2017) Feature Pyramid Networks for Object Detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, 2117-2125.
[Google Scholar] [CrossRef] 
[17] Zhao, H., Shi, J., et al. (2017) Pyramid Scene Parsing Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, 2881-2890.
[Google Scholar] [CrossRef] 
[18] Ren, S., He, K., et al. (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems, Volume 1, 91-99.
[19] He, K., Gkioxari, Dollar, G.P. and Girshick, R. (2017) Mask R-CNN. 2017 IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 2961-2969.
[Google Scholar] [CrossRef] 
[20] Liu, S., Qi, L., et al. (2018) Path Aggregation Network for Instance Segmentation. 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 8759-8768.
[Google Scholar] [CrossRef] 
[21] Chen, L.-C., Hermans, A., et al. (2018) Masklab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features. 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 4013-4022.
[Google Scholar] [CrossRef] 
[22] Lee, Y. and Park, J. (2020) CenterMask: Real-Time Anchor-Free Instance Segmentation. 2020 IEEE Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 13906-13915.
[Google Scholar] [CrossRef] 
[23] Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2018) DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834-848.
[Google Scholar] [CrossRef] 
[24] Chen, L.-C., Papandreou, G., Schroff, F. and Adam, H. (2018) Rethinking Atrous Convolution for Semantic Image Segmentation.
[25] Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018) Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. 2018 European Conference on Computer Vision, Munich, 8-14 September 2018, 801-818.
[Google Scholar] [CrossRef] 
[26] Visin, F., Kastner, K., et al. (2015) ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks.
[27] Byeon, W., Breuel, T.M., et al. (2015) Scene Labeling with LSTM Recurrent Neural Networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 3547-3555.
[Google Scholar] [CrossRef] 
[28] Xiang, Y. and Fox, D. (2017) DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks.
[Google Scholar] [CrossRef] 
[29] Chen, L.-C., Yang, Y., et al. (2016) Attention to Scale: Scale-Aware Semantic Image Segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 3640-3649.
[Google Scholar] [CrossRef] 
[30] Fu, J., Liu, J., et al. (2019) Dual Attention Network for Scene Segmentation. 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 3146-3154.
[Google Scholar] [CrossRef] 
[31] Choi, S., Kim, J.T. and Choo, J. (2020) Cars Can’t Fly Up in the Sky: Improving Urban-Scene Segmentation via Height-Driven Attention Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 9373-9383.
[Google Scholar] [CrossRef] 
[32] Zhang, H., Wu, C., et al. (2020) Resnest: Split-Attention Networks.
[33] Luc, P., Couprie, C., et al. (2016) Semantic Segmentation Using Adversarial Networks.
[34] Hung, W.-C., Tsai, Y.-H., et al. (2018) Adversarial Learning for Semi-Supervised Semantic Segmentation.
[35] Xue, Y., Xu, T., et al. (2018) SegAN: Adversarial Network with Multi-Scale L1 Loss for Medical Image Segmentation. Neuroinformatics, 16, 383-392.
[Google Scholar] [CrossRef] [PubMed]
[36] Xu, J.J., Zhang, Z.X. and Hu, X.L. (2022) Extracting Semantic Knowledge from GANs with Unsupervised Learning.
[37] Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., et al. (2017) A Review on Deep Learning Techniques Applied to Semantic Segmentation.
[38] Xu, K., Wen, L., et al. (2019) Spatiotemporal CNN for Video Object Segmentation. 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 1379-1388.
[Google Scholar] [CrossRef] 
[39] Cao, Z., Hidalgo, G., Simon, T., et al. (2018) OpenPose: Real-Time Multi-Person 2D Pose Estimation Using Part Affinity Fields.
[40] Zhu, Y., Zhou, Y., Xu, H., et al. (2019) Learning Instance Activation Maps for Weakly Supervised Instance Segmentation. 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 3111-3120.
[Google Scholar] [CrossRef] 
[41] Chen, K., et al. (2019) Hybrid Task Cascade for Instance Segmentation. 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 4969-4978.
[Google Scholar] [CrossRef] 
[42] Zhuang, P., Wang, Y. and Qiao, Y. (2020) Learning Attentive Pairwise Interaction for Fine-Grained Classification. 2020 AAAI Conference on Artificial Intelligence, New York, 7-12 February 2020, 13130-13137.
[Google Scholar] [CrossRef] 
[43] He, J., Deng, Z., Zhou, L., et al. (2019) Adaptive Pyramid Context Network for Semantic Segmentation. 2019 IEEE/ CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 7511-7520.
[Google Scholar] [CrossRef] 
[44] Liu, J., He, J., et al. (2020) Learning to Predict Context-Adaptive Convolution for Semantic Segmentation. 2020 European Conference on Computer Vision, Glasgow, 23-28 August 2020, 769-786.
[Google Scholar] [CrossRef] 
[45] Ke, L., Tai, Y.W. and Tang, C.K. (2021) Deep Occlusion-Aware Instance Segmentation with Overlapping Bi-Layers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 4018-4027.
[Google Scholar] [CrossRef]