基于深度学习通道交换的多模态脑肿瘤图像融合的分割模型
Multimodal Brain Tumor Image Fusion Segmentation Model Based on Deep Learning Channel Swapping
DOI: 10.12677/PM.2023.134103, PDF,  被引量   
作者: 张顾瀛, 贺光华：绍兴文理学院数理信息学院，浙江 绍兴
关键词: 深度学习；多模态融合；Swin-Transformer；Deep Learning； Multimodal Fusion； Swin-Transformer
摘要: 脑部肿瘤是患者面临极高致死率的疾病，手术切除是最佳治疗手段。术前精确分割整个脑部肿瘤区域对于手术切除至关重要。本文提出了一种基于深度学习通道交换的多模态脑肿瘤图像融合分割模型，该模型采用通道交换和Swin-Transformer技术，对不同模态进行分类，提取不同类别的特征模态并进行特征通道交换，以获得图像局部互补性特征。同时，该模型利用Swin-Transformer进一步融合全局和局部特征。实验结果表明，在BraTS2021数据集下，该模型具有竞争力的分割精度，整个肿瘤的Dice值为91.07。
Abstract: Brain tumor is a disease with extremely high mortality rate for patients, and surgical resection is the best treatment option. Accurate preoperative segmentation of the entire brain tumor region is crucial for surgical resection. In this paper, a multimodal brain tumor image fusion segmentation model based on deep learning channel swapping is proposed. The model utilizes channel swapping and Swin-Transformer technology to classify different modalities, extract feature modalities of different categories, and perform feature channel swapping to obtain locally complementary features. Meanwhile, the model utilizes Swin-Transformer to further fuse global and local features. Experimental results show that the model has competitive segmentation accuracy on the BraTS2021 dataset, with a Dice value of 91.07 for the entire tumor.
文章引用：张顾瀛, 贺光华. 基于深度学习通道交换的多模态脑肿瘤图像融合的分割模型[J]. 理论数学, 2023, 13(4): 976-986. https://doi.org/10.12677/PM.2023.134103

参考文献

[1] Ronneberger, O., Fischer, P. and Brox, T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W. and Frangi, A., Eds., MICCAI 2015: Medical Image Computing and Comput-er-Assisted Intervention—MICCAI 2015, Lecture Notes in Computer Science, Vol. 9351, Springer, Cham, 234-241. [Google Scholar] [CrossRef] 
[2] Jwaid, W.M., Al-Husseini, Z.S.M. and Sabry, A.H. (2021) Development of Brain Tumor Segmentation of Magnetic Resonance Imaging (MRI) Using U-Net Deep Learning. Eastern-European Journal of Enterprise Technologies, 4, 23-31. [Google Scholar] [CrossRef] 
[3] Pereira, S., Pinto, A., Amorim, J., Ribeiro, A., Alves, V. and Silva, C.A. (2019) Adaptive Feature Recombination and Recalibration for Semantic Segmentation with Fully Convolutional Networks. IEEE Transactions on Medical Imaging, 38, 2914-2925. [Google Scholar] [CrossRef] 
[4] Wang, W., Chen, C., Ding, M., Yu, H., Zha, S. and Li, J. (2021) TransBTS: Multi-Model Brain Tumor Segmentation Using Transformer. In: Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y. and Essert, C., Eds., MICCAI 2021: Medical Image Computing and Computer Assisted In-tervention—MICCAI 2021, Lecture Notes in Computer Science, Vol. 12901, Springer, Cham, 109-119. [Google Scholar] [CrossRef] 
[5] Liu, Z., et al. (2021) Swin Transformer: Hierarchical Vision Transformer Using shifted Windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 9992-10002. [Google Scholar] [CrossRef] 
[6] Cao, H., et al. (2021) Swin-Unet: Unet-Like Pure Trans-former for Medical Image Segmentation. (Preprint)
[7] Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. and Xu, D. (2022) Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. In: Crimi, A. and Bakas, S., Eds., BrainLes 2021: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, Vol. 12962, Springer, Cham, 272-284. (Preprint) [Google Scholar] [CrossRef] 
[8] Peiris, H., Hayat, M., Chen, Z., Egan, G. and Harandi M. (2021) A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S. and Li, S., Eds., MICCAI 2022: Medical Image Computing and Computer Assisted Intervention—MICCAI 2022, Lecture Notes in Computer Science, Vol. 13435, Springer, Cham, 162-172. (Preprint) [Google Scholar] [CrossRef] 
[9] Wu, Y., et al. (2022) D-Former: A U-Shaped Dilated Transformer for 3D Medical Image Segmentation. Neural Computing and Applications, 35, 1931-1944. (Preprint) [Google Scholar] [CrossRef] 
[10] Lin, A., Chen, B., Xu, J., Zhang, Z., Lu, G. and Zhang, D. (2022) DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation. IEEE Transactions on Instrumentation and Measurement, 71, 1-15. [Google Scholar] [CrossRef] 
[11] Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I. and Patel, V.M. (2021) Medical Transformer: Gated Axial-Attention for Medical Image Segmentation. In: Bruijne, M., et al., Eds., MICCAI 2021: Medical Image Computing and Computer Assisted Intervention—MICCAI 2021, Lecture Notes in Computer Science, Vol. 12901, Springer, Cham, 36-46. [Google Scholar] [CrossRef] 
[12] Zhang, Y., et al. (2022) mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S. and Li, S., Eds., MICCAI 2022: Medical Image Computing and Computer Assisted Intervention—MICCAI 2022, Lecture Notes in Computer Science, Vol. 13435, Springer, Cham, 101-117. (Preprint) [Google Scholar] [CrossRef] 
[13] Calhoun, V.D. and Sui, J. (2016) Multimodel Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1, 230-244, [Google Scholar] [CrossRef] [PubMed]
[14] Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J. and Wiest, R. (2015) The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Transactions on Medical Imaging, 34, 1993-2024, [Google Scholar] [CrossRef] 
[15] Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M. and Rozycki, M. (2018) Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge. (Preprint)
[16] Lin, J., Lin, J., Lu, C., Chen, H., Lin, H., Zhao, B., Shi, Z., Qiu, B., Pan, X. and Xu, Z. (2023) CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation. IEEE Transactions on Medical Imaging, 1-1. (Preprint) [Google Scholar] [CrossRef] 

为你推荐