为了解决感染区域比正常组织对比度低的问题，提出了一种基于三重注意力机制（triple attention mechanism，TAM）的新冠肺炎（corona virus disease 2019，COVID 19）病灶分割模型--TM-Net，并将其应用于条件生成对抗网络。MultiConv模块可以自动提取肺部切片中感染区域的特征，呈现了更丰富且包含不同类型的病灶信息。TAM同时集成了空间、通道、位置注意力模块，可以更精准地定位感染区域的病灶。使用的损失函数是由3种不同的损失函数组成的复合函数，能最小化预测图和真实标签的差异，更好地优化TM-Net模型。在COVID-19数据集上进行实验和评估的结果表明：基于TM-Net的磨玻璃影（ground-glass opacities，GGO）和固结（Consolidation）两类病灶的平均dice相似系数（dice similarity coefficient，DSC）比基于Attention U-Net和R2U-Net的DSC分别提高了1.4%和0.5%，证明TM-Net提升了新冠肺炎病灶分割的准确性。
In order to solve the problem of low intensity contrast between infected areas and normal tissues, A corona virus disease 2019 (COVID-19) segmentation model TMNet is proposed based on triple attention mechanism (TAM), and applied to conditional generative adversarial network in this paper. The MultiConv module in TM-Net can automatically extract rich features of infected areas in lung slices. These features contain different types of lesion information. The designed TAM, which integrates spatial, channel and positional attention modules, can accurately locate lesions in the infected area. By composing of three types of loss functions, the loss function of TM-Net can minimize the differences between prediction graphs and real labels, thus optimizing the TM-Net. Experiment and evaluations conducted on COVID-19 data sets show that the average dice similarity coefficient (DSC) of ground glass opacities (GGO) and consolidation of TM-Net are 1.4% and 0.5% higher than the results of attention U-Net and R2U-Net, respectively, proving the accuracy improvement of TM-Net in COVID-19 lesions segmentation.
[1] Zhang Z, Romero A, Muckley M J, et al. Reducing uncertainty in undersampled MRI reconstruction with active acquisition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019:2049-2058.
[2] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2014:580-587.
[3] Ronneberger O, Fischer P, Brox T. U-Net:convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and ComputerAssisted Intervention. Cham:Springer, 2015:234-241.
[4] Khened M, Kollerathu V A, Krishnamurthi G. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J]. Medical Image Analysis, 2019, 51:21-45.
[5] Kang H, Xia L, Yan F, et al. Diagnosis of coronavirus disease 2019(COVID-19) with structured latent multi-view representation learning[J]. IEEE Transactions on Medical Imaging, 2020, 39(8):2606-2614.
[6] Chen J, Wu L, Zhang J, et al. Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography[J]. Scientific Reports, 2020, 10(1):1-11.
[7] Zhou Z, Siddiquee M M R, Tajbakhsh N, et al. UNet++:a nested U-Net architecture for medical image segmentation[C]//The 4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and the 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018.[S.l.]:Springer Verlag, 2018:3-11.
[8] Zheng B, Liu Y, Zhu Y, et al. MSD-Net:multi-scale discriminative network for COVID-19 lung infection segmentation on CT[J]. IEEE Access, 2020, 8:185786-185795.
[9] Yang Q, Li Y, Zhang M, et al. Automatic segmentation of COVID-19 CT images using improved MultiResUNet[C]//2020 Chinese Automation Congress, 2020:1614-1618.
[10] Wu Y H, Gao S H, Mei J, et al. JCS:an explainable COVID-19 diagnosis system by joint classification and segmentation[J]. IEEE Transactions on Image Processing, 2021, 30:3113-3126.
[11] Fan D P, Zhou T, Ji G P, et al. Inf-Net:automatic COVID-19 lung infection segmentation from ct images[J]. IEEE Transactions on Medical Imaging, 2020, 39(8):2626-2637.
[12] Ouyang X, Huo J, Xia L, et al. Dual-sampling attention network for diagnosis of COVID-19 from community acquired pneumonia[J]. IEEE Transactions on Medical Imaging, 2020, 39(8):2595-2605.
[13] Xu X, Jiang X, Ma C, et al. A deep learning system to screen novel coronavirus disease 2019 pneumonia[J]. Engineering, 2020, 6(10):1122-1129.
[14] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.
[15] Huang G, Liu Z, van der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017:4700-4708.
[16] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[J/OL]. (2016-04-30)[2021-06-25]. https://arxiv.org/abs/1511.07122v2.
[17] Woo S, Park J, Lee J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision, 2018:3-19.
[18] He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[19] Mirza M, Osindero S. Conditional generative adversarial nets[J/OL]. (2014-11-06)[2021-06-25]. https://arxiv.org/abs/1411.1784.
[20] Yang X, He X, Zhao J, et al. COVID-CT-dataset:a CT scan dataset about COVID-19[J/OL]. (2020-06-17)[2021-06-25]. https://arxiv.org/abs/2003.13865.
