为解决脑血管造影人工质控的不稳定和非实时问题，实现实时脑血管造影质量评估，提出了一种轻量化的分割分类多任务学习模型。该模型分为特征提取主干模块、血管分割模块、造影质量分类模块3个部分，并用深度可分离卷积替代传统卷积以降低参数量；提出了一种局部-全局自注意力模块以增强全局信息的提取能力；在血管分割模块中设计了特征聚合模块以优化特征连接。通过结合质量分类模块和分割结果以及主干特征来评估造影质量，并为模型训练设计了联合损失函数。实验结果表明，提出的模型在参数量仅为3.434 2×106的情况下获得了较好的分割与分类性能，质量评估准确率达到0.8182，且实时性高。
To address the instability and non-real-time problems of manual quality control in cerebral angiography and realize real-time quality assessment, a lightweight segmentation-classification multi-task learning (MTL) model is proposed. This model comprises three main components: a feature extraction backbone module, a blood vessel segmentation module, and an angiography quality classification module. Depth-separable convolution is employed instead of traditional convolution to reduce the number of parameters. Additionally, a local-global self-attention module (L-GSAM) is proposed to enhance the model’s ability to extract global information. A feature aggregation module (FAM) is introduced in the vessel segmentation module to optimize feature connections. The segmentation results are then combined with backbone features to assess the quality of angiography in the classification module, and a joint loss function is designed for model training. Experimental results show that the proposed model achieves good segmentation and classification performance with only 3.4342×106 parameters, the quality assessment accuracy reaches 0.818 2, and the model exhibits high real-time performance.
[1] 钟华, 王滨, 张雪梅. 影响脑血管造影图像质量因素的探讨[J]. 新疆医科大学学报, 2009, 32(8): 1151-1152. Zhong H, Wang B, Zhang X M. A study of factors affecting the image quality of cerebral angiography [J]. Journal of Xinjiang Medical University, 2009, 32(8): 1151-1152. (in Chinese)
[2] Tummala S, Thadikemalla V S G, Kadry S, et al. EfficientNetV2 based ensemble model for quality estimation of diabetic retinopathy images from DeepDRiD [J]. Diagnostics, 2023, 13(4): 622.
[3] Shi C Y, Lee J, Wang G C, et al. Assessment of image quality on color fundus retinal images using the automatic retinal image analysis [J]. Scientific Reports, 2022, 12(1): 10455.
[4] Hu J H, Zhang C Y, Zhou K, et al. Chest X-ray diagnostic quality assessment: how much is pixel-wise supervision needed? [J]. IEEE Transactions on Medical Imaging, 2022, 41(7): 1711- 1723.
[5] Chen X, Deng Q S, Wang Q, et al. Image quality control in lumbar spine radiography using enhanced U-net neural networks [J]. Frontiers in Public Health, 2022, 10: 891766.
[6] Meng C, Sun K, Guan S Y, et al. Multiscale dense convolutional neural network for DSA cerebrovascular segmentation [J]. Neurocomputing, 2020, 373: 123-134.
[7] Cui Y, Su J J, Zhu J, et al. Spatial multi-scale attention U-improved network for blood vessel segmentation [J]. Signal, Image and Video Processing, 2023, 17(6): 2857-2865.
[8] Laibacher T, Weyde T, Jalali S. M2U-Net: effective and efficient retinal vessel segmentation for real-world applications [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019: 115-124.
[9] Bai R F, Liu X R, Jiang S, et al. Deep learning based real-time semantic segmentation of cerebral vessels and cranial nerves in microvascular decompression scenes [J]. Cells, 2022, 11(11): 1830.
[10] Xu Z W, Zou B J, Liu Q. A deep retinal image quality assessment network with salient structure priors [J]. Multimedia Tools and Applications, 2023, 82(22): 34005-34028.
[11] Wang Z W, Song Y X, Zhao B L, et al. A soft-reference breast ultrasound image quality assessment method that considers the local lesion area [J]. Bioengineering, 2023, 10(8): 940.
[12] Ruder S. An overview of multi-task learning in deep neural networks [DB/OL]. (2017-06-15) [2024-01-07]. https://arxiv.org/abs/1706.05098v1.
[13] Xu Q, Zeng Y, Tang W J, et al. Multi-task joint learning model for segmenting and classifying tongue images using a deep neural network [J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(9): 2481-2489.
[14] Howard A G, Zhu M L, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications [DB/OL]. (2017-04-17) [2024-01-07]. https://arxiv.org/abs/1704.04861v1.
[15] Tan M X, Le Q V. EfficientNet: rethinking model scaling for convolutional neural networks [DB/OL]. (2019-05-28) [2024-01-07]. https://arxiv.org/abs/1905.11946v5.
[16] Liu Z, Lin Y T, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021: 9992-10002.
[17] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation [C]//18th International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI), 2015: 234-241.
[18] Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: learning where to look for the pancreas [DB/OL]. (2018-05-20) [2024-01-07]. https://arxiv.org/abs/1804.03999v3.
[19] Vandenhende S, Georgoulis S, Van Gansbeke W, et al. Multi-task learning for dense prediction tasks: a survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3614-3633.
[20] Guo C L, Szemenyei M, Yi Y G, et al. SA-UNet: spatial attention U-net for retinal vessel segmentation [C]//202025th International Conference on Pattern Recognition (ICPR), 2021: 1236-1242.
[21] Chen J N, Lu Y Y, Yu Q H, et al. TransUNet: transformers make strong encoders for medical image segmentation [DB/OL]. (2021-02-08) [2024-01-07]. https://arxiv.org/abs/2102.04306v1.
[22] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[23] Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization [C]//2017 IEEE/CVF International Conference on Computer Vision (ICCV), 2017: 618-626.
