Abstract: To address the problems of small-object segmentation errors and holes in large-object segmentation results in scenes with significant differences in object sizes in real-time semantic segmentation, this paper proposed a real-time semantic segmentation algorithm based on a composite three-branch and deep feature encoding, consisting of a composite three-branch module (CTBM), a deep feature encoding module (DFEM), and a dual-branch multi-layer perceptron (DBMLP). The CTBM used a dual-layer multi-scale feature extraction and fusion strategy to comprehensively extract information from different perspectives, enabling the model to perceive the global relationships between features better and reduce the holes in the large-object segmentation results. The DFEM enhanced the model’ s ability to express deep features through encoding methods, better perceived the semantic information of small objects, and improved the segmentation accuracy of small objects. The DBMLP effectively integrated multi-scale semantic information by utilizing both global and local features, resulting in smoother edges and more accurate contours in segmentation results. Evaluation results on the Cityscapes and ADE20K datasets have shown that the algorithm not only meets real-time speed requirements but also achieves mIoU of 74.2% and 40.4% at 42.6 FPS and 45.3 FPS, respectively, significantly outperforming other real-time semantic segmentation algorithms.

Key words: real-time semantic segmentation, composite three-branch, deep feature encoding, dual-branch multi-layer perceptron