基于改进特征融合和区域生成网络的Mask R-CNN的管件分拣研究

doi:10.3969/j.issn.0255-8297.2023.05.010

应用科学学报 ›› 2023, Vol. 41 ›› Issue (5): 840-854.doi: 10.3969/j.issn.0255-8297.2023.05.010

• 计算机科学与应用 • 上一篇

基于改进特征融合和区域生成网络的Mask R-CNN的管件分拣研究

韩慧妍^1,2,3, 吴伟州^1,2,3, 王文俊^1,2,3, 韩燮^1,2,3

1. 中北大学计算机科学与技术学院, 山西太原 030051;
2. 机器视觉与虚拟现实山西省重点实验室, 山西太原 030051;
3. 山西省视觉信息处理及智能机器人工程研究中心, 山西太原 030051

收稿日期:2022-07-18 发布日期:2023-09-28
通信作者: 韩慧妍,博士,副教授,研究方向为人工智能、虚拟现实。E-mail:hhy980344@163.com E-mail:hhy980344@163.com
基金资助:
山西省自然科学基金（No.202303021211153）；国家自然科学基金（No.62106238）；山西省科技成果转化引导专项（No.202104021301055）；山西省研究生创新项目（No.2021Y626）资助

Research on Pipe Sorting Based on Improved Mask R-CNN Using Feature Fusion and Region Generation Network

HAN Huiyan^1,2,3, WU Weizhou^1,2,3, WANG Wenjun^1,2,3, HAN Xie^1,2,3

1. School of Computer Science and Technology, North University of China, Taiyuan 030051, Shanxi, China;
2. Shanxi Key Laboratory of Machine Vision and Virtual Reality, Taiyuan 030051, Shanxi, China;
3. Shanxi Province's Vision Information Processing and Intelligent Robot Engineering Research Center, Taiyuan 030051, Shanxi, China

Received:2022-07-18 Published:2023-09-28

摘要/Abstract

摘要： 针对管件分割任务中各类管件区分难度大，光线和阴影对分割精度存在干扰等问题，提出了一种改进的掩膜区域卷积神经网络（mask region-convolutional neural network,Mask R-CNN）的管件分拣算法。通过增加低层特征图以改进特征融合网络，提高小型管件的识别率；根据管件尺寸比例改进区域生长网络的生成框，以加快模型收敛速度；增加通道和空间注意力模块，提升管件识别精度及掩膜效果。将改进后的Mask R-CNN用于四类管件的分拣任务，实验结果表明，改进后Mask R-CNN的掩膜检测平均精度均值（mean averageprecision,mAP）和平均召回率（mean recall,mRecall）值分别提高了1.5%和1.7%，对管件位置、类型和尺寸的判别能力更强，能够满足实际生产中机器人分拣管件的精度要求。

关键词: 管件分拣, 低层特征, 区域生成网络, 混合注意力机制, 实例分割

Abstract: In order to solve the problems of difficulty in distinguishing various pipe fittings and interference of light and shadow on segmentation accuracy, a pipe fittings sorting algorithm based on improved Mask R-CNN is proposed. The feature fusion network is improved by incorporating low-level feature map, improving the recognition rate of small pipe fittings. The generation box of area growth network is modified according to the size ratio of pipe fitting, so to accelerate the convergence rate of the model. Introduction of the channel and space attention module enhances the identification accuracy of pipe fitting and mask effect. The improved Mask R-CNN is applied to the sorting task of four types of pipe fittings, demonstrating increased mAP and mRecall values for mask detection (1.5% and 1.7% improvement, respectively). The robot exhibits enhanced capabilities in discriminating the location, type and size of pipe fittings, thereby meeting the accuracy requirements of sorting pipe fittings in actual production.

Key words: pipes sorting, low-level feature, area generation network, mixed attention mechanism, instance segmentation

中图分类号:

P315.69

韩慧妍, 吴伟州, 王文俊, 韩燮. 基于改进特征融合和区域生成网络的Mask R-CNN的管件分拣研究[J]. 应用科学学报, 2023, 41(5): 840-854.

HAN Huiyan, WU Weizhou, WANG Wenjun, HAN Xie. Research on Pipe Sorting Based on Improved Mask R-CNN Using Feature Fusion and Region Generation Network[J]. Journal of Applied Sciences, 2023, 41(5): 840-854.

参考文献

[1] 张慧, 王坤峰, 王飞跃. 深度学习在目标视觉检测中的应用进展与展望[J]. 自动化学报, 2017, 43(8):1289-1305. Zhang H, Wang K F, Wang F Y. Advances and perspectives on applications of deep learning in visual object detection[J]. Acta Automatica Sinica, 2017, 43(8):1289-1305. (in Chinese)
[2] 贾鹤鸣, 李瑶, 姜子超, 等. 基于改进共生生物搜索算法的林火图像多阈值分割[J]. 计算机应用, 2021, 41(5):1465-1470. Jia H M, Li Y, Jiang Z C, et al. Multi-threshold segmentation of forest fire images based on modified symbiotic organisms search algorithm[J]. Journal of Computer Applications, 2021, 41(5):1465-1470. (in Chinese)
[3] Hariharan B, Arbeláez P, Girshick R, et al. Simultaneous detection and segmentation[C]//European Conference on Computer Vision, 2014:297-312.
[4] Pinheiro P O, Collobert R, Dollár P. Learning to segment object candidates[J]. Advances in Neural Information Processing Systems, 2015, 28.
[5] He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017:2961-2969.
[6] Ren S, He K, Girshick R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6):1137-1149.
[7] Zhang Y, Cheng W. Vision-based robot sorting system[C]//IOP Conference Series:Materials Science and Engineering, 2019, 592(1):012154.
[8] 梁新宇, 林洗坤, 权冀川, 等. 基于深度学习的图像实例分割技术研究进展[J]. 电子学报, 2020, 48(12):2476-2486. Liang X Y, Lin X K, Quan J C, et al. Research on the progress of image instance segmentation based on deep learning[J]. Acta Electronica Sinica, 2020, 48(12):2476-2486. (in Chinese)
[9] 王诗宇. 智能化工业机器人视觉系统关键技术研究[D]. 沈阳:中国科学院大学(中国科学院沈阳计算技术研究所), 2021.
[10] Qi C R, Su H, Mo K, et al. Pointnet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:652-660.
[11] Liang H, Ma X, Li S, et al. Pointnetgpd:detecting grasp configurations from point sets[C]//2019 International Conference on Robotics and Automation. IEEE, 2019:3629-3635.
[12] Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2117-2125.
[13] Nagaoka Y, Miyazaki T, Sugaya Y, et al. Text detection using multi-stage region proposal network sensitive to text scale[J]. Sensors, 2021, 21(4):1232.
[14] 戚裕绮, 范成杰, 陈欣, 等. 基于无锚框关键点算法的工件表面缺陷检测[J]. 计算机工程与设计, 2023, 44(1):203-209. Qi Y Q, Fan C J, Chen X, et al. Defect detection of workpiece surface based on anchorfree and keypoint algorithm[J]. Computer Engineering and Design, 2023, 44(1):203-209. (in Chinese)
[15] 林凯瀚, 赵慧民, 吕巨建, 等. 基于Mask R-CNN的人脸检测与分割方法[J]. 计算机工程, 2020, 46(6):274-280. Lin K H, Zhao H M, Lyu J J, et al. Face detection and segmentation method based on Mask R-CNN[J]. Computer Engineering, 2020, 46(6):274-280. (in Chinese)
[16] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[17] 季秀怡, 李建华. 基于双路注意力机制的化学结构图像识别[J]. 计算机工程, 2020, 46(9):213-220. Ji X L, Li J H. Chemical structure image recognition based on dual attention mechanism[J]. Computer Engineering, 2020, 46(9):213-220. (in Chinese)
[18] 周鑫, 钟琴. 基于离散余弦变换的高光谱图像复原方法[J]. 激光杂志, 2023, 44(2):118-122. Zhou X, Zhong Q. Hyperspectral image restoration method based on discrete cosine transform[J]. Laser Journal, 2023, 44(2):118-122. (in Chinese)
[19] Qin Z, Zhang P, Wu F, et al. FCANET:frequency channel attention networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021:783-792.
[20] Barukab O, Ali F, Alghamdi W, et al. DBP-CNN:Deep learning-based prediction of DNAbinding proteins by coupling discrete cosine transform with two-dimensional convolutional neural network[J]. Expert Systems with Applications, 2022, 197:116729.
[21] 李昂, 孙士杰, 张朝阳, 等. 改进YOLOv5s的轨道障碍物检测模型轻量化研究[J]. 计算机工程与应用, 2023, 59(4):197-207. Li A, Sun S J, Zhang Z Y, et al. Research on lightweight of improved YOLOv5s track obstacle detection model[J]. Computer Engineering and Applications, 2023, 59(4):197-207. (in Chinese)
[22] 唐超, 左文涛, 李小飞. 结合修剪均值与高斯加权中值滤波的图像去噪算法[J]. 计算机工程, 2021, 47(9):210-216. Tang C, Zuo W T, Li X F. Image denoising algorithm combining trimmed mean and gaussian weighted median filtering[J]. Computer Engineering, 2021, 47(9):210-216. (in Chinese)
[23] Wang X, Kong T, Shen C, et al. Solo:segmenting objects by locations[C]//Computer Vision-ECCV 2020, 2020:649-665.

基于改进特征融合和区域生成网络的Mask R-CNN的管件分拣研究

Research on Pipe Sorting Based on Improved Mask R-CNN Using Feature Fusion and Region Generation Network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	倪翠, 王朋, 朱元汀, 张东. 嵌套网络模型下的相似图像检索方法[J]. 应用科学学报, 2022, 40(3): 400-410.
[2]	吴东鹏, 王峥, 童薇, 叶枫, 宋楚翘. 基于决策树-逻辑回归模型精确识别僵尸企业[J]. 应用科学学报, 2021, 39(4): 569-580.
[3]	倪翠, 王朋, 张广渊, 李克峰. 一种基于双重网络模型的单幅图像超分辨率方法[J]. 应用科学学报, 2021, 39(2): 321-329.