面向电子装备装配引导的AR跟踪方法研究

doi:10.3969/j.issn.0255-8297.2024.03.004

应用科学学报 ›› 2024, Vol. 42 ›› Issue (3): 416-424.doi: 10.3969/j.issn.0255-8297.2024.03.004

面向电子装备装配引导的AR跟踪方法研究

杜小东¹, 王鹏², 史建成¹, 王月², 帅昊²

1. 中国电子科技集团公司第二十九研究所, 四川成都 610036;
2. 重庆邮电大学先进制造工程学院, 重庆 400065

收稿日期:2023-04-10 发布日期:2024-06-06
通信作者: 王鹏，博士，研究方向为AR/MR关键技术和AR装配引导技术。E-mail: ilovemymfandb@163.com E-mail:ilovemymfandb@163.com
基金资助:
国家重点研发计划（No. 2020YFB1710300）；航空科学基金（No. 2019ZE105001）；重庆市自然科学基金面上项目（No. CSTC2019JCYJ-MSXMX0530, No. CSTB2022NSCQ-MSX1153）资助

Research on AR Tracking Method for Electronic Equipment Assembly Guidance

DU Xiaodong¹, WANG Peng², SHI Jiancheng¹, WANG Yue², SHUAI Hao²

1. Southwest China Research Institute of Electronic Equipment, Chengdu 610036, Sichuan, China;
2. School of Advanced Manufacturing Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Received:2023-04-10 Published:2024-06-06

摘要/Abstract

摘要： 为了提高面向电子装备装配引导的增强现实（augmented reality,AR）跟踪方法的鲁棒性与适用性，在像素投票姿态估计网络结构的基础上，结合深度可分离卷积和通道注意力机制对AR跟踪算法进行优化。首先，针对电子装备六自由度姿态公共数据集稀缺与使用约束较多的问题，使用RGB-D相机采集并制作AR装配引导的电子装备的六自由度姿态训练数据集。然后，在基于像素投票的姿态估计网络结构基础上，使用深度可分离卷积对网络进行轻量化，并引入通道注意力机制对通道进行权重评估，以弥补网络轻量化造成的精度损失。最后，通过电子装备装配任务对提出的方法进行AR装配引导验证。实验结果表明：提出的跟踪注册方法相对于修改前的跟踪方法具有较好的鲁棒性，同时保持了装配引导的精度，能够对弱纹理的电子装备进行跟踪。

关键词: 电子装备, 增强现实, 三维跟踪, 深度可分离卷积, 通道注意力

Abstract: This paper aims to enhance the robustness and versatility of augmented reality (AR) tracking methods for electronic equipment assembly guidance by optimizing the structure of the position estimation network. This optimization involves integrating depthwise separable convolution with a channel attention mechanism. First, due to the lack of public datasets of 6 degrees of freedom (6-DOF) electronic equipment and various usage constraints, an RGB-D camera is used to collect and produce a 6-DOF training dataset for AR assembly guided electronic equipment. Then, using the structure of the position estimation network based on the pixel voting, depth-wise separable convolution is used to lighten the network, and the channel attention mechanism is introduced to evaluate the weight of the channels to compensate the accuracy loss caused by lightening the network. Finally, we verify the proposed network structure through AR assembly guidance by the electronic equipment task. Results show that the proposed tracking method exhibits superior robustness and maintains sound assembly guidance accuracy compared to existing method. Moreover, it can track the electronic equipment with weak texture and meet the real-time tracking requirements while ensuring accuracy.

Key words: electronic equipment, augmented reality, 3D tracking, depth-wise separable convolution, channel attention

中图分类号:

TP391.9

杜小东, 王鹏, 史建成, 王月, 帅昊. 面向电子装备装配引导的AR跟踪方法研究[J]. 应用科学学报, 2024, 42(3): 416-424.

DU Xiaodong, WANG Peng, SHI Jiancheng, WANG Yue, SHUAI Hao. Research on AR Tracking Method for Electronic Equipment Assembly Guidance[J]. Journal of Applied Sciences, 2024, 42(3): 416-424.

参考文献

[1] 韩玉仁, 李铁军, 杨冬. 增强现实中三维跟踪注册技术概述[J]. 计算机工程与应用, 2019, 55(21): 26-35. Han Y R, Li T J, Yang D. Overview of 3D tracking registration technology in augmented reality [J]. Computer Engineering and Applications, 2019, 55(21): 26-35. (in Chinese)
[2] 李旺, 王峻峰, 蓝珊, 等. 增强现实装配工艺信息内容编辑技术[J]. 计算机集成制造系统, 2019, 25(7): 1676-1684. Li W, Wang J F, Lan S, et al. Content authoring of augmented reality assembly process [J]. Computer Integrated Manufacturing Systems, 2019, 25(7): 1676-1684. (in Chinese)
[3] 方维, 许澍虹, 韩磊, 等. AR增强装配中的跟踪注册方法研究与应用进展[J]. 系统仿真学报, 2023, 35(7): 1438-1454. Fang W, Xu S H, Han L, et al. Research and application progress of tracking registration methods in AR assembly [J]. Journal of System Simulation, 2023, 35(7): 1438-1454. (in Chinese)
[4] 王崴, 洪学峰, 雷松贵. 基于MR的机电装备智能检测维修[J]. 图学学报, 2022, 43(1): 141-148. Wang W, Hong X F, Lei S G. Intelligent inspection and maintenance of mechanical and electrical equipment based on MR [J]. Journal of Graphics, 2022, 43(1): 141-148. (in Chinese)
[5] 张品, 王学渊. 一种基于增强现实的高精度自动配准和追踪技术[J]. 制造业自动化, 2021, 43(1): 78-82. Zhang P, Wang X Y. A high precision automatic registration and tracking technology based on augmented reality [J]. Manufacturing Automation, 2021, 43(1): 78-82. (in Chinese)
[6] Wang P, Zhang S S, Billinghurst M, et al. A comprehensive survey of AR/MR-based co-design in manufacturing [J]. Engineering with Computers, 2020, 36(4): 1715-1738.
[7] Wang P, Bai X L, Billinghurst M, et al. AR/MR remote collaboration on physical tasks: a review [J]. Robotics and Computer-Integrated Manufacturing, 2021, 72: 1-32.
[8] 董琼, 李斌, 董剑, 等. 面向头戴式眼镜的增强现实装配语音交互实现[J]. 制造业自动化, 2020, 42(10): 77-80. Dong Q, Li B, Dong J, et al. Realization of augmented reality assembly voice interaction for head-mounted glasses [J]. Manufacturing Automation, 2020, 42(10): 77-80. (in Chinese)
[9] 王月, 张树生, 白晓亮. 点云和视觉特征融合的增强现实装配系统三维跟踪注册方法[J]. 西北工业大学学报, 2019, 37(1): 143-151. Wang Y, Zhang S S, Bai X L. A 3D tracking and registration method based on point cloud and visual features for augmented reality aided assembly system [J]. Journal of Northwestern Polytechnical University, 2019, 37(1): 143-151. (in Chinese)
[10] 刘佳, 郭斌, 张晶晶, 等. 视触觉融合的增强现实三维注册方法[J]. 计算机工程与应用, 2021, 57(11): 70-76. Liu J, Guo B, Zhang J J, et al. 3D registration method for augmented reality based on visual and haptic integration [J]. Computer Engineering and Applications, 2021, 57(11): 70-76. (in Chinese)
[11] 赵新灿, 左洪福. 基于线特征的增强现实注册算法[J]. 应用科学学报, 2008, 26(1): 68-73. Zhao X C, Zuo H F. Augmented reality registration algorithm based on line features [J]. Journal of Applied Sciences, 2008, 26(1): 68-73. (in Chinese)
[12] Qi R, Su H, Mo K, et al. PointNet: deep learning on point sets for 3D classification and segmentation [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 77-85.
[13] 欧巧凤, 肖佳兵, 谢群群, 等. 基于深度学习的车检图像多目标检测与识别[J]. 应用科学学报, 2021, 39(6): 939-951. Ou Q F, Xiao J B, Xie Q Q, et al. Multi-target detection and recognition for vehicle inspection images based on deep learning [J]. Journal of Applied Sciences, 2021, 39(6): 939-951. (in Chinese)
[14] Howard A G, Zhu M L, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications [DB/OL]. 2017[2023-04-10]. https://arxiv.org/abs/1704.04861.
[15] Hu J, Shen L, Sun G, et al. Squeeze-and-excitation networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[16] Peng S D, Liu Y, Huang Q X, et al. PVNet: pixel-wise voting network for 6DoF pose estimation [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 4556-4565.
[17] Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate O(n) solution to the PnP problem [J]. International Journal of Computer Vision, 2009, 81(2): 155-166.
[18] Tekin B, Sinha S N, Fua P. Real-time seamless single shot 6D object pose prediction [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 292-301.

面向电子装备装配引导的AR跟踪方法研究

Research on AR Tracking Method for Electronic Equipment Assembly Guidance

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

本文评价

[1]	徐增敏, 陆光建, 陈俊彦, 陈金龙, 丁勇. 基于通道特征聚合的行人重识别算法[J]. 应用科学学报, 2023, 41(1): 107-120.
[2]	袁文杰, 田金鹏, 杨洁. 基于超深残差通道注意力网络的图像压缩感知重构[J]. 应用科学学报, 2022, 40(6): 887-895.
[3]	于群, 张建新, 魏小鹏, 张强. 基于级联可分离空洞残差U-Net的肝脏肿瘤分割[J]. 应用科学学报, 2021, 39(3): 378-377.
[4]	赵新灿;左洪福. 基于线特征的增强现实注册算法[J]. 应用科学学报, 2008, 26(1): 68-68 .
[5]	吴家麒, 陈金波. 空间靶标位姿的三维重建方法研究[J]. 应用科学学报, 2005, 23(6): 640-644.