-
管片自动拼装定位的机械装置是一个由液压油缸和电机组成的6自由度拼装机,包括三个滑动自由度(
$x,y,{\textit{z}}$ )和3个绕轴的旋转自由度(${\theta _x},{\theta _y},{\theta _{\textit{z}}}$ )。拼装机的下表面是一个用于管片抓取的真空吸盘,如图1(a)所示。管片自动拼装定位检测的基本原理就是分别利用视觉系统和激光测距系统计算待拼装管片的平面位姿和深度位姿信息,根据位姿信息计算拼装机进行管片拼装时所需的位姿信息。其中,平面位姿的确定是依靠测量管片表面不同位置的特征点坐标实现的,而深度位姿则是利用多个激光位移传感器确定管片不同位置的高度差实现的。用于管片自动拼装定位的检测装置固定在拼装机上,其分布见图1(b),所示包括2个相机和7个激光位移传感器。自动拼装定位过程包含拼装机从地面抓取管片和将抓取的管片放置在待拼装位置两步。进行第一步操作时,保证相机可以拍摄到待拼装管片标记凹槽的位置,激光位移传感器${A_0}$ 、${B_0}$ 、${C_0}$ 的光源投射到待抓取管片上;进行第1步操作时,要保证两个相机不仅可以拍摄到待拼装管片上的凹槽,还要拍摄到已经拼装管片上对应的标记凹槽,同时激光位移传感器${A_{\rm{1}}}$ 、${A_{\rm{2}}}$ 、${B_1}$ 、${B_{\rm{2}}}$ 投射到对应的已拼装管片表面。管片上的标志凹槽照片和尺寸分别如图1(c)所示,每个凹槽都对应着一个用于管片固定的螺栓孔。因此,只要对准待拼装管片与相邻管片的凹槽,就完成了平面位姿定位。图 1 (a) 拼装机机械结构图;(b) 检测系统布局图;(c) 凹槽的实物图和尺寸图(单位:mm)
Figure 1. (a) Mechanical structure diagram of the proposed erector; (b) Layout of the detection system; (c) Physical drawing and dimension drawing of groove (Unit: mm)
平面位姿检测过程中,首先利用特征提取算法提取凹槽的轮廓信息,然后利用最小外接矩形法拟合提取的轮廓,最后计算矩形的中心坐标作为位姿检测的坐标点。凹槽轮廓信息提取是该方法的重点。管片表面图像中,背景、目标和噪声成分互不相关,包含目标凹槽的管片图像灰度值由三者线性叠加,建立如下数学模型:
$$I\left( {x,y} \right) = T\left( {x,y} \right) + B\left( {x,y} \right) + N\left( {x,y} \right)$$ (1) 式中:
$\left( {x,y} \right)$ 为空间变量,即像素点的坐标;$I\left( {x,y} \right)$ 为图像的灰度值;$T\left( {x,y} \right)$ 为目标凹槽,$B\left( {x,y} \right)$ 为背景值;$N\left( {x,y} \right)$ 图像噪声。基于此模型,管片表面凹槽的轮廓提取就是通过抑制环境污染引入的噪声和去除背景,进而获得轮廓信息的过程。由于目标和背景的对比度较低,且随机的噪声会使特征信息丢失,很难建立准确的模型拟合出背景和噪声的分布并进行相应的计算。
针对上述模型,文中设计的双阶段深度神经网络的管片表面特征提取框架如图2所示,根据功能划分为图像复原阶段和凹槽轮廓提取阶段。其基本工作流程为:将原始图片送入到图像复原阶段的神经网络中,进行噪声的去除,该阶段包括特征提取、残差密集块和密集特征融合3个主要部分;然后,将去除污染的图像送入到凹槽轮廓提取阶段的神经网络中,实现目标确认和目标与背景的分割,该阶段包含特征提取、候选框提取以及分类和回归3个主要部分。
图 2 基于双阶段深度神经网络的管片表面特征提取框架
Figure 2. Segment surface feature extraction framework based on two stage deep neural network
图像复原阶段的网络结构如图3所示,主要包括浅层特征提取(Shallow Feature Extraction, SFE)、残差密集块(Residual Dense Blocks, RDBs)和密集特征融合(Dense Feature Fusion,DFF)[12]。
SFE使用两个Conv层来实现。提取浅层特征的主要原因是图像复原过程中需要尽可能保留图像的细节信息,CNN的浅层特征映射表示图像的细节信息。Conv层的通道数为3,卷积核大小为3×3[13]。
RDBs由多个RDB串联组成,每一个RDB包含Conv、ReLU、特征融合和残差操作,其主要作用是处理FSE过程中输入的图像浅层特征。RDB的输出与前一项中提取到的特征有关,可以表示为[12]:
$${F_d} = {H_{RDB - \left( d \right)}}\left( {{H_{RDB - \left( {d - 1} \right)}}\left( { \cdots \left( {{H_{RDB - \left( 1 \right)}}\left( {{F_0}} \right)} \right)} \right)} \right)$$ (2) 式中:
${H_{RDB - \left( d \right)}}\left( \cdot \right)$ 表示RDB的复合操作过程;${F_0}$ 指从SFE中第2层提取的浅层特征。每个RDB都有密集的连接层,这意味着RDB可以充分利用所有卷积层中的分层功能,这是整个图像增强网络通道的关键。RDB中所有的要素都通过Concat. 层,其作用就是将所有的特征图进行拼接,然后通过第3部分DFF中的1×1卷积层融合为:$${I_{out}} = {H_{DFF}}\left( {{F_n},{F_{n - 1}}, \cdots ,{F_1},{F_0}} \right)$$ (3) 然后通过3×3卷积层输出级联输出
${I_{out}}$ 。图像增强通道的损耗函数定义为:$$l\left( \Theta \right) = \frac{1}{{2N}}\sum\limits_{i = 1}^N {\left\| {I_{\left( i \right)}^{pred}\left( {x,y;\Theta } \right) - I_{\left( i \right)}^{gt}\left( {x,y} \right)} \right\|} _F^2$$ (4) 式中:
$N$ 表示训练集;$I_{(i)}^{pred}\left( {x,y;\Theta } \right)$ 表示从可训练参数$\Theta $ 获悉的${i^{\rm th}}$ 图像;$I_{(i)}^{gt}\left( {x,y} \right)$ 表示相应的基本事实;${\left\| \cdot \right\|_F}$ 表示Frobenius矩阵范数。图像分割阶段的网络结构主要包括特征提取(Feature Extraction Network, FEN)、区域生成网络(Region Proposal Network, RPN)和分类和回归网络(Classification and Regression Network,CRN)[14]。
Automatic assembly positioning method of shield tunnel segments based on deep learning vision and laser assistance
-
摘要: 隧道管片定位是实现盾构管片自动拼装的关键。文中提出了一种基于深度学习视觉和激光辅助相结合的盾构管片自动拼装定位方法,分别利用视觉系统和激光测距系统计算待拼装管片的平面位姿和深度位姿信息。其中,视觉系统基于特殊设计的双阶段卷积神经网络可以实现管片表面定位标志轮廓特征的有效提取,提取精度和识别率相比于现有算法具有明显的提高。实验表明所提出的盾构管片自动拼装定位方法能够满足盾构管片自动拼装定位需求。Abstract: The positioning of tunnel segments is the key to realize the automatic assembly of shield segments. This paper proposed a method for automatic assembly and positioning of shield segments based on the combination of deep learning vision and laser assistance. The plane pose and depth pose information of the segments to be assembled were obtained by vision system and laser ranging system, respectively. The vision system based on the specially designed two-stage convolutional neural network could effectively extract the contour features of the segment surface positioning marks, and the extraction accuracy and recognition rate were significantly improved compared with existing algorithms. Experiments show that the proposed automatic assembly positioning method of shield segment can meet the requirements of automatic assembly and positioning of shield segment.
-
Key words:
- deep learning /
- segment location /
- feature extraction
-
图 5 不同轮廓特征提取方法的比较。(a)和(b) 基于传统Mask R-CNN算法的结果;(c)和(d) 基于文中所提算法的处理结果;(e)和(f) 原始图像基于文中所提算法的处理结果
Figure 5. Comparison of different contour feature extraction methods. (a) and (b) Processing results based on the traditional Mask R-CNN algorithm; (c) and (d) Processing results based on proposed algorithm in the paper; (e) and (f) Processing results of original image based on proposed algorithm in the paper
-
[1] Koyama Y J T. Present status and technology of shield tunneling method in Japan [J]. Tunnelling and Underground Space Technology, 2003, 18(2/3): 145-159. doi: 10.1016/S0886-7798(03)00040-3 [2] Wada M. Automatic segment erection system for shield tunnels [J]. Advanced Robotics, 1990, 5(4): 429-443. doi: 10.1163/156855391X00304 [3] Yasuo Tanaka. Automatic segment assembly robot for shield tunneling machine [J]. Computer-Aided Civil and Infrastructure Engineering, 2010, 10(5): 325-337. doi: https://doi.org/10.1111/j.1467-8667.1995.tb00295.x [4] Wu Z, Zhang L, Wang S, et al. Automatic segment assembly method of shield tunneling machine based on multiple optoelectronic sensors[C]//International Conference on Optical Instruments and Technology 2019: Optical Sensor and Applications, 2019, 11436: 14360U. [5] Chow C K, Kaneko T. Boundary detection of radiographic images by a threshold method[M]//Frontiers of Pattern Recognition. Cambridge, Massachusetts: Academic Press, 1972: 61-82. [6] Zhang D D, Zhao S. An improved edge detection algorithm based on Canny operator [J]. Applied Mechanics and Materials, 2013, 347-350(4): 3541-3545. doi: https://doi.org/10.4028/www.scientific.net/AMM.347-350.3541 [7] Carson C, Thomas M, Belongie S, et al. Blobworld: A system for region-based image indexing and retrieval[M]//Visual Information and Information Systems: Proceedings of the Third International Conference on Visual Information and Information Systems. Switzerland: Springer, 1999. [8] Yan Y, Liu G, Wang S, et al. Graph-based clustering and ranking for diversified image search [J]. Multimedia Systems, 2014, 23(1): 41-52. [9] Smith L N. Cyclical learning rates for training neural networks[C]//2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017: 464-472. [10] Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017. [11] He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017. [12] X. Li, H. Li, Y. Lin, et al. Learning-based denoising for polarimetric images [J]. Opt Express, 2020, 28: 16309-16321. doi: 10.1364/OE.391017 [13] Zhang Y, Tian Y, Kong Y, et al. Residual dense network for image restoration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2480-2495. doi: 10.1109/TPAMI.2020.2968521 [14] Hu H, Zhang Y, Li X, et al. Polarimetric underwater image recovery via deep learning [J]. Optics and Lasers in Engineering, 2020, 133: 106152. doi: https://doi.org/10.1016/j.optlaseng.2020.106152