-
针对大尺寸标定基准参照物加工困难问题,采用自适应立体标靶制作方案,即根据测量空间尺寸,构建编码标志点参照物,利用近景摄影测量技术构建立体标定基准。以人体测量为例,立体标靶采用刚体多面立柱体,并在每一面上粘贴环状编码标志点,构成立体标靶。该部分涉及编码标志点的自动识别、圆心校正、光束平差等相关技术[44-47]。其主要步骤如下:
(1)用高分辨了数码相机(尼康D200)在58个不同的视角拍摄获得标靶图像,如图7所示(部分视角拍摄的图像)。
(2)对标靶图像中的编码标志点进行中心定位和解码,根据每个编码点不同的编码值获得其在不同图像之间的对应关系和图像坐标。 (3)利用光束平差(Bundle Adjustment,BA)方法,每个不同编码的世界坐标Xj在拍摄视角i下的重投影的图像坐标为 ${{\overset{\frown} m} _{ij}}(K,\theta ,{r_i},{t_i},{X_j})$ ,优化该重投影误差,如公式(1)所示。$${\rm{cost}}(\varepsilon ) = \sum\limits_{i = 1}^N {\sum\limits_{j = 1}^M {\left\{ {\;{{\left\| {{{\hat m}_{ij}} - {{\overset{\frown} m}_{ij}}(K,\theta ,{r_i},{t_i},{X_j})} \right\|}^2}} \right\}} } $$ (1) 式中:
$(K,\theta )$ 为摄像机的内部结构参数(K为线性内参:焦距、主点、倾斜因子,θ为镜头畸变的非线性内参:径向畸变、切向畸变等);$({r_i},{t_i})$ 为摄像机拍摄的位姿(旋转和平移的6个自由度外部参数);${\hat m_{ij}}$ 为该点圆心图像坐标,由此可得不同编码点的世界坐标Xj,完成标靶校正,如图7所示。 -
对于多节点三维传感器标定较为普遍的标定方案为分步标定法,即先对各个节点三维传感器逐一标定出其内部参数,再确定出各个节点传感器之间的外部结构参数。该方案思想较简洁明了,但由于该方案需对每个节点传感器逐一操作,较为繁琐、耗时、非自动化,而且,外部结构参数的确定仍需额外方法。由于文中三维传感器采用双目结构系统,并借助三维立体标靶,即可实现各节点传感器内部参数与各节点之间外部结构参数同时标定。因此,可将复杂的多节点三维传感器网络标定问题转化为多相机网络标定问题,减少了系统标定的复杂性。
将校正后的标靶置于三维成像系统的测量空间之内,控制系统内的各摄像机采集标靶的图像,根据图像中的标志点标定系统内各摄像机的参数以及各摄像机之间的相互坐标变换关系,完成系统的标定,如图8所示。而测量网的标定是解决问题的关键,每个摄像机的标定是测量网标定的基础,对测量网中第k台摄像机进行标定需要求解如下最优化问题:
$$ \mathop {\min }\limits_{K,\theta ,r,t} \sum\limits_{j = 1}^n {\left\{ {\,{{\left\| {{{\hat m}_j} - {\overset{\frown} m} (K,\theta ,r,t;{X_j})} \right\|}^2}} \right\}} $$ (2) 式中:
$\hat m$ 为基准点图像坐标;${\overset{\frown} m} $ 为由摄像机的非线性模型重投影得到的图像坐标,对摄像机获取的n个所有不同标志点的反投影误差求和即可得公式(2)所示的目标函数。其中,需要确定三维基准点Xj与其图像坐标
${\hat m_j}$ 的对应关系,该处通过环形编码实现,即解码获得每个基准的不同的编码值。在实际应用中,由于遮挡等原因,存在大量编码识别错误,造成对应点错误影响标定结果。为此,提出一种基于随机抽样一致(Random Sample Consensus,RANSAC)算法的局外点剔除方法。为简化一致性条件,仅采用摄像机的线性投影模型,Xj与${\hat m_j}$ 之间的对应关系可以表示为:$$ {\tilde m_j} = M \cdot {\tilde X_j} $$ (3) 式中:
${\tilde m_j}$ 、${\tilde X_j}$ 分别为基准点图像坐标和三维坐标的齐次坐标表示;M为大小为3×4的投影矩阵。能正确解码的基准点都能符合公式(3)模型,称之为“局内点”;受遮挡的编码点不能得到正确的编码值,不能符合公式(3)模型称之为“局外点”。理论上,如果每台摄像机能够一次拍摄到标定靶中足够多的非共面基准点,即可完成摄像机的标定和测量网的标定。为了进一步提高标定结果的可靠性,在实际标定中,通常改变标靶的姿态拍摄M组图像,利用多组图像以提供更多的约束方程,能更准确地优化摄像机内外参和畸变系数,有助于得到更稳定的传感器节点的结构参数和外参。对于某一节点i而言,将双目传感器的结构参数和该节点的外参作为待优化的参数向量,构造新的优化目标函数:
$$\begin{split} \cos t({\tau _i}) =\;& \sum\limits_{t = 1}^N \sum\limits_{s = 1}^M \{ {{\left\| {{{\hat m}_l}^{st} - {{\overset{\frown} m}_l}^{st}(K_l^i,\theta _l^i,r_s^i,t_s^i;{X_w})} \right\|}^2} + \\ & {{\left\| {{{\hat m}_r}^{st} - {{\overset{\frown} m}_r}^{st}(K_r^i,\theta _r^i,{r^i},{t^i},r_s^i,t_s^i;{X_w})} \right\|}^2}\} \end{split}$$ (4) 式中:上标s为系统的第s个拍摄姿态,t为标靶中第t个基准点;
${\tau _i} = \{ K_l^i,\theta _l^i,K_r^i,\theta _r^i,{r^i},{t^i},r_s^i,t_s^i\} $ 为传感器节点i待优化的参数向量,$\{ K_l^i,\theta _l^i\} $ 、$\{ K_r^i,\theta _r^i\} $ 分别为该节点i左右摄像机的内参和畸变,${r^i},{t^i}$ 为双目系统的结构参数,$r_s^i,t_s^i$ 为左摄像在第s次拍摄姿态下的外参;$\hat m_{l/r}^{st}$ 为左右摄像机中基准点的图像坐标;${{\overset{\frown} m} _l}^{st}( \bullet )$ 、${{\overset{\frown} m} _r}^{st}( \bullet )$ 为重投影图像坐标。通过最小化目标函数公式(4),实现对系统参数的最优化估计,得到该节点的结构参数$r_s^i,t_s^i$ ,用于公式实现该节点i的深度重建;将一次拍摄时标靶坐标系为全局世界坐标系,${r^i},{t^i}$ 则表示为节点i与全局世界坐标系变换关系,可用于后期不同传感器测量数据的初始匹配。
Techniques of structured light measurement network with 3D sensors
-
摘要:
针对大尺寸复杂物体的全自动、高精度、大数据密度、真彩色三维成像与测量,基于条纹结构光三维传感器,阐述了多节点三维传感器测量网络相关技术。涉及单三维传感器的条纹分析和相位重建、系统标定和三维重建两大关键技术点分析,多节点三维传感器测量网络的构建与优化、多节点三维传感器测量网络的标定、测量三维深度数据与纹理数据的匹配与融合等相关技术。并给出了部分实验原型机及实验结果。
Abstract:For achieving automatic, high-precision, high-density, and photorealistic 3D imaging and measurement of large-scale complex objects, the techniques about multi-node 3D sensor measurement network were described based on 3D sensing with the fringe structured light. It mainly involved the analysis of two key technologies (fringe analysis and phase reconstruction, system calibration and 3D reconstruction) of single 3D sensing, construction and optimization of multi-node 3D sensor measurement network, calibration of multi-node 3D sensor measurement network, matching and fusion of 3D depth data and texture data. Some experimental prototypes and experimental results were given.
-
[1] Sansoni G, Trebeschi M, Docchio F. State-of-the-art and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation [J]. Sensors, 2009, 9(1): 568−601. doi: 10.3390/s90100568 [2] Schwenke H, Neuschaefer-Rube U, Pfeifer T, et al. Optical methods for dimensional metrology in production engineering [J]. CIRP Annals-Manufacturing Technology, 2002, 51(2): 685−699. doi: 10.1016/S0007-8506(07)61707-7 [3] Blais F. Review of 20 years of range sensor development [J]. Journal of Electronic Imaging, 2004, 13(1): 231−240. doi: 10.1117/1.1631921 [4] Daanen H A M, TerHaar F B. 3D whole body scanners revisited [J]. Displays, 2013, 34: 270−275. doi: 10.1016/j.displa.2013.08.011 [5] Tong J, Zhou J, Liu L, et al. Scanning 3D full human bodies using kinects [J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(4): 643−650. doi: 10.1109/TVCG.2012.56 [6] Geng J. Structured-light 3D surface imaging: A tutorial [J]. Advances in Optics and Photonics, 2011, 3(2): 128−160. doi: 10.1364/AOP.3.000128 [7] He Jinying, Liu Xiaoli, Peng Xiang, et al. Integer pixel correlation searching for three-dimensional digital speckle based on gray constraint [J]. Chinese Journal of Lasers, 2017, 44(4): 150−157. (in Chinese) [8] Tang Q, Liu C, Cai Z, et al. An improved spatiotemporal correlation method for high- accuracy random speckle 3D reconstruction [J]. Optics and Lasers in Engineering, 2018, 110: 54−62. doi: 10.1016/j.optlaseng.2018.05.007 [9] Zuo C, Tao T, Feng S, et al. Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10,000 frames per second [J]. Optics and Lasers in Engineering, 2018, 102: 70−91. doi: 10.1016/j.optlaseng.2017.10.013 [10] Takeda M, Mutoh K. Fourier transform profilometry for the automatic measurement of 3-d object shapes [J]. Applied Optics, 1983, 22(24): 3977−3982. doi: 10.1364/AO.22.003977 [11] Zuo C, Feng S, Huang L, et al. Phase shifting algorithms for fringe projection profilometry: a review [J]. Optics and Lasers in Engineering, 2018, 109: 23−59. doi: 10.1016/j.optlaseng.2018.04.019 [12] Zhong J, Weng J. Spatial carrier-fringe pattern analysis by means of wavelet transform: Wavelet transform profilometry [J]. Applied Optics, 2004, 43(26): 4993−4998. doi: 10.1364/AO.43.004993 [13] Ma J, Wang Z, Pan B, et al. Two-dimensional continuous wavelet transform for phase determination of complex interferograms [J]. Applied Optics, 2011, 50(16): 2425−2430. doi: 10.1364/AO.50.002425 [14] Sutton M A, Zhao M, Mcneill S R, et al. Development and assessment of a single-image fringe projection method for dynamic applications [J]. Experimental Mechanics, 2001, 41(3): 205−217. doi: 10.1007/BF02323136 [15] Srinivasan V, Liu H C, Halioua M, et al. Automated phase-measuring profilometry of 3-d diffuse objects [J]. Applied Optics, 1984, 23(18): 3105−3108. doi: 10.1364/AO.23.003105 [16] Peng J, Liu X, Deng D, et al. Suppression of projector distortion in phase-measuring profilometry by projecting adaptive fringe patterns [J]. Optics Express, 2016, 24(19): 21846−21860. doi: 10.1364/OE.24.021846 [17] Pan J, Huang P S, Chiang F P. Color-coded binary fringe projection technique for 3-D shape measurement [J]. Optical Engineering, 2005, 44(2): 023606. doi: 10.1117/1.1840973 [18] Zhang Z, Towers C E, Towers D P. Time efficient color fringe projection system for simultaneous 3D shape and color using optimum 3-frequency selection [J]. Optics Express, 2006, 14(14): 6444−6455. doi: 10.1364/OE.14.006444 [19] Zuo C, Chen Q, Gu G, et al. High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection [J]. Optics and Lasers in Engineering, 2013, 51: 953−960. doi: 10.1016/j.optlaseng.2013.02.012 [20] Lei S, Zhang S. Flexible 3-D shape measurement using projector defocusing [J]. Optics Letters, 2009, 34(20): 3080−3082. doi: 10.1364/OL.34.003080 [21] Heist S, Mann A, Kühmstedt P, et al. Array projection of aperiodic sinusoidal fringes for high-speed three-dimensional shape measurement [J]. Optical Engineering, 2014, 53(11): 112208. doi: 10.1117/1.OE.53.11.112208 [22] Guan Y, Yin Y, Li A, et al. Dynamic 3D imaging based on acousto-optic heterodyne fringe interferometry [J]. Optics Letters, 2014, 39(12): 3678−3681. doi: 10.1364/OL.39.003678 [23] Du H, Wang Z. Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system [J]. Optics Letters, 2007, 32(16): 2438−2440. doi: 10.1364/OL.32.002438 [24] Huang Z, Xi J, Yu Y, et al. Improved geometrical model of fringe projection profilometry [J]. Optics Express, 2014, 22(26): 32220−32232. doi: 10.1364/OE.22.032220 [25] Zhang Z, Ma H, Zhang S, et al. Simple calibration of a phase-based 3D imaging system based on uneven fringe projection [J]. Optics Letters, 2011, 36(5): 627−629. doi: 10.1364/OL.36.000627 [26] Asundi A, Wensen Z. Unified calibration technique and its applications in optical triangular profilometry [J]. Applied Optics, 1999, 38(16): 3556−3561. doi: 10.1364/AO.38.003556 [27] Huang L, Chua P S K, Asundi A. Least-squares calibration method for fringe projection profilometry considering camera lens distortion [J]. Applied Optics, 2010, 49(9): 1539−1548. doi: 10.1364/AO.49.001539 [28] Léandry I, Brèque C, Valle V. Calibration of a structured-light projection system: development to large dimension objects [J]. Optics and Lasers in Engineering, 2012, 50(3): 373−379. doi: 10.1016/j.optlaseng.2011.10.020 [29] Legarda-Sáenz R, Bothe T, Jüptner W P. Accurate procedure for the calibration of a structured light system [J]. Optical Engineering, 2004, 43(2): 464−471. doi: 10.1117/1.1635373 [30] Yin Y, Peng X, Li A. Calibration of fringe projection profilometry with bundle adjustment strategy [J]. Optics Letters, 2012, 37(4): 542−544. doi: 10.1364/OL.37.000542 [31] Zhang S, Huang P S. Novel method for structured light system calibration [J]. Optical Engineering, 2006, 45(8): 083601. doi: 10.1117/1.2336196 [32] Chen X, Xi J, Jin Y, et al. Accurate calibration for a camera-projector measurement system based on structured light projection [J]. Optics and Lasers in Engineering, 2009, 47(3-4): 310−319. doi: 10.1016/j.optlaseng.2007.12.001 [33] Chen R, Xu J, Chen H, et al. Accurate calibration method for camera and projector in fringe patterns measurement system [J]. Applied Optics, 2016, 55(16): 4293−4300. doi: 10.1364/AO.55.004293 [34] Vargas J, Quiroga J A, Terron-Lopez M J, et al. Flexible calibration procedure for fringe projection profilometry [J]. Optical Engineering, 2007, 46(2): 023601. doi: 10.1117/1.2709855 [35] Huang J, Wu Q. A new reconstruction method based on fringe projection of three-dimensional measuring system [J]. Optics and Lasers in Engineering, 2014, 52: 115−122. doi: 10.1016/j.optlaseng.2013.07.002 [36] Cai Z, Liu X, Li A, et al. Phase-3D mapping method developed from back-projection stereovision model for fringe projection profilometry [J]. Optics Express, 2017, 25(2): 1262−1277. doi: 10.1364/OE.25.001262 [37] Guo J, Peng X, Li A, et al. Automatic and rapid whole-body 3D shape measurement based on multinode 3D sensing and speckle projection [J]. Applied Optics, 2017, 56(31): 8759−8768. doi: 10.1364/AO.56.008759 [38] Peng X, Liu X, Yin Y, et al. Optical measurement network for large-scale and shell-like objects [J]. Optics Letters, 2011, 36(2): 157−159. doi: 10.1364/OL.36.000157 [39] Liu X, Peng X, Chen H, et al. Strategy for automatic and complete three-dimensional optical digitization [J]. Optics Letters, 2012, 37(15): 3126−3128. doi: 10.1364/OL.37.003126 [40] Liu Xiaoli, Peng Xiang, Yin Yongkai, et al. 3D auto-inspection for large thin-wall object [J]. Acta Optica Sinica, 2011, 31(3): 0312006. (in Chinese) doi: 10.3788/AOS201131.0312006 [41] Chen S Y, Li Y F. Automatic sensor placement for model-based robot vision [J]. Part B: Cybernetics, IEEE Transactions on Systems, Man, and Cybernetics, 2004, 34(1): 393−408. doi: 10.1109/TSMCB.2003.817031 [42] Tarabanis K A, Allen P K, Tsai R Y. A survey of sensor planning in computer vision [J]. IEEE Transactions on Robotics and Automation, 1995, 11(1): 86−104. doi: 10.1109/70.345940 [43] Scott W R. Model-based view planning [J]. Machine Vision and Applications, 2009, 20(1): 47−69. doi: 10.1007/s00138-007-0110-2 [44] Liu X, Cai Z, Yin Y, et al. Calibration of fringe projection profilometry using an inaccurate 2D reference target [J]. Optics and Lasers in Engineering, 2017, 89: 131−137. doi: 10.1016/j.optlaseng.2016.05.025 [45] He D, Liu X, Peng X, et al. Eccentricity error identification and compensation for high-accuracy 3D optical measurement [J]. Measurement Science and Technology, 2013, 24(7): 075402. doi: 10.1088/0957-0233/24/7/075402 [46] Yin Yongkai, Liu Xiaoli, Li Ameng, et al. Sub-pixel location of circle target and its application [J]. Infrared and Laser Engineering, 2008, 37(4): 47−50. (in Chinese) [47] Yin Y, Peng X, Liu X, et al. Calibration strategy of optical measurement network for large-scale and shell-like objects [J]. Optics Communications, 2012, 285(8): 2048−2056. doi: 10.1016/j.optcom.2011.12.100 [48] Besl P J, McKay N D. A method for registration of 3D shapes [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239−256. doi: 10.1109/34.121791 [49] Salvi J, Matabosch C, Fofi D, et al. A review of recent range image registration methods with accuracy evaluation [J]. Image and Vision Computing, 2007, 25(5): 578−596. doi: 10.1016/j.imavis.2006.05.012 [50] Liu Xiaoli. Key techniques in multiple range images modeling [D]. Tianjin: Tianjin University, 2008. (in Chinese) [51] Liu X, Peng X, He D, et al. Automatic 3D imaging and modeling system with color information for cultural heritage digitization [C]//Fringe 2013, 2013: 821-826. [52] Li A, Peng X, Yin Y, et al. Optical 3D digitizer for photorealistic imaging of movable cultural heritage [J]. Acta Photonica Sinica, 2013, 42(12): 1421−1429. (in Chinese) doi: 10.3788/gzxb20134212.1421 [53] Liu X, Peng X, Yin Y, et al. Generation of photorealistic 3D image using optical digitizer [J]. Applied Optics, 2012, 51(7): 1304−1311. [54] Liu Xiaoli, Peng Xiang, Yin Yongkai, et al. A method for global registration of range data combined with markers [J]. Acta Optica Sinica, 2009, 29(4): 1010−1014. (in Chinese) doi: 10.3788/AOS20092904.1010 [55] Liu X, He X, Liu Z, et al. Automatic registration of range images combined with the system calibration and global ICP [C]//SPIE, 2012: 8499-1X. [56] Liu Xiaoli, Peng Xiang, Yin Yongkai, et al. Introduction and comparison of range image registration methods [J]. Laser & Optoelectronics Progress, 2010, 47(12): 121001. (in Chinese) [57] Liu Xiaoli, Peng Xiang, Li Ameng, et al. Range images registration combined with texture information [J]. Journal of Computer-aided Design & Computer Graphics, 2007, 19(3): 340−345. (in Chinese) [58] Liu Xingming, Liu Xiaoli, Yin Yongkai, et al. Texture blending of 3D photo-realistic model [J]. Journal of Computer-aided Design & Computer Graphics, 2012, 24(11): 1440−1446. (in Chinese) [59] Liu X, Li A, Zhao X, et al. Model-based optical metrology and visualization of 3-D complex objects [J]. Optoelectronics Letters, 2007, 3(2): 115−118. doi: 10.1007/s11801-007-7018-y [60] Liu Xiaoli, Peng Xiang, Li Ameng, et al. Integration of multiple range images based on ray casting [J]. Journal of Computer-aided Design & Computer Graphics, 2007, 19(10): 1286−1291. (in Chinese)
计量
- 文章访问数: 1223
- HTML全文浏览量: 982
- 被引次数: 0