-
文中系统综述了集成成像3D信息获取技术的进展,包括微透镜阵列、摄像机阵列和深度相机,还介绍了集成成像3D信息实时获取的发展历程和研究现状,但这些依然无法涵盖集成成像3D信息获取技术的全部研究工作。从集成成像的角度来看,集成成像3D信息获取的主要目的是为对应的显示过程提供3D内容,这就使得无论何种采用获取方式,都需要将得到的原始数据进行处理,才能够与显示过程匹配。不同的获取方式各自具有不同的优势,也面临着不同的挑战。表1横向比较了文中所述的三种集成成像3D信息获取方式在结构、获取速度、数据量、获取分辨率、采集视场、微图像阵列质量、成本及体积等方面的参数。可以看出,基于微透镜阵列的集成成像3D信息获取方式结构最简单,获取速度最快,数据量最小,然而获取分辨率却最低、采集视场也最小;基于摄像机阵列的集成成像3D信息获取方式采集视场最大,获取分辨率最高,但是结构也最为复杂,数据量大,获取速度慢,成本也最高;基于深度相机的集成成像3D信息获取方式平衡了上述两种获取方式的多项参数,整体3D信息获取性能处于中等。
表 1 三种集成成像3D信息获取方式的参数比较
Table 1. Parameter comparison between three integral imaging 3D information acquisition technologies
Complexity of system Pickup speed Amount of data Pickup resolution FOV EIA quality Cost Size Microlens array Low High Small Low Small Low Low Small Camera array High Low Large High Large High High Large Depth camera Moderately low Moderately low Moderately small Moderately high Moderately large Moderately low Moderately low Small 除了为集成成像3D显示提供内容外,集成成像3D信息获取还可以作为一种多维度信息获取手段,在被遮挡场景的探测成像和目标识别追踪、2D/3D信息加密、显微3D成像等领域实现应用。就2D/3D信息加密来说,集成成像3D信息获取得到的微图像阵列同时包含了目标场景不同空间位置和不同方向的信息,具有冗余特性,因此将获取过程作为加密算法的一部分,具有较高的安全性,能够抵抗多种类型的攻击。可以看出,集成成像获取3D信息的应用空间还有很大,在未来的研究中,与不同领域实际应用的结合也是相关科研人员的一个努力方向。
总之,集成成像3D信息获取技术能够记录目标场景不同角度的视差信息,不仅为集成成像3D显示提供匹配片源,还可以作为新型的多维度信息获取手段,为计算机视觉等方向提供新的思路,在科学研究、工业检查、农业生产、医疗影像、环境检查及军事侦察等领域具有较大的应用价值。随着新型成像器件的不断研发和人工智能技术的迭代更新,集成成像3D信息获取技术的潜力将不断被开发,相应的应用方向也将不断拓展。
3D information acquisition technology of integral imaging
-
摘要: 集成成像作为一种新型裸眼三维(3D)显示技术,能够完整获取3D信息并还原再现,是目前最具发展前景的裸眼3D显示技术之一。其中,对3D信息的完整获取通过记录场景不同角度的视差信息,并以微图像阵列形式进行呈现,不仅能够为集成成像显示提供3D内容,更作为一种多维度信息获取手段,在被遮挡场景的探测成像、目标识别、2D/3D信息加密和显微3D成像等多个领域得到应用。文中对集成成像3D信息获取的基本原理、不同类型的集成成像3D信息获取技术以及集成成像3D信息实时获取技术进行综述,并讨论现存的一些问题以及未来发展趋势。Abstract: As a novel auto-stereoscopic three-dimensional (3D) display technology, integral imaging is able to acquire complete 3D information of the 3D scene and reconstruct it. Thus, integral imaging is one of the most promising technologies of the glasses-free 3D displays. In the integral imaging 3D information acquisition process, different perspectives of the 3D scene are recorded to generate the elemental image array to be the 3D content for the integral imaging display. On the other hand, the integral imaging 3D acquisition can be a multi-dimensional information pickup approach applied to the detection and imaging of occluded scenes, target recognition, two-dimensional (2D) / 3D information encryption and microscopic 3D imaging fields. The basic principles, different types of the integral imaging 3D information acquisition approaches, and real-time 3D information acquisition technologies were reviewed in this paper. Some challenging problems and future development trend of the technology were also discussed.
-
表 1 三种集成成像3D信息获取方式的参数比较
Table 1. Parameter comparison between three integral imaging 3D information acquisition technologies
Complexity of system Pickup speed Amount of data Pickup resolution FOV EIA quality Cost Size Microlens array Low High Small Low Small Low Low Small Camera array High Low Large High Large High High Large Depth camera Moderately low Moderately low Moderately small Moderately high Moderately large Moderately low Moderately low Small -
[1] Lippmann G. La photographie integrale [J]. Comptes Rendus Academie Sci, 1908, 146: 446−451. [2] Martínez-Corral M, Javidi B. Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems [J]. Advances in Optics and Photonics, 2018, 10(3): 512−566. doi: 10.1364/AOP.10.000512 [3] Martínez-Corral M, Dorado A, Barreiro J C, et al. Recent advances in the capture and display of macroscopic and microscopic 3-D scenes by integral imaging [J]. Proceedings of the IEEE, 2017, 105(5): 825−836. doi: 10.1109/JPROC.2017.2655260 [4] Yang J C, Everett M, Buehler C, et al. A real-time distributed light field camera [J]. Rendering Techniques, 2002, 2002: 77−86. [5] Ng R, Levoy M, Brédif M, et al. Light field photography with a hand-held plenoptic camera [J]. Computer Science Technical Report CSTR, 2005, 2(11): 1−11. [6] Xiao X, Javidi B, Martinez-Corral M, et al. Advances in three-dimensional integral imaging: sensing, display, and applications [J]. Applied Optics, 2013, 52(4): 546−560. doi: 10.1364/AO.52.000546 [7] Martinez-Cuenca R, Pons A, Saavedra G, et al. Optically-corrected elemental images for undistorted integral image display [J]. Optics Express, 2006, 14(21): 9657−9663. doi: 10.1364/OE.14.009657 [8] Arai J, Okano F, Hoshino H, et al. Gradient-index lens-array method based on real-time integral photography for three-dimensional images [J]. Applied Optics, 1998, 37(11): 2034−2045. doi: 10.1364/AO.37.002034 [9] Hahn J, Kim Y, Kim E H, et al. Undistorted pickup method of both virtual and real objects for integral imaging [J]. Optics Express, 2008, 16(18): 13969−13978. doi: 10.1364/OE.16.013969 [10] Ives H E. Optical properties of a Lippmann lenticulated sheet [J]. Journal of the Optical Society of America, 1931, 21(3): 171−176. doi: 10.1364/JOSA.21.000171 [11] Jang J S, Javidi B. Two-step integral imaging for orthoscopic three-dimensional imaging with improved viewing resolution [J]. Optical Engineering, 2002, 41(10): 2568−2572. doi: 10.1117/1.1505961 [12] Okano F, Hoshino H, Arai J, et al. Real-time pickup method for a three-dimensional image based on integral photography [J]. Applied Optics, 1997, 36(7): 1598−1603. doi: 10.1364/AO.36.001598 [13] Navarro H, Martínez-Cuenca R, Saavedra G, et al. 3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC) [J]. Optics Express, 2010, 18(25): 25573−25583. doi: 10.1364/OE.18.025573 [14] Xiao X, Shen X, Martinez-Corral M, et al. Multiple-planes pseudoscopic-to-orthoscopic conversion for 3D integral imaging display [J]. Journal of Display Technology, 2015, 11(11): 921−926. doi: 10.1109/JDT.2014.2387854 [15] Martínez-Corral M, Dorado A, Navarro H, et al. Three-dimensional display by smart pseudoscopic-to-orthoscopic conversion with tunable focus [J]. Applied Optics, 2014, 53(22): E19−E25. doi: 10.1364/AO.53.000E19 [16] Wang Z, Lv G, Feng Q, et al. A fast-direct pixel mapping algorithm for displaying orthoscopic 3D images with full control of display parameters [J]. Optics Communications, 2018, 427: 528−534. doi: 10.1016/j.optcom.2018.06.067 [17] Yan Z, Jiang X, Yan X. Performance-improved smart pseudoscopic to orthoscopic conversion for integral imaging by use of lens array shifting technique [J]. Optics Communications, 2018, 420: 157−162. doi: 10.1016/j.optcom.2018.03.061 [18] Deng H, Wang Q H, Li D H, et al. Realization of undistorted and orthoscopic integral imaging without black zone in real and virtual fields [J]. Journal of Display Technology, 2011, 7(5): 255−258. doi: 10.1109/JDT.2011.2106761 [19] Jang J S, Javidi B. Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging [J]. Optical Engineering, 2003, 42(7): 1869−1871. doi: 10.1117/1.1584054 [20] Zhang J L, Wang X R, Chen Y J, et al. Feasibility study for pseudoscopic problem in integral imaging using negative refractive index materials [J]. Optics Express, 2014, 22(17): 20757−20769. doi: 10.1364/OE.22.020757 [21] Algorri J F, Urruchi V, Bennis N, et al. Integral imaging capture system with tunable field of view based on liquid crystal microlenses [J]. IEEE Photonics Technology Letters, 2016, 28(17): 1854−1857. doi: 10.1109/LPT.2016.2572258 [22] Martínez-Corral M, Javidi B, Martínez-Cuenca R, et al. Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays [J]. Applied Optics, 2004, 43(31): 5806−5813. doi: 10.1364/AO.43.005806 [23] Luo C G, Wang Q H, Deng H, et al. Extended depth-of-field in integral-imaging pickup process based on amplitude-modulated sensor arrays [J]. Optical Engineering, 2015, 54(7): 073108. doi: 10.1117/1.OE.54.7.073108 [24] Luo C G, Deng H, Li L, et al. Integral imaging pickup method with extended depth-of-field by gradient-amplitude modulation [J]. Journal of Display Technology, 2016, 12(10): 1205−1211. doi: 10.1109/JDT.2016.2580098 [25] Yim J, Choi K H, Min S W. Real object pickup method of integral imaging using offset lens array [J]. Applied Optics, 2017, 56(13): F167−F172. doi: 10.1364/AO.56.00F167 [26] Jeong Y, Kim J, Yeom J, et al. Real-time depth controllable integral imaging pickup and reconstruction method with a light field camera [J]. Applied Optics, 2015, 54(35): 10333−10341. doi: 10.1364/AO.54.010333 [27] Jang J S, Javidi B. Three-dimensional synthetic aperture integral imaging [J]. Optics Letters, 2002, 27(13): 1144−1146. doi: 10.1364/OL.27.001144 [28] Wilburn B, Joshi N, Vaish V, et al. High performance imaging using large camera arrays[C]//ACM Transactions on Graphics (TOG), 2005, 24(3): 765-776. [29] Taguchi Y, Koike T, Takahashi K, et al. TransCAIP: A live 3D TV system using a camera array and an integral photography display with interactive control of viewing parameters [J]. IEEE Transactions on Visualization and Computer Graphics, 2009, 15(5): 841−852. doi: 10.1109/TVCG.2009.30 [30] Deng H, Wang Q, Li D. Method of generating orthoscopic elemental image array from sparse camera array [J]. Chinese Optics Letters, 2012, 10(6): 061102. doi: 10.3788/COL201210.061102 [31] Xing Y, Xiong Z L, Zhao M, et al. Real-time integral imaging pickup system using camera array[C]//Advances in Display Technologies VIII. International Society for Optics and Photonics, 2018, 10556: 105560D. [32] Xiong Z L, Xing Y, Deng H, et al. 19‐1: Planar parallax based camera array calibration method for integral imaging three‐dimensional information acquirement[C]//SID Symposium Digest of Technical Papers. 2016, 47(1): 219−222. [33] Danesh Panah M, Javidi B, Watson E A. Three dimensional imaging with randomly distributed sensors [J]. Optics Express, 2008, 16(9): 6368−6377. doi: 10.1364/OE.16.006368 [34] Xiao X, DaneshPanah M, Cho M, et al. 3D integral imaging using sparse sensors with unknown positions [J]. Journal of Display Technology, 2010, 6(12): 614−619. doi: 10.1109/JDT.2010.2070485 [35] Zhang M, Zhong Z, Piao Y, et al. Three-dimensional integral imaging with circular non-uniform distribution [J]. Optics and Lasers in Engineering, 2020, 126: 105912. doi: 10.1016/j.optlaseng.2019.105912 [36] Yi F, Lee J, Moon I. Three-dimensional integral imaging by using unknown sensor array position calibration with a closed-form solution [J]. Optical Engineering, 2016, 55(2): 023104. doi: 10.1117/1.OE.55.2.023104 [37] Sotoca J M, Latorre-Carmona P, Pla F, et al. Integral imaging techniques for flexible sensing through image-based reprojection [J]. Journal of the Optical Society of America A, 2017, 34(10): 1776−1786. doi: 10.1364/JOSAA.34.001776 [38] Wei J, Wang S, Zhao Y, et al. Synthetic aperture integral imaging using edge depth maps of unstructured monocular video [J]. Optics Express, 2018, 26(26): 34894−34908. doi: 10.1364/OE.26.034894 [39] Shin D H, Lee S H, Kim E S. Optical display of true 3D objects in depth-priority integral imaging using an active sensor [J]. Optics Communications, 2007, 275(2): 330−334. doi: 10.1016/j.optcom.2007.03.072 [40] Li G, Kwon K C, Shin G H, et al. Simplified integral imaging pickup method for real objects using a depth camera [J]. Journal of the Optical Society of Korea, 2012, 16(4): 381−385. doi: 10.3807/JOSK.2012.16.4.381 [41] Jeong J S, Kwon K C, Erdenebat M U, et al. Development of a real-time integral imaging display system based on graphics processing unit parallel processing using a depth camera [J]. Optical Engineering, 2014, 53(1): 015103. doi: 10.1117/1.OE.53.1.015103 [42] Yoo H. Transformations and their analysis from a RGBD image to elemental image array for 3D integral imaging and coding [J]. KSII Transactions on Internet and Information Systems, 2018, 12(5): 2273−2286. [43] Hong S, Dorado A, Saavedra G, et al. Three-dimensional integral-imaging display from calibrated and depth-hole filtered kinect information [J]. Journal of Display Technology, 2016, 12(11): 1301−1308. doi: 10.1109/JDT.2016.2594076 [44] Erdenebat M U, Piao Y L, Darkhanbaatar N, et al. Advanced mobile three-dimensional display based on computer-generated integral imaging[C]//Optics, Photonics, and Digital Technologies for Imaging Applications V. International Society for Optics and Photonics, 2018, 10679: 106790I. [45] Jeong J S, Erdenebat M U, Kwon K C, et al. Real object-based integral imaging system using a depth camera and a polygon model [J]. Optical Engineering, 2017, 56(1): 013110. doi: 10.1117/1.OE.56.1.013110 [46] Hong S, Saavedra G, Martinez-Corral M. Full parallax three-dimensional display from Kinect v1 and v2 [J]. Optical Engineering, 2016, 56(4): 041305. doi: 10.1117/1.OE.56.4.041305 [47] Ikeya K, Arai J, Mishina T, et al. Capturing method for integral three-dimensional imaging using multiviewpoint robotic cameras [J]. Journal of Electronic Imaging, 2018, 27(2): 023022. [48] Erdenebat M U, Kim B J, Piao Y L, et al. Three-dimensional image acquisition and reconstruction system on a mobile device based on computer-generated integral imaging [J]. Applied Optics, 2017, 56(28): 7796−7802. doi: 10.1364/AO.56.007796 [49] Xiong Z L, Wang Q H, Xing Y, et al. Active integral imaging system based on multiple structured light method [J]. Optics Express, 2015, 23(21): 27094−27104. doi: 10.1364/OE.23.027094 [50] Halle M. Multiple viewpoint rendering[C]//SIGGRAPH, 1998, 98: 243−254. [51] Min S W, Park K S, Lee B, et al. Enhanced image mapping algorithm for computer-generated integral imaging system [J]. Japanese Journal of Applied Physics, 2006, 45(7L): L744. [52] Min S W, Kim J, Lee B. New characteristic equation of three-dimensional integral imaging system and its applications [J]. Japanese Journal of Applied Physics, 2004, 44(1L): L71. [53] Kwon K C, Park C, Erdenebat M U, et al. High speed image space parallel processing for computer-generated integral imaging system [J]. Optics Express, 2012, 20(2): 732−740. doi: 10.1364/OE.20.000732 [54] Kim D H, Erdenebat M U, Kwon K C, et al. Real-time 3D display system based on computer-generated integral imaging technique using enhanced ISPP for hexagonal lens array [J]. Applied Optics, 2013, 52(34): 8411−8418. doi: 10.1364/AO.52.008411 [55] Jiao S, Wang X, Zhou M, et al. Multiple ray cluster rendering for interactive integral imaging system [J]. Optics Express, 2013, 21(8): 10070−10086. doi: 10.1364/OE.21.010070 [56] Li S L, Wang Q H, Xiong Z L, et al. Multiple orthographic frustum combing for real-time computer-generated integral imaging system [J]. Journal of Display Technology, 2014, 10(8): 704−709. doi: 10.1109/JDT.2014.2315665 [57] Chen G, Ma C, Fan Z, et al. Real-time lens based rendering algorithm for super-multiview integral photography without image resampling [J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(9): 2600−2609. [58] Xing S, Sang X, Yu X, et al. High-efficient computer-generated integral imaging based on the backward ray-tracing technique and optical reconstruction [J]. Optics Express, 2017, 25(1): 330−338. doi: 10.1364/OE.25.000330 [59] Wikipedia. Ray tracing (graphics)[EB/OL]. [2020-01-10]. https://en.wikipedia.org/wiki/Ray_tracing_(graphics). [60] Huo W, Sang X, Xing S, et al. Backward ray tracing based rectification for real-time integral imaging display system [J]. Optics Communications, 2020, 458: 124752. doi: 10.1016/j.optcom.2019.124752 [61] Choi J G, Choi H M, Hwang Y S, et al. Real-time sensing and three-dimensional display of far outdoor scenes based on asymmetric integral imaging [J]. Optics and Lasers in Engineering, 2017, 94: 44−57. doi: 10.1016/j.optlaseng.2017.02.013 [62] Zhao Min, Xiong Zhaolong, Xing Yan, et al. Real-time integral imaging pickup system based on binocular stereo camera [J]. Infrared and Laser Engineering, 2017, 46(11): 1103007. (in Chinese) doi: 10.3788/IRLA201746.1103007
计量
- 文章访问数: 1931
- HTML全文浏览量: 716
- 被引次数: 0