Special issue-Optical 3D imaging and sensing
2020, 49(3): 0303001.
doi: 10.3788/IRLA202049.0303001
Three-dimensional (3D) imaging and sensing technologies, as valuable information acquisition tools for perceiving the real 3D world, provide data bases for the reconstruction of the geometric shape of objects and subsequent 3D modeling, detection, and recognition. Recently the development of computer vision and optoelectronic imaging technology, as well as the growing demand for 3D technologies in consumer electronics and personal authentication, have promoted the thriving growth of 3D imaging and sensing technologies. After the imaging revolution from monochrome to color, low resolution to high resolution, and static image to dynamic video, the transition of the camera from 2D to 3D will become the new "fourth imaging revolution." This issue of "Infrared and Laser Engineering" organizes a special topic on "Optical 3D Imaging and Sensing", which contains 20 high-quality articles, including 15 review papers and 5 research papers. These papers systematically introduce the research progress or trends of the cutting-edge research topics in the field of optical 3D imaging and sensing, and their themes comprehensively cover the current hot research directions in the field of 3D optical imaging: structured-light 3D imaging, fringe projection profilometry, interferometry, phase measuring deflectometry, 3D display technologies (such as holographic display, and integral/light field display), and the interdisciplinary fields of 3D sensing technologies and computational imaging technologies (such as 3D ghost imaging). As the preface of this issue, this paper summarizes the typical 3D sensing technologies and focuses on the current status, key technologies, and typical applications of the 3D structured-light sensor technologies, discusses its existing challenges, and looks forward to its future development directions.
2020, 49(3): 0303002.
doi: 10.3788/IRLA202049.0303002
Mobile device has been the most important personal computational platform. Researches increasingly focus on glasses-free 3D display that features thin form factor and low power consumption. Once glasses-free 3D display is applied in the mobile devices, it will affect or even change the observation habit and thinking manner of human being. In this paper, the solutions that were compatible with flat display were introduced. Parallax barrier, lenticular lens array, spatio-temporal multiplexing 3D display, integral 3D display, compressive lightfield display, and vector 3D display were successively introduced in detail. Furthermore, the limitations that currently prevent these technologies from commercialization were discussed in depth. Finally, the potential research trends were highlighted.
2020, 49(3): 0303003.
doi: 10.3788/IRLA202049.0303003
As a novel auto-stereoscopic three-dimensional (3D) display technology, integral imaging is able to acquire complete 3D information of the 3D scene and reconstruct it. Thus, integral imaging is one of the most promising technologies of the glasses-free 3D displays. In the integral imaging 3D information acquisition process, different perspectives of the 3D scene are recorded to generate the elemental image array to be the 3D content for the integral imaging display. On the other hand, the integral imaging 3D acquisition can be a multi-dimensional information pickup approach applied to the detection and imaging of occluded scenes, target recognition, two-dimensional (2D) / 3D information encryption and microscopic 3D imaging fields. The basic principles, different types of the integral imaging 3D information acquisition approaches, and real-time 3D information acquisition technologies were reviewed in this paper. Some challenging problems and future development trend of the technology were also discussed.
2020, 49(3): 0303004.
doi: 10.3788/IRLA202049.0303004
Three-dimensional (3D) imaging technique based on structured illumination has been deeply studied and widely applied in recent years. Among numerous 3D imaging methods, the imaging technique based on Gray-coded structured illumination has been applied in industrial inspection, digitalization of antique, biomedicine detection and so on, owing to its robustness and anti-noise ability. To realize 3D imaging, the projected Gray-coded pattern can either be used to modulate the height information or be used to assist acquisition and computation of the height information of the tested object. In this paper, a review was given to introduce principles and progresses of the imaging techniques based on Gray-coded structured illumination and the research achievements of the author's research group about this technique were also shown. In addition, the advantages, disadvantages and application scope of binary Gray-coded patterns in 3D shape measurement were discussed and the developing trends of this technique was pointed out as well.
2020, 49(3): 0303005.
doi: 10.3788/IRLA202049.0303005
The dynamic three-dimensional(3D) surface imaging technology based on fringe projection was summarized and the characteristics of typical strategies were analyzed. Firstly, the basic principle of the technique was introduced, including the basic structure of the system, the principle of 3D surface imaging, and the two typical phase extraction methods, phase shifting method and Fourier transform method. Secondly, the dynamic imaging scheme including high speed projection scheme and dynamic 3D surface imaging method using relatively low speed camera was introduced. Thirdly, the absolute phase acquisition methods in dynamic 3D image were introduced, mainly analog and digital coding and area statistical coding, were introduced. Finally, the methods of accurate dynamic phase measurement were introduced.
2020, 49(3): 0303006.
doi: 10.3788/IRLA202049.0303006
There are a large number of three-dimensional (3D) shape measurement requirements and applications for objects with shiny surfaces, such as ceramics, ancient artifacts, and metal workpieces. However, the traditional fringe projection method cannot accurately measure shiny surfaces due to the large range of reflectivity of the objects with shiny surfaces and the limited maximum gray level of the camera. This paper reviewed the recent developments, application fields and future research directions of high dynamic range 3D shape measurement technologies of shiny surface. First, according to the principle and measurement method, the existing 3D measurement technologies of high dynamic range were divided into the following six categories: multiple exposures methods, projected patterns intensities adjusting methods, polarizing filters methods, color invariants methods, photometric stereo technology and miscellaneous technologies. Then, the advantages and disadvantages of these technologies were compared in detail and their adaptability was summarized. Finally, this paper summarized the application fields of shiny surfaces in 3D shape measurement technology and prospects the future research directions. It can provide users with the optimal 3D measurement method according to the different application requirements and the different measurement conditions, eventually the 3D shape of shiny surfaces can be reconstructed more accurately.
2020, 49(3): 0303007.
doi: 10.3378/IRLA202049.0303007
For achieving automatic, high-precision, high-density, and photorealistic 3D imaging and measurement of large-scale complex objects, the techniques about multi-node 3D sensor measurement network were described based on 3D sensing with the fringe structured light. It mainly involved the analysis of two key technologies (fringe analysis and phase reconstruction, system calibration and 3D reconstruction) of single 3D sensing, construction and optimization of multi-node 3D sensor measurement network, calibration of multi-node 3D sensor measurement network, matching and fusion of 3D depth data and texture data. Some experimental prototypes and experimental results were given.
2020, 49(3): 0303008.
doi: 10.3788/IRLA202049.0303008
Fringe projection profilometry (FPP) can well balance the system flexibility and the measurement accuracy, which is the mainstream technology of optical 3D imaging and shape measurement. When working with FPP, first an appropriate system should be established, and then the system parameters describing the system model should be determined via system calibration, finally the 3D shape is generated with 3D reconstruction by using the calibrated system model. System calibration and system model are tightly coupled, which have a direct impact on the performance of 3D imaging. The principle of FPP can be divided into two categories, i.e. the phase-3D mapping and the binocular stereo vision. This paper reviewed the system model and calibration strategy of FPP, which corresponded to the above two categories. Finally the method and basis for the accuracy evaluation of FPP were summarized.
2020, 49(3): 0303009.
doi: 10.3788/IRLA202049.0303009
In fringe projection profilometry, the luminance nonlinearity of the projector has been recognized as one of the most crucial factors decreasing the measurement accuracy. It induces the ripple-like artifacts on the measured phase map. The self-adaptive correcting algorithms, i.e., self-correcting algorithms, allow us to suppress the effect of the projector nonlinearity without a prior calibration for the projector intensities or phase errors. This paper introduces the research progress in the self-correcting algorithms. Among them, the first algorithm is to determine a nonlinear curve representing the projector nonlinearity, directly from the captured fringe patterns, thus correcting the phase errors using this curve. The second one is to recognize and remove the nonlinearity-induced errors, directly from a calculated phase map. With the last one, error function coefficients are estimated from a couple of phase maps having different frequencies. Measurement results demonstrate these self-correcting algorithms to be effective in suppressing influences of the projector nonlinearity in the absence of any calibration information.
2020, 49(3): 0303010.
doi: 10.3788/IRLA202049.0303010
Structured light has developed rapidly as a non-contact active three-dimensional measurement method. The phase coding method based on the principle of structured light embeds the codeword into the intensity of the fringe in the form of phase, which is insensitive to surface contrast, ambient light and camera noise due to the use of phase instead of intensity to determine the code word, so it has stronger robustness. The research progress of three dimensional measurement methods of phase-coding was reviewed, and the three-dimensional measurement principles based on phase encoding was summarized, including the design method of initial phase, the generation method of encoded fringes, phase calculation methods, acquisition and correction of fringe order, and acquisition of absolute phase. Some experimental results were given, and the advantages and disadvantages of the phase coding method were pointed out. Finally, the challenges faced by the technology were discussed, and the future development direction of the field was pointed out.
2020, 49(3): 0303011.
doi: 10.3788/IRLA202049.0303011
Phase shifting profilometry (PSP) can reach high accuracy for the 3D shape measurement of static object. However, errors will be introduced when moving object was reconstructed. The fundamental reason was PSP required multiple fringe patterns to reconstruct the object and the traditional PSP did not contain movement information. Aiming at the above, a new method for the 3D reconstruction multiple 2D moving objects was proposed. Different objects can have different movement. Firstly, the multiple objects were identified and the areas of interests are defined. Then, the KCF algorithm was used to track the moving object and SIFT algorithm was used to retrieve the feature points of the object before movement and after movement. The rotation matrix and translation vector describing the movement was then obtained. With the help of the reconstruction model with movement information, the least-square algorithm was employed to retrieve the correct phase value. The results show that the proposed method can reduce the errors introduced by the movement and has the potential to be applied in industrial field.
2020, 49(3): 0303012.
doi: 10.3788/IRLA202049.0303012
With the quick development and progress of the synchrotron radiation and free electron laser facilities, the figure error requirement for X-ray delivering and focusing mirrors is getting higher. To preserve the wave-front without introducing additional wave-front error, the mirror surface figure error is typically specified approaching the sub-nm root mean square level. This kind of ultimate specification challenges the X-ray mirror metrology and inspection. In addition to the profile scanners based on the angular measurement, such as the long trace profilometer, which have been widely used in the synchrotron light source in various countries, stitching interferometry has also been developed as an effective method for synchrotron mirror metrology. In this work, the dedicated stitching metrology platform for X-ray mirror metrology was developed. Several stitching methods with varying stitching parameters were investigated at the proposed stitching interferometric system. In this paper, we focused on the principle and measurements of using the software stitching mode. The measurement results were compared with those from several different instruments in different research institutions to verify the effectiveness of the stitching interferometry for the synchrotron mirror inspection and to demonstrate the performance of the stitching platform. According to the measurement data, the repeatability of measuring an X-ray flat mirror in the software stitching mode is at 0.1 nm RMS level. When measuring a curved mirror with its radius of curvature changing from 50 m to 130 m, the repeatability is around 0.2-0.3 nm RMS. Basically, it meets the requirement for the routine inspection and fabrication feedback of flat and near-flat (radius of curvature larger than 50 m) X-ray mirrors.
2020, 49(3): 0303013.
doi: 10.3788/IRLA202049.0303013
Fringe pattern analysis is an important content for optical measurement. The fringe pattern analysis have been researched for a long time, which have introduced the Fourier transform, the windowed Fourier transform, the wavelet transform, the S transform, the recent variable mode decomposition and empirical mode decomposition (EMD) and etc. Compared with other methods, EMD has strong abilities being adaptive and suitable for complex signals, so it gets more attentions in recent years. In this paper, combined with the key contents of fringe pattern analysis, the EMD algorithms and their applications in fringe pattern processing were reviewed. The remaining key issues and technical difficulties were concluded, which were expected to provide some reference for future researches in relevant fields.
2020, 49(3): 0303014.
doi: 10.3788/IRLA202049.0303014
The simultaneous non-contact measurement of the front and rear surfaces of parallel plates by using wavelength phase-shifting interference technology was of great significance in the field of optical determination. A time-domain weighting algorithm was proposed to solve the overlapping interference signals of the front and rear surfaces of parallel plates: weighted 36-step sampling algorithm. Firstly, based on the principle and constraints of the algorithm, the basic distribution parameters of the weighted multi-step sampling algorithm were designed, and then the sampling weights of the interference signals of front surface, rear surface and thickness variance were obtained. Through the calculation of the basic distribution parameters, the sampling weight of each surface can be obtained, and the initial phase distribution of each interference information can be obtained by using the weight operation. Based on Zernike polynomials, the simulation of multi-surface interference phase solution was carried out, and the maximum error between the simulation result and the true value was not more than 0.06 nm. In addition, the influence of various errors on the measurement results was analyzed. A parallel plate with a thickness of 20 mm has been measured. The experiment verifies the effectiveness of the algorithm in the actual measurement process.
2020, 49(3): 0303015.
doi: 10.3378/IRLA202049.0303015
Optical three-dimensional (3D) measurement technology is widely used in different fields due to its advantages of non-contact, non-destructive and rapid measurement. The existing technologies of Fringe Projection Measurement and Fringe Reflection Measurement are designed for the measurement of diffusely reflective and specularly reflective surfaces respectively. However, there are many complex surfaces which have both diffused and specular reflective surfaces together in aerospace and advanced manufacturing. In this paper, a method based on structured light projection and reflection was proposed to realize rapid measurement of complex surfaces. First, a Digital Light Projector (DLP) projected blue sinusoidal fringe onto the tested surface, and red fringes displayed by a screen were reflected by its specular part simultaneously. Second, deformed fringe patterns modulated by the measured surface were captured by a color Charge Coupled Device (CCD) camera. Then, deformed fringes of different reflection surface were extracted from different color channels of the camera and then absolute phase information could be calculated. Finally, after system calibration to build up the relationship between phase and depth, 3D shape data of the measured complex object was obtained. Experimental results show that the proposed method can not only measure complex surfaces effectively, but obtain 3D shape of isolated diffused and specular surfaces simultaneously.
2020, 49(3): 0303016.
doi: 10.3788/IRLA202049.0303016
Different from traditional array-detector imaging technology, single pixel imaging(SPI) technology, which is considered as a novel computational imaging technology, uses a bucket detector with no spatial resolution capability and combines with the spatial light field modulation technology to reconstruct the intensity of the object by correlation algorithm, which has received extensive attention. In recent years, SPI has been widely used with different light source, especially in some special wavebands, where the array-detectors are expensive or even impossible to be made. SPI has gradually developed into an alternative technology with low cost and high quality. In addition, a large number of works of SPI have been proposed in 3D imaging technology. The basic principle and application development of single pixel imaging was introduced. Its application in fringe projection 3D imaging was also introduced emphatically.
2020, 49(3): 0303017.
doi: 10.3788/IRLA202049.0303017
Traditional optical 3D shape measurement methods, such as Fringe projection techniques, cannot acquire high-quality and high-accuracy 3D measurement results in the presence of global illumination. Typical global illumination effects contain interreflections and subsurface scattering. Interreflections occur in concave surfaces with glossy reflection, and subsurface scattering occurs in translucent materials. Single-pixel imaging (SI) techniques can capture a scene through a detector with no spatial resolution. However, traditional pixelated imaging sensors are commonly adopted in most modern digital cameras. Here, we extended SI to pixelated imaging sensors, in which every pixel on an imaging sensor was considered an independent unit that can simultaneously obtain an image. Our experiments show that the SI can completely decompose direct and global illumination. Furthermore, high-quality and high-accuracy 3D profile in the presence of global illumination can be reconstructed.
2020, 49(3): 0303018.
doi: 10.3788/IRLA202049.0303018
Fringe projection(structured light) 3D imaging is a widely used 3D imaging method. In recent years, the integrated three-dimensional sensor has developed rapidly, especially the three-dimensional sensor based on the principle of structured light has gradually become an essential sensor unit for high-end smart phones. However, with the increasing requirements from applications, people have higher and higher requirements on the efficiency, accuracy, stability and other aspects for the fringe projection technique. At the same time, the rapid development of deep learning technology has opened a new door for the development of optical imaging technology, and from this door we notice that with the introduction of the concept of artificial intelligence, the development of fringe projection technology is also experiencing a new breakthrough. In this paper, the basic theory of fringe projection 3D imaging was introduced. Then, by using the deep learning technology, the fringe projection technology based on the physical model can become a technology driven by "data", and in this case, it showed the potential to surpass the traditional algorithm. Finally, the challenges and future research directions in this field from the aspects of neural network model, training data, training methods and so on were discussed.
2020, 49(3): 0303019.
doi: 10.3378/IRLA202049.0303019
A method for three-dimensional (3D) reconstruction from four-dimensional (4D) light fields was presented. The 4D light field image recorded the direction and intensity of all rays passing through the scene and contained useful information to estimate scene depth. Point 3D coordinates were obtained by calculating relative positional relationships between rays emitted from one point in the scene. However, it was very difficult in practice to determine these light rays from 4D light field data. The proposed method used fringe projection to mark object surfaces. Light ray information can then be accurately and quickly determined from the phase marker and 3D data calculated. The 4D light field matrix was light ray phase rather of intensity as in the conventional method and can record rays with various directions. Thus, shadow, occlusion, and surface specular reflection problems can be addressed. Feasibility and accuracy of the proposed method were verified experimentally.
2020, 49(3): 0303020.
doi: 10.3378/IRLA202049.0303020
In a Computational Ghost Imaging (CGI) system, the axial depth of the target can be obtained by estimating the degree of blur of the reconstructed image. However, this method is easy to be affected by background noise and requires a long working distance for the image quality evaluation function, so this method needs more samplings and the practicability is reduced. To solve this problem, a target depth estimated algorithm with adapted focusing window was proposed. Firstly the local search interval was divided according to the global characteristics of the evaluation function, and then the actual axial depth of the target was searched iteratively in a given region. In iterations, the use of adaptive window decreased the area of background and contained the whole target. Experiments show that the proposed method greatly reduces the necessary working distance, increases the robustness of this method, reduces the effect of background noise on the evaluation function, and achieves the depth of target under low samplings. This work promotes the development of depth estimation method based on computational ghost imaging system.
- First
- Prev
- 1
- 2
- Next
- Last
- Total:2
- To
- Go
