-
单像素成像技术通过空间光调制器调制光场产生掩膜图案,同时用一个不具空间分辨能力的单像素探测器收集光强。如果掩膜图案采用正交化的Hadamard矩阵[26-29],则N次(场景的总像素个数为N)测量后可以完美重建场景:
$$ I = \frac{1}{N}\sum\limits_{i = 0}^N {{P_i} \cdot {S_i}} $$ (1) 式中:
$ I $ 表示重建的图像;$ {P_i} $ 表示Hadamard图案;$ {S_i} $ 表示单像素探测器收集的光强值。 -
紧凑双光路单像素成像系统由镜头、 DMD、平面反射镜、光束收集透镜、单像素探测器组成,如图1(a)所示。DMD是一种二值调制器件,集成了数万个可独立控制的微反射镜。微反射镜根据控制信号沿其对角线方向进行±12°翻转,分别对应1和0调制。Hadamard矩阵是由±1元素组成的二值矩阵,由于DMD无法直接投影Hadamard矩阵,所以通常需要将Hadamard矩阵中−1元素换成0元素,得到的掩膜图案称为正模式;同时将正模式中的元素互换,得到的掩膜图案称为负模式[30]。用正模式减去负模式就可以模拟Hadamard矩阵,同时还可以抑制环境噪声[31]。由于DMD调制产生两条光路并且二者对应的掩膜图案正好相反,因此利用其互补特性设计了双光路单像素成像系统。
图 1 紧凑双光路单像素成像系统示意图。(a)系统内部结构图;(b)实物图
Figure 1. Schematic diagram of the compact dual optical path single-pixel imaging system. (a) Internal structure diagram of the camera; (b) Physical diagram
当系统工作时,带有场景信息的光束经过镜头成像在DMD工作面上;DMD调制光束并将其分为两路,分别沿±12°两个方向出射;光束依次经过平面反射镜、光束收集透镜,最后由单像素探测器接收。
为保证系统的可拓展性和通用性,按照Nikon标准镜头法兰距, 相机法兰距设定为46.5 mm。同时,系统可根据应用场景需求随时替换不同波长的探测器。系统整体尺寸为78 mm×49.91 mm×90 mm,紧凑化的结构设计提高了空间利用率;同时,双光路设计提高了成像系统的能量利用率。
-
搭建好的成像系统实物图如图1(b)所示,前端装配镜头(Nikon AF 85 mm);左右两侧通过透镜套筒分别连接单像素探测器1(Thorlabs PDA 100A-EC,工作波长340~1 100 nm)和探测器2(Thorlabs PDA 50B-2,工作波长800~1 800 nm);套筒内放置光束收集透镜;后端安装DMD及其控制装置(Vialux,V-7000,刷新速率22 kHz);内部沿DMD调制的±12°出射方向分别安装平面反射镜;数据采集装置采用高速采集卡NI USB-6361 (采样率500 kSPS)。
-
传统单光路差分成像[10,31]通过DMD连续投影两组相反的Hadamard图案实现差分测量,可有效提高重建的信噪比,但是会增加一倍的采样次数,限制了实时帧率。
双光路差分成像只用一次投影即可获取差分信号,无需增加额外的采样次数。当系统应用双光路差分成像模式时,需更换为两个相同型号的探测器,并对探测器进行标定,以保证二者增益一致。探测器1收集沿DMD+12°方向出射光束的光强,探测器2收集沿DMD-12°方向出射光束的光强,则图像可重建为:
$$ I = \frac{1}{N}\sum\limits_{i = 0}^N {{P_i} \cdot \left( {{S_i} - \overline {{S_i}} } \right)} $$ (2) 式中:
$ I $ 表示重建的图像;$ {P_i} $ 表示当前投影的Hadamard图案;$ {S_i} $ 表示探测器1收集的光强值,$ \overline {{S_i}} $ 表示探测器2收集的光强值。文中采用以下公式计算信噪比[32-33] (Signal-Noise Ratio, SNR):
$$ SNR = \frac{{\left\langle {{I_f}} \right\rangle - \left\langle {{I_b}} \right\rangle }}{{\dfrac{{{\sigma _f} + {\sigma _b}}}{2}}} $$ (3) 式中:
$ \left\langle {{I_f}} \right\rangle $ 表示图像中的特征区域的测量信号平均值(此处用图2中红色虚线内的白色区域计算);$ \left\langle {{I_b}} \right\rangle $ 是图像中背景部分的测量信号平均值(此处用图2中蓝色虚线内的黑色区域计算);$ {\sigma _f} $ 和$ {\sigma _b} $ 分别是图像特征与背景的测量信号强度标准差[34]。图 2 不同模式下重建图像的信噪比(64×64 pixel上采样到128×128 pixel)。(a)非差分成像;(b)双光路差分成像;(c)双光路平均降噪
Figure 2. The signal-to-noise ratio of the reconstructed image in different modes (64×64 pixel up-sampled to 128×128 pixel). (a) Non-differential imaging; (b) Dual optical path differential imaging; (c) Dual optical path average noise reduction
实验结果如图2(a)和图2(b)所示,非差分单像素成像的SNR为13.45,光源波动噪声较为明显;双光路差分成像SNR为72.33,抑制了大部分噪声后重建质量显著提高;对于64×64 pixel的场景,单光路差分成像模式下的帧率为2.6 fps,非差分成像和双光路差分成像模式下的帧率为5.2 fps。可见,双光路差分成像利用差分测量抑制噪声从而提高图像信噪比,同时没有增加额外的采样次数,能够保持与非差分成像相同的帧率。
-
系统可通过双光路平均降噪模式来提高图像的重建质量。单像素重建的图像可表示为无噪声图像和加性噪声的叠加:
$$ I\left( {x,y} \right) = o\left( {x,y} \right) + \beta \left( {x,y} \right) $$ (4) 式中:
$ I\left( {x,y} \right) $ 表示重建的图像;$ o\left( {x,y} \right) $ 表示无噪声图像;$ \beta \left( {x,y} \right) $ 表示加性噪声。则对M幅重建的图像进行平均操作后满足:$$ \overline I \left( {x,y} \right) = \frac{1}{M}\sum\limits_{i = 1}^M {{I_i}\left( {x,y} \right)} $$ (5) $$ E\left\{ {\overline I \left( {x,y} \right)} \right\} = o\left( {x,y} \right) $$ (6) $$ \sigma _{\overline I \left( {x,y} \right)}^2 = \frac{1}{M}\sigma _{\beta \left( {x,y} \right)}^2 $$ (7) 式中:
$ \overline I \left( {x,y} \right) $ 是平均操作后的图像;$ E\left\{ {\overline I \left( {x,y} \right)} \right\} $ 表示平均图像$ \overline I \left( {x,y} \right) $ 的期望值;$ \sigma _{\overline I \left( {x,y} \right)}^2 $ 和$ \sigma _{\beta \left( {x,y} \right)}^2 $ 分别是$ \overline I \left( {x,y} \right) $ 和$ \beta \left( {x,y} \right) $ 在点$ \left( {x,y} \right) $ 的方差。公式(6)、(7)表明噪声会随着M增加而减小[35]。双光路平均降噪模式利用上述特点,将探测器1和探测器2所在光路独立重建的图像取平均值来降低噪声。
该实验中用双光路独立差分成像的结果取平均值,如图2(c)所示。经计算,SNR为111.48,相比于图2(b)差分重建的图像,信噪比得到明显改善。在不追求高帧率的情况下,例如当场景静止时,采用双光路平均降噪模式可以进一步提高成像的重建质量以便获取更多目标物体的细节。
-
系统可实现对可见光和特殊波段同时成像。在本实验中光源采用白炽灯(宽光谱光源),探测器1使用Thorlabs PDA 100A-EC,所在光路对可见光成像,由于其工作波长与近红外有部分重叠,因此在该探测器套筒端加近红外吸收型中性密度滤光片以屏蔽近红外光。探测器2使用Thorlabs PDA 50B-2,所在光路对近红外成像。并事先在字母U处涂一层红外油墨(透过波长900~1 100 nm,透过率约85%)达到屏蔽可见光和透过近红外光的效果。
实验结果如图3(a)所示,可以观察到近红外波段重建的图像中油墨涂层下的字母U,而可见光波段重建的图像则不能,验证了系统在宽谱波段光源照明下对可见光和近红外同时成像的性能。除此以外,系统仍适用于其他特殊波段(紫外、太赫兹等波段),只需更换为相应工作波长的探测器,突破了现有阵列探测器高成本的限制。
-
系统可通过双光路交替采样来提高实时帧率。双光路交替采样是指单像素探测器1所在光路从第1个掩膜图案投影开始测量,到第n个掩膜图案投影结束后完成一次单像素成像,显示一帧图像;而探测器2所在光路从第n/2+1个掩膜图案投影开始测量,到下一组第n/2个掩膜图案投影结束后完成一次单像素成像,显示一帧图像,按此规律交替采样,如图3(b)所示。以32×32 pixel图像为例,从第一组Hadamard图案投影完开始,每完成512次测量,就进行重建图像并刷新一帧。
实验表明,对于32×32 pixel的场景单光路差分成像模式下的帧率为10 fps,而在该模式下差分成像的帧率为20 fps,帧率提升了一倍。在追求高帧率而目标细节可忽略的情况下,采用双光路交替采样模式进行非差分成像可进一步提升实时帧率至40 fps,在无人驾驶、目标跟踪等场景能够发挥出重要作用。
Compact dual optical path single-pixel imaging system (Invited)
-
摘要: 不同于使用阵列探测器的常规数码相机,单像素相机使用不具空间分辨能力的单像素探测器对目标进行成像。由于其工作波长覆盖广、灵敏度高,单像素相机在特殊波段和弱光照明等特殊场景中较普通相机更有优势,在遥感探测、显微成像、军事侦察等领域得到广泛应用。提出一种紧凑型单像素成像系统,该系统利用数字微镜阵列的工作特性形成了对称的折叠双光路以快速完成差分测量。紧凑的结构降低了相机系统的体积,使系统可采用标准尼康镜头作为成像透镜。系统具有双光路差分成像、双光路平均降噪、宽谱波段成像、双光路交替采样等多种模式,并且可根据场景对重建图像信噪比、实时帧率、成像波段的不同需求切换应用模式。基于系统样机的实验结果表明其能实现预期的功能、达到相应的性能。紧凑型单像素成像系统的提出是一次较成功的单像素成像工程化尝试,为单像素成像技术后续的实际应用奠定了较好的技术和工程基础。Abstract: Unlike conventional digital cameras that used array detectors, single-pixel cameras used single-pixel detectors without spatial resolution to image targets. Due to its wide operating wavelength coverage and high sensitivity, single-pixel cameras are more advantageous than ordinary cameras in special scenarios such as special wavelengths and low light illumination, and are widely used in remote sensing detection, microscopic imaging, military reconnaissance, and other fields. A compact single-pixel imaging system was proposed, which used the working characteristics of the digital micro-mirror array to form a symmetrical folded double optical path to quickly complete differential measurement. The compact structure reduced the size of the camera system and allowed the system to use a standard Nikon lens as the imaging lens. The system had various modes such as dual-optical differential imaging, dual-optical averaging noise reduction, broad-spectrum band imaging, and dual-optical alternate sampling, and the application modes could be switched according to the different needs of the scene for reconstructed image signal-to-noise ratio, real-time frame rate, and imaging band. The experimental results based on the system prototype showed that it could realize the expected functions and achieve the corresponding performance. The proposed compact single-pixel imaging system was a successful engineering attempt of single-pixel imaging, which laid a good technical and engineering foundation for the subsequent practical application of single-pixel imaging technology.
-
图 2 不同模式下重建图像的信噪比(64×64 pixel上采样到128×128 pixel)。(a)非差分成像;(b)双光路差分成像;(c)双光路平均降噪
Figure 2. The signal-to-noise ratio of the reconstructed image in different modes (64×64 pixel up-sampled to 128×128 pixel). (a) Non-differential imaging; (b) Dual optical path differential imaging; (c) Dual optical path average noise reduction
-
[1] Pittman T B, Shih Y H, Strekalov D V, et al. Optical imaging by means of two-photon quantum entanglement [J]. Physical Review A, 1995, 52(5): R3429. doi: 10.1103/PhysRevA.52.R3429 [2] Bennink R S, Bentley S J, Boyd R W. "Two-Photon" coincidence imaging with a classical source [J]. Physical Review Letters, 2002, 89(11): 113601. doi: 10.1103/PhysRevLett.89.113601 [3] Shapiro J H. Computational ghost imaging [C]// Conference on Lasers & Electro-optics, 2008: 061802. [4] Candès E, Romberg J. Sparsity and incoherence in compressive sampling [J]. Inverse Problems, 2007, 23(3): 969-985. doi: 10.1088/0266-5611/23/3/008 [5] Donoho D L. Compressed sensing [J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306. doi: 10.1109/TIT.2006.871582 [6] Baraniuk R G. Compressive sensing [lecture notes] [J]. IEEE Signal Processing Magazine, 2007, 24(4): 118-120. doi: 10.1109/MSP.2007.4286571 [7] Candès E J. Compressive sampling[C]// Proceedings of the 2006 International Congress of Mathematicians, 2006: 1433-1452. [8] Llull P, Liao X, Yuan X, et al. Coded aperture compressive temporal imaging [J]. Optics Express, 2013, 21(9): 10526-10545. doi: 10.1364/OE.21.010526 [9] Duarte M F, Davenport M A, D Takhar, et al. Single-pixel imaging via compressive sampling [J]. IEEE Signal Processing Magazine, 2008, 25(2): 83-91. doi: 10.1109/MSP.2007.914730 [10] Ferri F, Magatti D, Lugiato L A, et al. Differential ghost imaging [J]. Physical Review Letters, 2010, 104(25): 253603. doi: 10.1103/PhysRevLett.104.253603 [11] Sun B, Welsh S, Edgar M P, et al. Normalized ghost imaging [J]. Optics Express, 2012, 20(15): 16892-16901. doi: 10.1364/OE.20.016892 [12] Sun M J, Edgar M P, Phillips D B, et al. Improving the signal-to-noise ratio of single-pixel imaging using digital microscanning [J]. Optics Express, 2016, 24(10): 10476. doi: 10.1364/OE.24.010476 [13] Edgar M P, Gibson G M, Bowman R W, et al. Simultaneous real-time visible and infrared video with single-pixel detectors [J]. Scientific Reports, 2015, 5(1): 745-755. [14] Gibson G M, Sun B Q, Edgar M P, et al. Real-time imaging of methane gas leaks using a single-pixel camera [J]. Optics Express, 2017, 244(10): 62-66. [15] Liu S, Yao X R, Liu X F, et al. Pile-up effect in an infrared single-pixel compressive LiDAR system [J]. Optics Express, 2019, 27(16): 22138-22146. doi: 10.1364/OE.27.022138 [16] Sun B Q, Jiang S, Ma Y Y, et al. Application development of single-pixel imaging in special wavebands and 3D imaging [J]. Infrared and Laser Engineering, 2020, 49(3): 0303016. (in Chinese) doi: 10.3788/IRLA202049.0303016 [17] Klein Y, Schori A, Dolbnya I P, et al. X-ray computational ghost imaging with single-pixel detector [J]. Optics Express, 2019, 27(3): 3284-3293. doi: 10.1364/OE.27.003284 [18] He Y H, Zhang A X, Li M F, et al. High-resolution sub-sampling incoherent x-ray imaging with a single-pixel detector [J]. APL Photonics, 2020, 5(5): 056102. doi: 10.1063/1.5140322 [19] Yu H, Lu R, Han S, et al. Fourier-transform ghost imaging with hard X rays [J]. Physical Review Letters, 2016, 117(11): 113901. doi: 10.1103/PhysRevLett.117.113901 [20] Stantchev R I, Yu X, Blu T, et al. Real-time terahertz imaging with a single-pixel detector [J]. Nature Communications, 2020, 11(1): 2535. doi: 10.1038/s41467-020-16370-x [21] Watts C M, Shrekenhamer D, Montoya J, et al. Terahertz compressive imaging with metamaterial spatial light modulators [J]. Nature Photonics, 2014, 8(8): 605-609. doi: 10.1038/nphoton.2014.139 [22] Stantchev R I, Sun B, Hornett S M, et al. Non-invasive, near-field terahertz imaging of hidden objects using a single pixel detector [J]. Science Advances, 2016, 2(6): e1600190. doi: 10.1126/sciadv.1600190 [23] Yan Y Q, Zhao C Q, Xu W D, et al. Research on terahertz active correlation imaging technology [J]. Chinese Journal of Lasers, 2018, 45(8): 280-287. (in Chinese) [24] Cheng T, Li Z W, Wang J L, et al. Single-pixel camera imaging system using compressed sensing theory [J]. Optics and Precision Engineering, 2012, 20(11): 2523-2530. (in Chinese) doi: 10.3788/OPE.20122011.2523 [25] Peng J Y, Jing H Q, Shi J H, et al. High-speed single-pixel camera data acquisition system [J]. Optics and Precision Engineering, 2014, 22(4): 837-843. (in Chinese) doi: 10.3788/OPE.20142204.0837 [26] Sun M J, Zhang J M. Single-pixel imaging and its application in three-dimensional reconstruction: A brief review [J]. Sensors, 2019, 19(3): 732. doi: 10.3390/s19030732 [27] Li M F, Kan B X, Huo J, et al. Single-pixel imaging experiment through 34 km horizontal atmosphere [J]. Infrared and Laser Engineering, 2019, 48(9): 0925002. (in Chinese) doi: 10.3788/IRLA201948.0925002 [28] Lochocki B, Gambín A, Manzanera S, et al. Single pixel camera ophthalmoscope [J]. Optica, 2016, 3(10): 1056-1059. doi: 10.1364/OPTICA.3.001056 [29] Sun B Q, Wang Y P. Time-domain ghost imaging and its application [J]. Chinese Journal of Lasers, 2021, 48(12): 1212001. (in Chinese) doi: 10.3788/CJL202148.1212001 [30] Zhang Z B, Wang X Y, Zheng G A, et al. Hadamard single-pixel imaging versus Fourier single-pixel imaging [J]. Optics Express, 2017, 25(16): 19619-19639. doi: 10.1364/OE.25.019619 [31] Phillips D B, Sun M J, Taylor J M, et al. Adaptive foveated single-pixel imaging with dynamic supersampling [J]. Science Advances, 2017, 3(4): 1601782. doi: 10.1126/sciadv.1601782 [32] Redding B, Choma M A, Cao H. Speckle-free laser imaging using random laser illumination [J]. Nature Photonics, 2012, 6(6): 355-359. doi: 10.1038/nphoton.2012.90 [33] Schechner Y, Nayar S K, Belhumeur P N. Multiplexing for optimal lighting [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2007, 29(8): 1339-1354. [34] Yan S M, Sun M J, Chen W, et al. Illumination calibration for computational ghost imaging [J]. Photonics, 2021, 8(2): 59. doi: 10.3390/photonics8020059 [35] Kohler R J, Howell H K. Photographic image enhancement by superposition of multiple images [J]. Photogr Sci Eng, 1963, 7(4): 241-245. [36] Sun M J, Edgar M P, Gibson G M, et al. Single-pixel three-dimensional imaging with time-based depth resolution [J]. Nature Communications, 2016, 7: 12010. doi: 10.1038/ncomms12010 [37] Sun M J, Meng L T, Edgar M P, et al. A russian dolls ordering of the hadamard basis for compressive single-pixel imaging [J]. Scientific Reports, 2017, 7(1): 3464. doi: 10.1038/s41598-017-03725-6