-
基于稀疏约束关联成像光谱相机(Ghost Imaging via Sparsity Constraints,GISC)的系统光路图如图1所示,目标物体置于焦距为
${f_1}$ 的透镜焦平面上,经焦距为${f_2}$ 的前置成像透镜成像到视场光阑所在的前置成像面上,在前置成像面上获得一幅宽波段图像。调制模块中的空间相位调制器将前置成像面上宽波段图像上每个点发出的光场进行随机相位调制,经调制后形成一幅散斑图样。中继成像透镜将调制后的散斑图放大成像于面阵光电探测器上。通过对散斑场的单次曝光探测,结合事先标定的测量矩阵和压缩感知等重构算法,利用计算机重构出目标的三维光谱图像[15]。在实际探测时,探测面获取的光强分布为:
$${I_c}({r_t}) = \iint {{I_b}({r_i},{\lambda _l}){h_I}({r_t};{r_i},{\lambda _l}){\rm d}{r_i}{\rm d}{\lambda _l}}$$ (1) 式中:
${I_c}({r_t})$ 表示探测面上的光强分布;${I_b}({r_i},{\lambda _l})$ 表示多光谱物体的光强分布;${h_I}({r_t};{r_i},{\lambda _l})$ 表示物面上${r_i}$ 处位置波长为${\lambda _l}$ 的单色点光源发出的光在探测面上产生的光强分布。对前置成像面上视场范围进行如下的像素网格划分:1,1 1,2 $ \cdots $ 1,Q 2,1 2,2 $ \cdots $ 2,Q $ \vdots $ $ \vdots $ $ \ddots $ $ \vdots $ P,1 P,2 $ \cdots $ P,Q 对探测面进行像素网格划分(每个网格代表探测器上的一个像元):
1,1 1,2 $ \cdots $ 1,N 2,1 2,2 $ \cdots $ 2,N $ \vdots $ $ \vdots $ $ \ddots $ $ \vdots $ M,1 M,2 $ \cdots $ M,N 多光谱关联成像探测过程如下:
$${y_{mn}} = \sum\limits_{s = 1}^S {\sum\limits_{p = 1}^P {\sum\limits_{q = 1}^Q {A_{mn,pq}^{{\lambda _s}}X_{pq}^{{\lambda _s}}} } } + {\varsigma _{mn}}$$ (2) 式中:
${X}_{pq}^{{\lambda }_{s}}\left(p=1,\cdots ,P;\;q=1,\cdots ,Q\right)$ 表示前置成像面上第p行q列像素在s个谱段中$s\left( {s = 1, \cdots ,S} \right)$ 重构的多光谱图像信息;${A}_{mn,pq}^{{\lambda }_{s}}\left(m=1,\cdots ,M; \;n=1,\cdots ,N\right)$ 表示成像系统标定点光源在物面上第p行q列像素上移动时,探测面CCD第m行n列像元得到的光强度值;$y_{mn}$ 表示成像时探测面上第m行n列探测信号的强度分布;$\varsigma $ 表示探测信号噪声。设探测面上的探测信号强度分布为:
$$ Y = {[{y_{11}}, \cdots ,{y_{M1}}, \cdots ,{y_{12}}, \cdots {y_{M2}}, \cdots \cdots ,{y_{1N}}, \cdots ,{y_{MN}}]^{\rm{T}}} $$ (3) 根据GISC相机成像模型,目标物体的多光谱图像重构通过求解如下优化问题实现:
$$\mathop {\min }\limits_x \left\| {\left. {Y - AX} \right\|} \right._2^2 + {\mu _1}{\left\| {\left. {{\nabla _{i,j}}X} \right\|} \right._1} + {\mu _2}{\left\| {\left. X \right\|} \right._ * },s.t.x \geqslant 0$$ (4) 其中,A为前置成像面上各个像素发出的单色光经过空间相位调制器生成的散斑矩阵:
$$A = \left[ {\begin{array}{*{20}{c}} {a_{11,11}^{{\lambda _1}}}& \cdots &{a_{11,p1}^{{\lambda _1}}}& \cdots &{a_{11,1q}^{{\lambda _1}}}& \cdots &{a_{11,pq}^{{\lambda _1}}}& \cdots &{a_{11,11}^{{\lambda _s}}}& \cdots &{a_{11,p1}^{{\lambda _s}}}& \cdots &{a_{11,1q}^{{\lambda _s}}}& \cdots &{a_{11,pq}^{{\lambda _s}}} \\ \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots \\ {a_{m1,11}^{{\lambda _1}}}& \cdots &{a_{m1,p1}^{{\lambda _1}}}& \cdots &{a_{m1,1q}^{{\lambda _1}}}& \cdots &{a_{m1,pq}^{{\lambda _1}}}& \cdots &{a_{m1,11}^{{\lambda _s}}}& \cdots &{a_{m1,p1}^{{\lambda _s}}}& \cdots &{a_{m1,1q}^{{\lambda _s}}}& \cdots &{a_{m1,pq}^{{\lambda _s}}} \\ {a_{12,11}^{{\lambda _1}}}& \cdots &{a_{12,p1}^{{\lambda _1}}}& \cdots &{a_{12,1q}^{{\lambda _1}}}& \cdots &{a_{12,pq}^{{\lambda _1}}}& \cdots &{a_{12,11}^{{\lambda _s}}}& \cdots &{a_{12,p1}^{{\lambda _s}}}& \cdots &{a_{12,1q}^{{\lambda _s}}}& \cdots &{a_{12,pq}^{{\lambda _s}}} \\ \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots \\ {a_{m2,11}^{{\lambda _1}}}& \cdots &{a_{m2,p1}^{{\lambda _1}}}& \cdots &{a_{m2,1q}^{{\lambda _1}}}& \cdots &{a_{m2,pq}^{{\lambda _1}}}& \cdots &{a_{m2,11}^{{\lambda _s}}}& \cdots &{a_{m2,p1}^{{\lambda _s}}}& \cdots &{a_{m2,1q}^{{\lambda _s}}}& \cdots &{a_{m2,pq}^{{\lambda _s}}} \\ \vdots & \vdots & \vdots &{}& \vdots & \vdots & \vdots &{}& \vdots & \vdots & \vdots &{}& \vdots & \vdots & \vdots \\ {a_{1n,11}^{{\lambda _1}}}& \cdots &{a_{1n,p1}^{{\lambda _1}}}& \cdots &{a_{1n,1q}^{{\lambda _1}}}& \cdots &{a_{1n,pq}^{{\lambda _1}}}& \cdots &{a_{1n,11}^{{\lambda _s}}}& \cdots &{a_{1n,p1}^{{\lambda _s}}}& \cdots &{a_{1n,1q}^{{\lambda _s}}}& \cdots &{a_{1n,pq}^{{\lambda _s}}} \\ \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots &{}& \vdots & \ddots & \vdots \\ {a_{mn,11}^{{\lambda _1}}}& \cdots &{a_{mn,p1}^{{\lambda _1}}}& \cdots &{a_{mn,1q}^{{\lambda _1}}}& \cdots &{a_{mn,pq}^{{\lambda _1}}}& \cdots &{a_{mn,11}^{{\lambda _s}}}& \cdots &{a_{mn,p1}^{{\lambda _s}}}& \cdots &{a_{mn,1q}^{{\lambda _s}}}& \cdots &{a_{mn,pq}^{{\lambda _s}}} \end{array}} \right]$$ (5) X为需要计算重构的多光谱图像信息按像素网格中先列后行顺序排列而成的列向量:
$$\begin{split} X = &[x_{11}^{{\lambda _1}}, \cdots ,x_{P1}^{{\lambda _1}}, \cdots \cdots ,x_{1Q}^{{\lambda _1}}, \cdots x_{PQ}^{{\lambda _1}}, \cdots \cdots ,x_{11}^{{\lambda _s}}, \cdots ,\\ &x_{P1}^{{\lambda _s}}, \cdots \cdots ,x_{1Q}^{{\lambda _s}}, \cdots x_{PQ}^{{\lambda _s}}]^{\rm T} \end{split}$$ (6) ${\left\| {\left. {{\nabla _{i,j}}X} \right\|} \right._1}$ 为梯度范数,相当于提取图像的分段边缘,使变换后的图像更加稀疏;${\left\| {\left. X \right\|} \right._ * }$ 表示矩阵核范数,表明多光谱图像矩阵的低秩性;${\mu _1},{\mu _2} \geqslant 0$ 为各约束项的权重系数。文中采用TV-RANK压缩感知算法进行图像重构[16]。上述GISC成像过程示意图如图2所示。
首先,通过单次曝光获取CCD探测信号,代入公式(4)得到重构的多光谱灰度图像,然后对不同光谱的重构图像进行伪彩色处理,最后通过图像融合算法将不同波长的多光谱图像合成一幅彩色图像。
-
基于运动目标的单次曝光多光谱关联成像外场实验装置如图3所示。原理样机成像系统由一个GISC相机和一个监视相机组成,固定在地面三脚架上。实验参数为:GISC相机光谱范围450~700 nm,15个光谱通道,光谱分辨率<20 nm,像元分辨率≥0.5 mrad。GISC相机和监视相机的曝光时间设置为40 ms。待测目标物体由长123 mm、高50 mm的彩色玩具车和长28 mm、高34 mm的黄色安全警示锥形桶组成,其中彩色玩具车作为运动目标物体,黄色安全警示锥形桶作为静止参考物体。利用步进精度为0.001 mm/s的SC100型步进电机以30 Hz的运行速度带动目标物体运动成像。采用日光作为照明光源,待测目标物体在距原理样机3.5 m处,成像视场宽范围60 mm×610 mm。
图 3 外场实验装置图。①目标物体; ② SC100型步进电机; ③ SC步进电机控制器; ④ GISC相机; ⑤ RGB相机;⑥ KINGJOY VT-2500三脚架
Figure 3. Diagram of the experimental setup in the field. ① Target object; ② SC100 stepping motor; ③ SC stepping motor controller; ④ GISC camera; ⑤ RGB camera; ⑥ KINGJOY VT-2500 tripod
单帧多光谱关联成像实验结果如图4所示。当CCD记录的电子数均值为1300 e−时,GISC相机通过单次曝光获取的CCD探测信号;利用事先标定好的测量矩阵和公式(4)得到重构的多光谱灰度图像如图4(a)所示,15个光谱通道波长分别为461、474、487、501、515、530、545、561、578、590、614、633、653、674、698 nm。对图4(a)的不同波长重构图像进行伪彩色处理,得到多光谱图像重构结果如图4(b)所示,最后合成的彩色融合图为图4(c)。与图3中①图待测目标物体相比较,得到了较高质量的目标物体形状和光谱信息重构结果。
为进一步考察GISC相机的探测信号值对多光谱图像重构质量的影响,将CCD记录的电子数均值参数设置为以100e−的间隔从200e−增加至1300e−,CCD探测信号实际结果如图5(a)所示,对应不同电子数均值的多光谱图像彩色融合图重构结果如图5(b)所示。对比图5(a)和图5(b)可以发现:当电子数均值为200e−时,重构图像质量较低,存在噪声较大;随着电子数均值逐渐增大,重构图像质量也相应不断提高,噪声随之减小;当电子数均值为1300e−时,可以获得较高质量的重构图像。
图 5 不同电子数均值时,单帧探测信号和重构图像实验结果
Figure 5. Experimental results of single-frame detection signal and reconstructed image with different electron number averages
为进一步说明探测信号值对多光谱图像重构质量的影响,采用相对均方根误差(rRMSE)来评价图像重构质量,定义如下:
$${{rRMSE}} = \sqrt {\frac{{\rm{1}}}{{{{M}} \times {{N}}}}\sum\nolimits_{{{m}},n = 1}^{M,N} {\left[ {x_{ref}^ * (m,n) - {x^ * }(m,n)} \right]} } $$ (7) 式中:M和N为CCD探测面上的行数和列数;
$x_{ref}^ * $ 为参考图像(实验中为电子数均值为1400 e−的单帧多光谱图像);${x^ * }$ 为重构图像。rRMSE值越小,重构图像质量越好。图6为
$rRMSE$ 随电子数均值变化关系曲线图。由图6可以看出,当电子数均值为200 e−时,重构图像的$rRMSE$ 值为0.1989,当电子数均值增加至1300 e−时,$rRMSE$ 值为0.0119,随着电子数均值的增加,$rRMSE$ 值单调下降,重构图像质量明显提高。图 6
$rRMSE$ 和电子数均值关系曲线图Figure 6. Curve of the relationship between rRMSE and the mean number of electrons
下面分析运动目标的多光谱关联成像规律。利用步进电机以30 Hz的速度带动物体以0.6 mm/s的速度做连续匀速直线运动,多光谱关联成像重构结果如图7(b)所示。在初始位置,彩色玩具车与黄色安全警示锥形桶距离450 mm。当运动时间为100 s时,彩色玩具车向黄色安全警示锥形桶移动了60 mm;当运动时间增加到200 s时,彩色玩具车移动了120 mm,距离锥形桶330 mm;当运动时间为500 s时,彩色玩具车移动的总位移为300 mm,距离锥形桶150 mm。对比图7(a)与图7(b)可以发现:GISC相机拍摄的运动目标多光谱图像重构结果与监视相机在不同时刻所拍摄目标物体的RGB图像运动规律完全相同;但在相同位置处,RGB相机只能拍摄3个光谱通道,而由图4可知GISC相机能够获得15个光谱通道,可以从待测运动目标重构图像中提取更加丰富的光谱特征。
图 7 运动目标在不同时刻多光谱关联成像结果
Figure 7. Results of multi-spectral correlation imaging of moving targets at different moments
为了进一步说明实验的可行性,采用USB4000-VIS-NIR光谱仪对目标物体中450~700 nm不同颜色的光谱分布进行了测试,并与GISC相机拍摄的重构图像实测光谱曲线进行对比分析,如图8所示。采用商用光谱仪测试目标物体中蓝色、绿色、黄色、橙色和红色部分光谱曲线的波峰分别位于481、513、587、608、638 nm,实测重构图像中对应部分光谱曲线的波峰位于490、519、593、614、640 nm。实验结果与商用光谱仪测试结果误差小于10 nm,主要是由于实验标定系统产生的误差。
Research on multispectral correlation imaging of moving target based on phase modulation
-
摘要: 为克服扫描方式多光谱成像无法捕获动态场景下的多光谱数据,提出了一种基于相位调制实现运动目标单次曝光多光谱成像方法。该方法将关联成像技术、压缩感知技术与光谱成像相结合,在成像光路中引入空间随机相位调制器,对运动目标物体三维图谱信息数据进行调制和压缩,然后利用探测器获取二维混叠信号,实现单次曝光获取运动目标的三维图谱信息重构,具有光能利用率高、成像时间短、系统结构简单等优点。实验结果表明:单帧CCD探测信号的电子数均值从200 e−按100 e−的间隔增加到1300 e−时,随着电子数均值增加,重构图像相对均方根误差rRMSE值对应减小,重构图像质量提高;当步进电机以30 Hz速度带动目标物体连续运动时,可获得较好质量运动物体的多光谱重构图像;采用光谱仪对目标物体中不同谱段的光谱分布曲线进行测试,所得结果与重构图像的光谱分布曲线相吻合,证明了该方法的有效性。研究结果对多光谱关联成像技术在无人机平台、动态监测等领域的应用提供了有益借鉴。Abstract: In order to overcome the inability of scanning multi-spectral imaging to capture multi-spectral data in dynamic scenes, a single-exposure multispectral imaging method for moving targets was proposed based on phase modulation. This method combined associated imaging technology, compressed sensing technology and spectral imaging, introduced a spatial random phase modulator into the imaging light path, modulated and compressed the three-dimensional map information data of the moving target object, then used the two-dimensional aliasing signal obtained by the detector to reconstruct the three-dimensional map information to achieve a single exposure and simultaneously obtained the three-dimensional map information of the moving target. It had the advantages of high utilization rate of light energy, short imaging time, and simple system structure. The experimental results show that when the average electron number of a single frame of CCD detection signal increases from 200 e− at intervals of 100 e− to 1300 e−, as the average rRMSE value of the electron number increases, the relative root mean square error of the reconstructed image decreases correspondingly, and the reconstruction improved image quality; when the stepper motor drives the target object to continuously move at a speed of 30 Hz, a multi-spectral reconstructed image of the moving object with better quality can be obtained; a spectrometer is used to test the spectral distribution curves of different spectrum bands in the target object, and the results obtained are basically consistent with the spectral distribution curves of the reconstructed image, which proves the effectiveness of the method. The research results provide a useful reference for the application of multi-spectral correlation imaging technology in UAV platforms, dynamic monitoring and other fields.
-
Key words:
- multispectral /
- correlated imaging /
- compressed sensing /
- moving target /
- detection signal
-
1,1 1,2 $ \cdots $ 1,Q 2,1 2,2 $ \cdots $ 2,Q $ \vdots $ $ \vdots $ $ \ddots $ $ \vdots $ P,1 P,2 $ \cdots $ P,Q 1,1 1,2 $ \cdots $ 1,N 2,1 2,2 $ \cdots $ 2,N $ \vdots $ $ \vdots $ $ \ddots $ $ \vdots $ M,1 M,2 $ \cdots $ M,N -
[1] 曹汛, 周凯来, 戴琼海. 计算光谱成像的前沿进展[J]. 中国计算机学会通讯, 2020, 16(9): 11-14. Cao Xun, Zhou Kailai, Dai Qionghai. Recent advances about computational spectral imaging[J]. Communications of The CCF, 2020, 16(9): 11-14. (in Chinese) [2] 周成. 基于非相干光关联成像方案研究[D]. 长春理工大学, 2017. Zou Cheng. Research on incoherent light correlation imaging scheme[D]. Changchun: Changchun University of Science and Technology, 2017. (in Chinese) [3] Tan Shiyu, Liu Zhentao, Li Enrong, et al. Hyperspectral compressed sensing based on prior images [J]. Acta Optica Sinica, 2015, 35(8): 120-128. (in Chinese) [4] Arce G R, Brady D J, Carin L, et al. Compressive coded aperture spectral imaging: An introduction [J]. IEEE Signal Processing Magazine, 2013, 31(1): 105-115. [5] Correa C V, Arguello H, Arce G R. Snapshot colored compressive spectral imager [J]. Journal of the Optical Society of America A, 2015, 32(10): 1754-1763. doi: 10.1364/JOSAA.32.001754 [6] Parada-Mayorga A, Arce G R. Colored coded aperture design in compressive spectral imaging via minimum coherence [J]. IEEE Transactions on Computational Imaging, 2017, 3(2): 202-216. doi: 10.1109/TCI.2017.2692649 [7] Fu C, Don M L, Arce G R. Compressive spectral imaging via polar coded aperture [J]. IEEE Transactions on Computational Imaging, 2016, 3(3): 408-420. [8] Shapiro J H. Computational ghost imaging [J]. Physical Review A, 2008, 78(6): 061802. doi: 10.1103/PhysRevA.78.061802 [9] Wu Jianrong, Shen Xia, Yu Hong, et al. Snapshot compresive imaging by phase modulation [J]. Acta Optica Sinica, 2014, 34(10): 121-128. (in Chinese) [10] Brady D J, Gehm M E. Compressive imaging spectrometers using coded apertures [C]// Proceedings of SPIE-The International Society for Optical Engineering, 2006, 6246: 62460A. [11] Wagadarikar A A, Pitsianis N P, Sun X, et al. Video rate spectral imaging using a coded aperture snapshot spectral imager [J]. Optics Express, 2009, 17(8): 6368-6388. doi: 10.1364/OE.17.006368 [12] Zhang Cong, Gong Wenlin, Han Shensheng. Ghost imaging for moving targets and its application in remote sensing [J]. Chinese Journal of Lasers, 2012, 39(12): 1214003. (in Chinese) [13] Zunwang B, Wenlin G, Shensheng H. Motion de-blurring by second-order intensity-correlated imaging [J]. Chinese Optics Letters, 2016, 14(7): 070301. [14] Wu J, Li E, Shen X, et al. Experimental results of the balloon-borne spectral camera based on ghost imaging via sparsity constraints [J]. IEEE Access, 2018, 6: 68740-68748. doi: 10.1109/ACCESS.2018.2879849 [15] Li Meixuan, Wang Xue, Wang Hong, et al. Compressed multi-spectral ghost imaging using pu sh-broom based on superposing detected signals [J]. Acta Optica Sinica, 2020, 49(7): 0711002. (in Chinese) [16] 谭诗语. 结构压缩感知在多光谱强度关联成像中的应用研究[D]. 北京: 中国科学院大学, 2015: 31-55. Tan Shiyu. The application research of structured compressed sensing in multispectral ghost imaging[D]. Beijing: University of Chinese Academy of Sciences, 2015: 31-55. (in Chinese)