曹军峰, 丁庆海, 邹德鹏, 秦恒加, 罗海波. 基于深度学习的红外成像退化模型辨识及超分辨率成像方法[J]. 红外与激光工程, 2024, 53(5): 20240049. DOI: 10.3788/IRLA20240049
引用本文: 曹军峰, 丁庆海, 邹德鹏, 秦恒加, 罗海波. 基于深度学习的红外成像退化模型辨识及超分辨率成像方法[J]. 红外与激光工程, 2024, 53(5): 20240049. DOI: 10.3788/IRLA20240049
Cao Junfeng, Ding Qinghai, Zou Depeng, Qin Hengjia, Luo Haibo. Deep learning-based infrared imaging degradation model identification and super-resolution reconstruction[J]. Infrared and Laser Engineering, 2024, 53(5): 20240049. DOI: 10.3788/IRLA20240049
Citation: Cao Junfeng, Ding Qinghai, Zou Depeng, Qin Hengjia, Luo Haibo. Deep learning-based infrared imaging degradation model identification and super-resolution reconstruction[J]. Infrared and Laser Engineering, 2024, 53(5): 20240049. DOI: 10.3788/IRLA20240049

基于深度学习的红外成像退化模型辨识及超分辨率成像方法

Deep learning-based infrared imaging degradation model identification and super-resolution reconstruction

  • 摘要: 红外成像系统由于制造工艺和成本制约,分辨率仍然较低。图像超分辨率重建技术是提高图像分辨率的有效方法,获得了广泛研究,并在仿真图像上获得了很好的效果,但应用于实际图像时效果不甚理想,主要原因是实际成像退化更加复杂,包括红外光学系统像差和装配误差引起的空间非一致模糊,以及受工作温度影响导致的模糊核变化。针对上述问题,提出一种基于深度学习的红外成像退化模型辨识方法和基于退化模型约束的超分辨率重建方法,通过在不同工作温度下采集标定靶标图像,标定不同工作温度、不同空间位置的模糊核;采用卷积神经网络建立成像退化模型,并利用定标数据进行模型参数求解,为超分辨率重建提供更多先验信息;设计迭代超分辨率重建网络,交替进行退化参数估计和超分辨率重建,经过多次迭代逐步提高重建效果。实验结果表明,采用卷积神经网络求解的成像退化模型可准确描述模糊核变化规律,基于退化模型约束和退化参数在线学习的超分辨率重建方法可显著提高红外超分辨率成像的效果,具有较高的工程应用价值。

     

    Abstract:
      Objective  The limited resolution of infrared devices, constrained by cost and manufacturing technology, remains a challenge. While deep learning-based single image super-resolution (SISR) has shown promise in enhancing image resolution, its application in real-world infrared images is hindered by the complexity of actual degradation, including spatial non-uniform blur caused by optical aberration and assembly error, as well as variations in the blur kernel due to environmental temperature changes. A deep learning-based approach for infrared imaging degradation model identification and super-resolution reconstruction is proposed to tackle these challenges. This method entails solving the degradation model using a convolutional neural network to describe the evolution of blur kernels, along with a super-resolution reconstruction method that adheres to the constraints of the degradation model and incorporates online learning of degradation parameters.
      Methods  Images of calibration targets are captured using an infrared camera placed in a high and low temperature chamber, along with a portable target simulator placed outside it (Fig.1-2). These images are utilized to calibrate the blur kernels. A convolutional neural network (CNN) is employed to construct a model that characterizes the relationship between blur kernel, pixel coordinate, and operating temperature (Fig.3). The model is trained using the calibrated blur kernels. Additionally, a super-resolution network is developed and trained (Fig.4). The operating temperature is initially estimated using the low-resolution image. Next, the initial blur kernels are estimated by inputting the operating temperature into the kernel model. Subsequently, super-resolution reconstruction is conducted based on the estimated blur kernels, and the reconstructed image is utilized to refine the operating temperature and blur kernel estimation. Iterative processes improve the accuracy of blur kernel estimation, leading to enhanced reconstruction outcomes.
      Results and Discussions  The blur kernels of the infrared imaging system exhibit significant variation in response to temperature changes and spatial locations (Fig.6). The trained blur kernel model effectively predicts blur kernels using temperature and pixel coordinate inputs (Fig.7). The average PSNR between predicted and actual blur kernels across different operational temperatures is consistently high, with a minimum of 32.2 dB and an average of 37.1 dB, indicating precise predictions (Fig.8). The calibration and modeling of blur kernels provide valuable prior information for super-resolution reconstruction, resulting in enhanced reconstruction outcomes. Consequently, the proposed algorithm produces visually appealing results with improved detail (Fig.10-11) and enhances objective quality evaluation metrics such as the natural image quality evaluator (NIQE), perception-based image quality evaluator (PIQE), and blind/referenceless image spatial quality evaluator (BRISQUE) (Tab.1).
      Conclusions  A novel approach is proposed for infrared super-resolution imaging, including degradation model identification and iterative super-resolution reconstruction. The degradation model is based on a convolutional neural network and is solved using offline calibration data. It can predict blur kernels across various temperatures and spatial positions, reducing the need for extensive calibration work. Online degradation parameter correction is achieved through an iterative optimization network alternating between estimating the blur kernel and reconstructing the super-resolution image. By leveraging the degradation model, the complex high-dimensional blur kernel estimation problem is simplified into a low-dimensional operating temperature estimation problem, streamlining the solution process. Through iterations, the accuracy of blur kernel estimation improves, leading to superior super-resolution reconstruction outcomes. Experimental results demonstrate that calibrating and modeling blur kernels enhance prior information for super-resolution reconstruction, yielding superior results. Additionally, the proposed method adapts to a wider temperature range, reducing the stringency of athermalization design requirements for infrared optical systems.

     

/

返回文章
返回