Survey of research methods in infrared image dehazing
-
摘要: 红外图像去雾是指通过去除雾霾、烟雾等介质对红外图像的影响,恢复红外成像系统对比度和视觉质量的过程。红外图像凭借全天时、不受光照限制等优势,在军事、安防、医疗、能源勘探等领域广泛地应用。然而,由于大气介质对红外图像的干扰,这些应用往往受到限制,因此红外图像去雾成为一个重要的研究领域。近年来,随着计算机视觉、深度学习等技术的不断发展,红外图像去雾技术也取得了一系列重要进展,为红外图像应用的发展提供了强有力的支持。根据红外图像去雾过程中所依赖数据的不同,将现有的红外图像去雾方法划分为多信息融合和单帧图像处理两大类,其中多信息融合因为需要额外的信息来帮助图像恢复而使其应用受到限制;而目前基于单帧图像处理的主流方案包括图像增强和图像重建两个发展方向。对各种分类的代表算法进行了简要梳理,并分析了其原理、优势及不足。最后,对红外图像去雾的发展趋势做出了预测。该工作既可以帮助初学者快速了解该领域的研究现状和发展趋势,也可作为其他研究者的参考资料。Abstract:
Significance Infrared image dehazing refers to the process of restoring the contrast and visual quality of the infrared imaging system by removing the influence of haze, smoke and other media on the infrared image in the presence of atmospheric turbulence. Infrared images are widely used in military, security, medical, energy exploration and other fields by virtue of the advantages of all-day and no light limitation. Enhanced Image Visibility : Infrared images captured in hazy or foggy conditions often suffer from reduced visibility and degraded image quality. Dehazing techniques aim at improving the visibility of these images, allowing for better interpretation and analysis. Improved Object Detection and Recognition: Dehazing infrared images can enhance the performance of object detection and recognition algorithms. By removing the haze, important visual features of objects can be more clearly revealed, leading to more accurate and reliable results in various applications such as surveillance, target tracking, and autonomous vehicles. Enhanced Environmental Monitoring: Infrared imaging is widely used in environmental monitoring, including forest fire detection, air pollution monitoring, and thermal inspection of infrastructure. Dehazing techniques can help improve the accuracy and reliability of these monitoring systems by providing clearer and more detailed infrared images. Enhanced Human Perception: Dehazing infrared images can also benefit human observers by providing clearer and more understandable visual information. This is particularly important in applications where human operators rely on infrared images for decision-making, such as search and rescue operations, firefighting, and security surveillance. Advancements in Computer Vision Research: Dehazing infrared images presents a challenging problem in computer vision research. Developing effective dehazing algorithms for infrared images requires the exploration and development of novel techniques, such as image enhancement, deconvolution, and scene understanding. The research in this area can contribute to the advancement of overall computer vision research and benefit other related fields. Progress In recent years, with the continuous development of computer vision and deep learning technologies, significant progress has been made in infrared image dehazing techniques, providing support for the development of infrared image applications. According to the different types of data relied upon in the process of infrared image dehazing, existing methods can be divided into two categories: multi-information fusion and single-frame image processing. Image dehazing is a highly challenging task because the degradation level of an image is influenced by factors such as the concentration of suspended particles and the distance between the target and the detector. These pieces of information are difficult to directly obtain from the image, making image dehazing a very challenging task. Researchers have proposed multi-information fusion algorithms to assist in the restoration of infrared images by fusing additional information acquired through sensor fusion or multiple images. These methods mainly include polarization image dehazing (Fig.2) and fusion-weighted image dehazing methods. Single-frame image processing refers to the technique of digital or image processing applied to individual static images. In practical applications, single-frame image processing is often combined with machine learning, deep learning, and other technologies to achieve better results. This article mainly discusses image enhancement and image reconstruction in single-frame image processing. Image enhancement combines the MSR (Fig.5) with the CLAHE algorithm to achieve image enhancement of foggy images (Fig.3, Fig.4). Image reconstruction applied to the field of infrared image dehazing can estimate unknown information based on the characteristics of known information, which can be used to restore the degraded image quality caused by haze conditions. The main methods include: Dark Channel Prior, Super pixel and MRF (Fig.7), Atmospheric Light Estimation-based (Fig.8), Color Attenuation Prior-based (Fig.9), Detail Transmission Prior-based Image, Gradient Channel Prior-based Dehazing Algorithm. Overall, both multi-modal fusion and single-frame image processing approaches contribute to the advancement of infrared image dehazing techniques by leveraging different types of data and image processing algorithms. Conclusions and Prospects Infrared image dehazing technology will become more intelligent. Researchers are more inclined to use deep learning and convolutional neural network (CNN) techniques to achieve automated haze removal processing. In the future, infrared image dehazing technology is expected to be deeply integrated with other image processing techniques. Multi-modal fusion is a technique used to extract the most useful information from multiple data sources in order to improve the understanding and processing of image data, to enhance image quality and processing efficiency. To improve the accuracy of infrared image dehazing, it can be beneficial to incorporate visible light images or depth images. -
图 9 雾霾浓度与亮度和饱和度的差值呈正相关。(a)模糊的影像;(b)密集雾霾区域的特写图及其直方图;(c)中等模糊区域的近景斑块及其直方图;(d)无雾区域的近景斑块及其直方图
Figure 9. Haze concentration is positively correlated with the difference between brightness and saturation. (a) Blurry images; (b) Close-up of areas of dense haze and histograms; (c) Close-up patches in moderately blurred areas and their histograms; (d) Close-up patches in fog-free areas and their histograms
-
[1] Weichao H, Zhi Y, Shangbin J, et al. Research on color image defogging algorithm based on MSR and CLAHE [C]//2020 Chinese Automation Congress (CAC), 2020: 7301-7306. [2] Peng Dongliang, Wen Chenglin, Xue Anke. Theory and Application of Multi-sensor Multi-source Information Fusion [M]. Beijing: Science Press, 2010. (in Chinese) [3] Yang Yuqing. Research on learning based super-resolution reconstruction of single frame image [D]. Beijing: Beijing University of Technology, 2018. (in Chinese) [4] Chen Kewen, Zhang Zuping, Long Jun. Research progress and new trends in multi-source information fusion [J]. Computer Science, 2013, 40(8): 6-13. (in Chinese) [5] Zhang Haiyu. Research on super-resolution reconstruction algorithm of single frame image based on generative adversarial network [D]. Xi'an: Shaanxi University of Science and Technology, 2019. (in Chinese) [6] Wu Q, Zhang J, Ren W, et al. Accurate transmission estimation for removing haze and noise from a single image [J]. IEEE Transactions on Image Process, 2020, 29: 2583-2597. doi: 10.1109/TIP.2019.2949392 [7] Schechner Y, Narasimhan S G, Nayar S K. Polarization-based vision through haze [J]. Appl Opt, 2003, 42(3): 511-525. doi: 10.1364/AO.42.000511 [8] Kudo Y, Kubota A. Image dehazing method by fusing weighted near infrared image [C]//Proceedings of the International Workshop on Advanced Image Technology (IWAIT), 2018: 1-2. [9] Gao X, Tian Y, Song F, et al. Research on the defogging algorithm based on image enhancement [C]//2021 3rd International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI), 2021: 478-481. [10] Huang Weichao, Yang Zhi, Jiao Shangbin, et al. Research on color image defogging algorithm based on MSR and CLAHE [C]//2020 Chinese Automation Congress (CAC), 2020: 7301-7306. [11] Yang Jianneng. Research on optimization of image enhancement algorithm based on histogram equalization [D]. Urumqi: Xinjiang University, 2021. (in Chinese) [12] Han Shaogang. Research on image enhancement algorithm based on multi-histogram equalization [D]. Anqing: Anqing Normal University, 2020. (in Chinese) [13] Dai Shengkui, Zhong Zheng, Huang Zhengwei. Double histogram equalization algorithm based on maximum entropy model [J]. Acta Electronica Sinica, 2019, 47(3): 678-685. (in Chinese) [14] Patel S. Goswami comparative analysis of histogram equalization techniques [C]//2014 International Conference on Contemporary Computing and Informatics (IC3I), 2014: 167-168. [15] Kim T K, Paik J K, Kang B S. Contrast enhancement system using spatially adaptive histogram equalization with temporal filtering[J]. IEEE Transactions on Consumer Electronics, 1998, 44(1): 82-87. [16] Wen Haiqiong, Li Jiancheng. Adaptive threshold image enhancement algorithm based on histogram equalization [J]. China Integrated Circuits, 2021, 26: 135-136. (in Chinese) [17] Zuiderveld K. Contrast limited adaptive histogram equalization [J]. Graphics Gems, 1994, 39: 45-52. [18] Land E H. The retinex [J]. American Scientist, 1964, 52(2): 247-264. [19] Jobson D J, Rahman Z. Properties and performance of a center/surround retinex [J]. IEEE Transactions on Image Processing, 1997, 6(3): 451-462. doi: 10.1109/83.557356 [20] Li Hua Shuo. Research on low-illumination image enhancement based on wavelet and multi-scale Retinex fusion algorithm [D]. Beijing: China University of Mining and Technology, 2021. (in Chinese) [21] Zhang H, Dai S, Image inpainting based on wavelet decomposition [C]//2012 International Workshop on Information and Electronics Engineering (IWIEE), 2012, 29: 3674-3678. [22] He Xingzhen. Research on image restoration algorithm based on attention model and wavelet decomposition [D]. Chengdu: Chengdu University of Technology, 2020. (in Chinese) [23] Wang Xin, Xu Pingping, Wu Fei. Infrared image enhancement algorithm based on exponential homomorphic filtering coupled with detail sharpening rule [J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(10): 9-16. (in Chinese) [24] Dong Jingwei, Zhao Chunli, Hai Bo. Research on image defogging algorithm based on fusion of homomorphic filter and Wavelet transform [J]. Journal of Harbin University of Science and Technology, 2019, 24(1): 66-70. (in Chinese) [25] Zhang Ke, Liao Yulong, Luo Yalun, et al. Infrared image enhancement algorithm based on improved homomorphic filter [J]. Advances in Laser and Optoelectronics, 2023, 60(10): 63-69. (in Chinese) [26] He K, Sun J, Tang X. Single image haze removal using dark channel prior [J]. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 2341-2353. doi: 10.1109/TPAMI.2010.168 [27] Tan Y, Wang G. Image haze removal based on super pixels and Markov random field [J]. IEEE Access, 2020, 8: 60728-60736. doi: 10.1109/ACCESS.2020.2982910 [28] Ancuti C, Ancuti C O, Vleeschouwer C D, et al. Day and nighttime dehazing by local airlight estimation [J]. IEEE Trans Image Process, 2020, 29: 6264-6275. doi: 10.1109/TIP.2020.2988203 [29] Zhu Q, Mai J, Shao L. A fast single image haze removal algorithm using color attenuation prior [J]. IEEE Trans Image Process, 2015, 24: 3522-3533. doi: 10.1109/TIP.2015.2446191 [30] Li Zhang H, Yuan D, et al. Single image dehazing using the change of detail prior [J]. Neurocomputing, 2015, 156: 1-11. doi: 10.1016/j.neucom.2015.01.026 [31] Singh D, Kumar V, Kaur M. Single image dehazing using gradient channel prior [J]. Appl Intell, 2019, 49: 4276-4293. doi: 10.1007/s10489-019-01504-6 [32] Chang Liang, Deng Xiaoming, Zhou Mingquan, et al. Convolutional neural networks in image understanding [J]. Journal of Autochemistry, 2016, 42(9): 1300-1312. (in Chinese) [33] Hu Zhongyuan, Xue Yu, Zha Jiajie, et al. Survey on evolutionary recurrent neural networks [J]. Computer Science, 2023, 50(3): 254-265. (in Chinese) [34] Liu Cuilian. Research on hyperspectral image classification based on CNN and LSTM [D]. Chongqing: Chongqing University of Posts and Telecommunications, 2022. (in Chinese) [35] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets [C]//Neural Information Processing Systems, 2014. [36] Bengio Y, Courville A, Vincent P. Representation learning: A review and new perspectives [C]//Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828. [37] He Jun, Zhang Caiqing, Li Xiaozhen, et al. Research review of multi-modal fusion technology for deep learning [J]. Computer Engineering, 2020, 46(5): 1-11. (in Chinese)