-
多特征融合是模式识别中的一种经典方法,可以通过综合不同类别特征的优势提高识别的稳健性。针对红外图像目标分类问题,文中选用PCA、LBP以及SIFT对目标相关特性进行综合描述。PCA特征基于红外图像的像素分布获取,能够利用低维度的特征矢量描述整体红外图像的分布规律。LBP作为一种局部描述子,能够有效反映目标的局部纹理特性。SIFT作为一种经典的点特征,具有尺度不变性,并且与红外图像中目标的局部结构关联。为此,上述三类特征在描述红外图像中目标特性方面具有良好的互补性。它们共同使用可为后续的分类决策提供更为充分的信息,有利于提高目标分类性能。
-
MCCA[17-18]作为传统典型相关分析(canonical correlation analysis,CCA)的改进形式,能够对多个高斯分布随机变量进行分析处理。假设有n个随机变量,记为
$ {X_1},{X_2}, \cdots,{X_n} $ ,各个随机变量对应的维度分别为$ {m_i}(i = 1,2, \cdots,n) $ ,并且$ {m_1} $ 为最小。对每个随机变量进行去均值处理后,可利用MCCA进行分析,其准则函数描述如下:$$ {J_{{\text{MCCA}}}}({\alpha _1},{\alpha _2}, \cdots,{\alpha _n}) = \frac{{\displaystyle\sum\limits_{i = 1}^n {\displaystyle\sum\limits_{j = 1}^n {\alpha _i^{\text{T}}{S_{ij}}{\alpha _j}} } }}{{\sqrt {\displaystyle\sum\limits_{i = 1}^n {\alpha _i^{\text{T}}{S_{ii}}{\alpha _i}} } }} $$ (1) 式中:
$ {S_{ij}} = E({X_i}X_j^{\text{T}}) $ ,表示$ {X_i} $ 与$ {X_j} $ 的互协方差矩阵。上述函数按照公式(2)进行最大化:$$ \begin{array}{l} \mathop {\max }\limits_{{\alpha _1},{\alpha _2}, \cdots,{\alpha _n}} \displaystyle\sum\limits_{i = 1}^n {\displaystyle\sum\limits_{j = 1}^n {\alpha _i^{\text{T}}{S_{ij}}{\alpha _j}} }\\ {{s}}{{.t}}{\text{. }}\displaystyle\sum\limits_{i = 1}^n {\alpha _i^{\text{T}}{S_{ii}}{\alpha _i}} {\text{ = }}1 \end{array}$$ (2) 采用Lagrange乘子法对上述优化问题进行求解,转化为如下形式:
$$ \left( {\begin{array}{*{20}{c}} {{S_{11}}}& \ldots &{{S_{1n}}}\\ \vdots & \ddots & \vdots \\ {{S_{n1}}}& \cdots &{{S_{nn}}} \end{array}} \right)\left( \begin{array}{l} {\alpha _1}\\ \vdots \\ {\alpha _n} \end{array} \right) = \left( {\begin{array}{*{20}{c}} {{\lambda _1}{S_{11}}}& \ldots &0\\ \vdots & \ddots & \vdots \\ 0& \cdots &{{\lambda _n}{S_{nn}}} \end{array}} \right)\left( \begin{array}{l} {\alpha _1}\\ \vdots \\ {\alpha _n} \end{array} \right)$$ (3) 对公式(3)进行求解可得变换矩阵
$ A = {[{\alpha _1},{\alpha _2}, \cdots,} {{\alpha _n}]^{\text{T}}} $ ,其中:$$ \begin{array}{l} {\alpha _1} = {\left[ {{\alpha _{11}},{\alpha _{12}}, \cdots,{\alpha _{1{m_1}}}} \right]_{{m_1} \times {m_1}}} \\ {\alpha _2} = {\left[ {{\alpha _{21}},{\alpha _{22}}, \cdots,{\alpha _{2{m_1}}}} \right]_{{m_2} \times {m_1}}} \\ \;\; \;\; \;\; \;\; \;\; \;\;{\text{ }} \vdots \\ {\alpha _n} = {\left[ {{\alpha _{n1}},{\alpha _{n2}}, \cdots,{\alpha _{n{m_1}}}} \right]_{{m_n} \times {m_1}}} \end{array} $$ (4) 公式(4)包含了不同随机变量
$ {X_i} $ 的最优投影方向。在此基础上,可以对多个随机变量(即多源数据)进行融合处理,获得统一的表示形式:$$ Z = \alpha _1^{\text{T}}{X_1} + \alpha _2^{\text{T}}{X_2} + \cdots + \alpha _n^{\text{T}}{X_n} $$ (5) 文中采用MCCA对红外图像中提取的PCA、LBP以及SIFT特征进行融合分析,据此构造单一的特征矢量。融合后的特征矢量不仅继承了三类原始特征的鉴别信息,还能高效去除它们中存在的冗余成分。因此,融合后的特征矢量有利于提升后续分类决策的精度和效率。
-
ELM作为一种的新的分类机制,在图像识别问题中得到了广泛运用[19-21]。假设有N个训练样本,记为
$ ({x_j},{t_j}),j = 1,2, \cdots,N $ 。每一个训练样本中,${x_j} = {({x_{j1}},} {{x_{j2}}, \cdots,{x_{jm}})^{\text{T}}} \in {{{R}}^m}$ 为样本特征矢量,${t_j} = {({t_{j1}},{t_{j2}}, \cdots,{t_{jn}})^{\text{T}}} \in {{{R}}^n}$ 对应$ {x_j} $ 的期望输出。假设有L为隐藏层单元数目,ELM模型描述如下[19-21]:$$ \sum\limits_{i=1}^{L}{g({{w}_{i}}\cdot {{x}_{j}}+{{b}_{i}})}{{\beta }_{i}}={{o}_{j}},j=1,2,\cdots ,N$$ (6) 式中:
$ g(x) $ 为激活函数;$ {o_j} $ 为$ {x_j} $ 的实际输出;$ {w_i} = {({w_{i1}},{w_{i2}}, \cdots,{w_{im}})^{\text{T}}} $ 和$\; {\beta _i} = {({\beta _{i1}},{\beta _{i2}}, \cdots,{\beta _{in}})^{\text{T}}} $ 对应不同输入输出单元之间的权值向量;$ {b_i} $ 表示第$ i $ 个隐单元的偏置值;$ {{w}_{i}}\cdot {{x}_{j}} $ 表示计算$ {w_i} $ 与$ {x_j} $ 两者的内积。ELM旨在最小化期望与实际输出的之间差异。存在$ {w_i},{b_i},{\beta _i} $ 使得下式成立:$$ \sum\limits_{i=1}^{L}{g({{w}_{i}}\cdot {{x}_{j}}+{{b}_{i}})}{{\beta }_{i}}={{t}_{j}},j=1,2,\cdots ,N $$ (7) 通过矩阵形式对公式(3)进行变换,获得:
$$ H\beta = T $$ (8) $ H,\beta,T $ 分别构造如下:$$ \begin{array}{*{20}{l}} {H{\rm{ = }}{{\left[ {\begin{array}{*{20}{c}} {g({w_1}\cdot {x_1} + {b_1})}& \cdots &{g({w_L}\cdot {x_1} + {b_L})}\\ \vdots &{}& \vdots \\ {g({w_1}\cdot {x_N} + {b_1})}& \cdots &{g({w_L}\cdot {x_N} + {b_L})} \end{array}} \right]}_{N \times L}}}\\ {\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\beta {\rm{ = }}{{\left[ {\begin{array}{*{20}{l}} {\beta _1^{\rm{T}}}\\ {\beta _2^{\rm{T}}}\\ \vdots \\ {\beta _L^{\rm{T}}} \end{array}} \right]}_{L \times N}},T{\rm{ = }}{{\left[ {\begin{array}{*{20}{l}} {t_1^{\rm{T}}}\\ {t_2^{\rm{T}}}\\ \vdots \\ {t_N^{\rm{T}}} \end{array}} \right]}_{N \times n}}} \end{array} $$ (9) 具体训练过中,在初始化向量
$ {w_i} $ 和偏置$ b $ 的条件下,矩阵$ H $ 可相应确定。然后,通过最小二乘解算法进行网络训练,求得矩阵$ \; \beta {\text{ = }}{H^ + }T $ 。根据现有文献,ELM在图像识别问题中已经得到应用和验证,并且对噪声干扰、遮挡等复杂条件具有一定的适应性。为此,文中选用ELM作为基础分类器对融合后的特征矢量进行分类。
根据上述特征提取和分类器的思路,文中提出的红外目标分类方法的基本流程如图1所示,可划分为特征提取、特征融合和分类决策三个阶段。首先,分别获取目标红外图像的PCA、LBP和SIFT三类特征。在此基础上,基于设计的MCCA对三者的特征矢量进行融合处理,获得单一特征矢量。最终,采用训练样本对ELM分类器进行训练,并用于未知样本的类别确认。
-
采用飞机目标红外图像作为样本对提出方法进行测试,四类目标的图像如图2所示。对于每一类目标,选用其120幅红外图像作为训练样本,100幅图像作为测试样本。
实验中,利用现有文献中的几类方法与提出方法进行对比。具体包括参考文献[6]中基于LBP的方法,参考文献[9]中基于SIFT的方法,以及参考文献[15]中基于CNN的方法。后续实验中,首先基于原始样本对几类方法进行测试,然后对原始样本进行噪声添加处理,测试不同方法在模拟噪声干扰条件下的分类性能。
-
按照上述实验设置,对四类飞机目标的原始测试样本进行分类,据此得出文中方法的分类结果如图3所示。图中,横坐标表示测试样本的类别,纵坐标表示对应的正确分类样本数目。因此,通过对角线元素可直观地看出不同飞机目标正确识别的样本数,从而计算得到它们对应的正确分类精度分别为98%、93%、99%和94%。可以发现,第二类和第四类目标的正确识别精度相对较低,主要因为两者的外形相似度较高(从图2可以看出),导致两个类别之间出现了更为严重误分类。通过测试几类对比方法,获得LBP、SIFT和CNN方法的平均分类精度分别为94.6%、95.1%和95.6%,文中方法以96%的结果优于三类对比方法,反映了其在当前场景下的有效性。与LBP和SIFT两类采用单一特征的方法相比,提出方法通过有效融合PCA、LBP和SIFT三类特征,并采用ELM进行决策,较大幅度提升了最终的分类精度。CNN方法在当前条件下的性能仅次于提出方法,表明其较强的分类性能。
-
目标红外图像获取过程中不可避免地受到各类噪声的干扰,导致图像质量的降低。因此,有必要在受到噪声影响条件下对红外目标分类方法的分类性能进行进一步测试和验证。文中在原始测试样本的基础上,通过白噪声添加的方式获取不同信噪比(signal-to-noise ratio,SNR)条件下的测试样本。具体地,以原始测试图像的能量为参照,根据预设的SNR计算需要添加的噪声方差,最后将原始样本与噪声混合,即可获得对应噪声水平的测试样本。利用不同方法对不同SNR条件下的测试集进行目标分类,获得如表1所示的结果。根据表1可以得出如下结论:一是噪声干扰对红外图像目标分类性能有着较为显著的影响,各类方法的平均分类精度均随着SNR的降低而出现降低;二是提出方法可在不同SNR条件下保持最高分类精度,反映其噪声稳健性。与LBP、SIFT采用单一特征的方法相比,提出方法通过融合三类特征并采用ELM分类提出了噪声稳健性。CNN方法在低SNR条件下性能降低十分显著,主要因为训练样本(SNR较高)对于测试样本的描述能力出现下降。
表 1 噪声干扰条件下的性能对比
Table 1. Performance comparison under noise corruption
Method SNR/dB 8 4 0 −4 −8 Proposed 94.8 88.7 79.3 70.4 59.8 LBP 93.2 85.2 77.6 67.5 57.3 SIFT 93.4 85.9 78.2 68.2 58.0 CNN 93.9 87.2 78.7 69.0 58.4
Target classification method in infrared images via combination of multi-feature fusion and extreme learning machine
-
摘要: 针对红外图像目标分类问题,提出了结合多特征融合和极限学习机(extreme learning machine,ELM)的方法。采用主成分分析(principal component analysis,PCA)、局部二值模式(local binary pattern,LBP)以及尺度不变特征变换(scale-invariant feature transform,SIFT)三类特征分别描述红外图像中目标的像素分布、局部纹理以及特征点信息。三类特征从不同侧面反映红外图像目标特性,因此具有互为补充的优势。在此基础上,基于多重集典型相关分析(multiset canonical correlations analysis,MCCA)对三类特征进行融合处理,获得统一的特征矢量。融合后的特征不仅继承了原始三类特征的鉴别特性,还有效去除了冗余信息。分类过程中,采用极限学习机作为基础分类器对融合特征矢量进行分类。极限学习机具有参数少、效率高、精度高和稳健性强等显著特点,有利于提高红外目标分类的整体性能。因此,所提出的方法通过结合多特征和极限学习机的优势综合提升了目标识别性能。在实验过程中,采用四类飞机目标的红外图像对所提出方法进行了性能测试。根据与现有几类方法的对比,实验结果证明了提出方法的性能优势。Abstract: For the problem of infrared image target classification, a method combining multi-feature fusion and extreme learning machine (ELM) was proposed. Three types of features, i.e., principal component analysis (PCA), local binary pattern (LBP) and scale-invariant feature transform (SIFT) were used to describe the pixel distribution, local texture and feature point information of the target in the infrared image. The three types of features reflected the characteristics of infrared image targets from different aspects, so they had complementary advantages. Afterwards, the three types of features were fused based on multiset canonical correlations analysis (MCCA) to obtain a unified feature vector. The fused features not only inherited the distinguishing characteristics of the original three types of features, but also effectively removed redundant information. In the classification process, The ELM was used as a basic classifier to classify the fused feature vector. ELM had the obvious characteristics of few parameters, high efficiency, high precision and strong robustness, so it was helpful to improve the overall performance of infrared target classification. Therefore, the proposed method comprehensively improved the target recognition performance by combining the advantages of multiple features and ELM. During the experiment, the infrared images of four types of aircraft targets were used to test the performance of the proposed method. According to the comparison with several existing methods, the experimental results prove the performance advantages of the proposed method.
-
表 1 噪声干扰条件下的性能对比
Table 1. Performance comparison under noise corruption
Method SNR/dB 8 4 0 −4 −8 Proposed 94.8 88.7 79.3 70.4 59.8 LBP 93.2 85.2 77.6 67.5 57.3 SIFT 93.4 85.9 78.2 68.2 58.0 CNN 93.9 87.2 78.7 69.0 58.4 -
[1] 王园园, 赵耀宏, 罗海波, 等. 海面红外图像的动态范围压缩及细节增强[J]. 红外与激光工程, 2019, 48(1): 0126003. doi: 10.3788/IRLA201948.0126003 Wang Yuanyuan, Zhao Yaohong, Luo Haibo, et al. Dynamic range compression and detail enhancement of sea-surface infrared image [J]. Infrared and Laser Engineering, 2019, 48(1): 0126003. (in Chinese) doi: 10.3788/IRLA201948.0126003 [2] 王洪庆, 许廷发, 孙兴龙, 等. 目标运动轨迹匹配式的红外-可见光视频自动配准[J]. 光学精密工程, 2018, 26(6): 1533-1541. doi: 10.3788/OPE.20182606.1533 Wang Hongqing, Xu Tingfa, Sun Xinglong, et al. Infrared-visible video registration with matching motion trajectories of targets [J]. Optics and Precision Engineering, 2018, 26(6): 1533-1541. (in Chinese) doi: 10.3788/OPE.20182606.1533 [3] 乔铁英, 蔡立华, 李宁, 等. 基于红外辐射特性系统实现对面目标测量[J]. 中国光学, 2018, 11(5): 804-811. doi: 10.3788/co.20181105.0804 Qiao Tieying, Cai Lihua, Li Ning, et al. Opposite target measurement based on infrared radiation characteristic system [J]. Chinese Optics, 2018, 11(5): 804-811. (in Chinese) doi: 10.3788/co.20181105.0804 [4] 吴彩莲, 郝永平, 张乐, 等. 基于多特征融合的红外目标识别算法 [J]. 弹箭与制导学报, 2019, 39(3): 39–44. Wu Cailian, Hao Yongping, Zhang Le, et al. Infrared target recognition algorithm based on multi-feature fusion [J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2019, 39(3): 39-44. (in Chinese) [5] 苏娟, 杨罗, 张阳阳. 基于轮廓片段匹配和图搜索的红外目标识别方法[J]. 兵工学报, 2015, 36(5): 854–860. doi: 10.3969/j.issn.1000-1093.2015.05.013 Su Juan, Yang Luo, Zhang Yangyang. Infrared target recognition algorithm based on fragment matching and graph searching [J]. Acta Armamentarii, 2015, 36(5): 854-860. (in Chinese) doi: 10.3969/j.issn.1000-1093.2015.05.013 [6] 谢志华, 刘国栋. 基于多尺度局部二元模式共生直方图的红外人脸识别 [J]. 红外与激光工程, 2015, 44(1): 391–397. doi: 10.3969/j.issn.1007-2276.2015.01.065 Xie Zhihua, Liu Guodong. Infrared face recognition based on co-occurrence histogram of multi-scale local binary patterns [J]. Infrared and Laser Engineering, 2015, 44(1): 391-397. (in Chinese) doi: 10.3969/j.issn.1007-2276.2015.01.065 [7] Zhao Aigang, Wang Hongli, Yang Xiaogang, et al. An affine invariant method of forward looking infrared target recognition [J]. Laser & Optoelectronics Progress, 2015, 52(7): 071501. (in Chinese) [8] 李萍, 张波, 尚怡君. 基于红外图像和特征融合的飞机目标识别方法[J]. 电光与控制, 2016, 23(8): 92–96. Li Ping, Zhang Bo, Shang Yijun. Aircraft target identification based on infrared image and feature fusion [J]. Electronics Optics & Control, 2016, 23(8): 92-96. (in Chinese) [9] 王世亮, 杨帆, 张志伟, 等. 基于目标红外特征与SIFT特征相结合的目标识别算法[J]. 红外技术, 2012, 34(9): 503–507. Wang Shiliang, Yang Fan, Zhang Zhiwei. Target recognition method based on infrared features and SIFT [J]. Infrared Technology, 2012, 34(9): 503-507. (in Chinese) [10] 张迪飞, 张金锁, 姚克明, 等. 基于SVM分类的红外舰船目标识别[J]. 红外与激光工程, 2016, 45(1): 0104004. doi: 10.3788/irla201645.0104004 Zhang Difei, Zhang Jinsuo, Yao Keming, et al. Infrared ship-target recognition based on SVM classification [J]. Infrared and Laser Engineering, 2016, 45(1): 0104004. (in Chinese) doi: 10.3788/irla201645.0104004 [11] 杨春伟, 王仕成, 廖守忆, 等. 基于核稀疏编码的红外目标识别方法[J]. 红外技术, 2016, 38(3): 230–235. doi: 10.11846/j.issn.1001_8891.201603010 Yang Chunwei, Wang Shicheng, Liao Shouyi. An infrared target recognition method based on kernel sparse coding [J]. Infrared Technology, 2016, 38(3): 230-235. (in Chinese) doi: 10.11846/j.issn.1001_8891.201603010 [12] 杜珺, 高九萍. 一种改进型PSO-BP算法在红外目标中的应用[J]. 火力与指挥控制, 2020, 45(6): 62–66. Du Jun, Gao Jiuping. Research and application of an improved PSO-BP algorithm in infrared targets [J]. Fire Control & Command Control, 2020, 45(6): 62-66. (in Chinese) [13] 黄乐弘, 曹立华, 李宁, 等. 深度学习的空间红外弱小目标状态感知方法[J]. 中国光学, 2020, 13(3): 527-536. Huang Lehong, Cao Lihua, Li Ning, et al. A state perception method for infrared dim and small targets with deep learning [J]. Chinese Optics, 2020, 13(3): 527-536. (in Chinese) [14] D’acremont A, Fablet R, Baussard A, et al. CNN-based target recognition and identification for infrared imaging in defense systems [J]. Sensors, 2019, 19: 2040. doi: 10.3390/s19092040 [15] 许来祥, 刘刚, 刘森, 等. 基于改进CNN的红外目标识别方法研究[J]. 火力与指挥控制, 2020, 45(8): 136–141. doi: 10.3969/j.issn.1002-0640.2020.08.023 Xu Laixiang, Liu Gang, Liu Sen, et al. Research on infrared target recognition based on improved convolution neural network [J]. Fire Control & Command Control, 2020, 45(8): 136-141. (in Chinese) doi: 10.3969/j.issn.1002-0640.2020.08.023 [16] 史国军. 深度特征联合表征的红外图像目标识别方法[J]. 红外与激光工程, 2021, 50(3): 20200399. doi: 10.3788/IRLA20200399 Shi Guojun. Target recognition method of infrared imagery via joint representation of deep features [J]. Infrared and Laser Engineering, 2021, 50(3): 20200399. (in Chinese) doi: 10.3788/IRLA20200399 [17] Peng J L, Li Q, El-latif A A, et al. Linear discriminant multi-set canonical correlations analysis (LDMCCA): An efficient approach for feature fusion of finger biometrics [J]. Multimedia Tools and Applications, 2015, 74(13): 4469-4486. doi: 10.1007/s11042-013-1817-x [18] 邱爱昆, 朱嘉钢. 基于集成学习的多重集典型相关分析方法[J]. 计算机工程与应用, 2017, 53(6): 162–169. doi: 10.3778/j.issn.1002-8331.1510-0147 Qiu Aikun, Zhu Jiagang. Multi-set canonical correlations analysis based on ensemble learning [J]. Computer Engineering and Applications, 2017, 53(6): 162-169. (in Chinese) doi: 10.3778/j.issn.1002-8331.1510-0147 [19] 王鹏, 张肖敏, 白艳萍. 基于CNN-ELM的SAR图像分类识别 [J]. 数学的实践与认识, 2018, 48 (23): 75–80. Wang Peng, Zhang Xiaomin, Bai Yanping. Classification and recognition of SAR image based on CNN-ELM [J]. Mathematics in Practice and Theory, 2018, 48(23): 75-80. (in Chinese) [20] 刘志超, 屈百达. 结合BM3D去噪与极限学习机的SAR目标分类方法 [J]. 电光与控制, 2021, 28(6): 29–32. doi: 10.3969/j.issn.1671-637X.2021.06.007 Liu Zhichao, Qu Baida. An SAR target classification method based on BM3D denoising and extreme learning machine [J]. Electronics Optics & Control, 2021, 28(6): 29-32. (in Chinese) doi: 10.3969/j.issn.1671-637X.2021.06.007 [21] 张建明, 刘阳春, 吴宏林. 基于极限学习机与子空间追踪的人脸识别算法 [J]. 计算机工程, 2016, 42(1): 168–173. doi: 10.3969/j.issn.1000-3428.2016.01.030 Zhang Jianming, Liu Yangchun, Wu Honglin. Face recognition algorithm based on extreme learning machine and subspace pursuit [J]. Computer Engineering, 2016, 42(1): 168-173. (in Chinese) doi: 10.3969/j.issn.1000-3428.2016.01.030