-
循环神经网络(Recurrent Neural Network,RNN)可记忆上一时刻信息并应用于当前时刻输出计算,其隐藏层节点相互连接。假设
$ {{\boldsymbol{x}}_t} \in {\mathbb{R}^M} $ 与$ {{\boldsymbol{h}}_t} \in {\mathbb{R}^D} $ 分别表示$ t $ 时刻网络输入和隐藏层状态,在t时刻网络的输出如公式(1)、(2)所示:$$ {{\boldsymbol{h}}_t} = \tanh ({\boldsymbol{W}}{{\boldsymbol{h}}_{t - 1}} + {\boldsymbol{{I}}}{{\boldsymbol{x}}_t}) $$ (1) $$ {{\boldsymbol{o}}_t} = g({\boldsymbol{V}}{{\boldsymbol{h}}_t}) $$ (2) 式中:
$ g( \cdot ) $ 为softmax激活函数$ {\boldsymbol{W}} $ 为输入权重矩阵;$ {\boldsymbol{I}} $ 为输入的样本权重矩阵;$ {\boldsymbol{V}} $ 为输出的样本权重矩阵。 -
LSTM是一种引入了门控机制的RNN,由Hochreiter和Schmidhube[21]提出用于解决RNN时间延迟与梯度消失的问题。LSTM在每个时间步长中引入新的内部状态
$ {{\boldsymbol{c}}_t} \in {\mathbb{R}^D} $ 进行信息传递,并输出信息给隐藏层的外部状态$ {{\boldsymbol{h}}_t} \in {\mathbb{R}^D} $ ,t时刻的网络可由公式(3)、(4)描述:$$ {{\boldsymbol{c}}_t} = {{\boldsymbol{f}}_t} \odot {{\boldsymbol{c}}_{t - 1}} + {{\boldsymbol{i}}_t} \odot {{\boldsymbol{\tilde c}}_t} $$ (3) $$ {{\boldsymbol{h}}_t} = {{\boldsymbol{o}}_t} \odot \tanh ({{\boldsymbol{c}}_t}) $$ (4) 式中:
$ {{\boldsymbol{f}}_t} $ 、$ {{\boldsymbol{i}}_t} $ 、$ {{\boldsymbol{o}}_t} $ 分别为遗忘门、输入门与输出门;$ \odot $ 为向量元素乘积;$ {{\boldsymbol{\tilde c}}_t} $ 为通过非线性激活函数得到的候选状态;$ {{\boldsymbol{c}}_t} $ 为记录了到当前时刻为止的历史信息。 -
注意力机制旨在通过资源分配从大量信息中选择对当前任务目标最关键的信息[22],被广泛应用于深度学习领域[23]。假设D维向量
$ {{\boldsymbol{K}}_n} \in {\mathbb{R}^D},n \in [1,N] $ 表示一组输入信息,则$ \left( {{\boldsymbol{K}},{\boldsymbol{V}}} \right) = \left[ {({{\boldsymbol{k}}_1},{{\boldsymbol{v}}_1}), \cdots ,({{\boldsymbol{k}}_N},{{\boldsymbol{v}}_N})} \right] $ 表示$ N $ 组输入信息,已知查询量$ {\boldsymbol{q}} $ ,注意力函数可表示为:$$ \begin{split} att\left( {({\boldsymbol{K}},{\boldsymbol{V}}),{\boldsymbol{q}}} \right) = & \sum\limits_{n = 1}^N {{\alpha _n}{{\boldsymbol{v}}_n}} = \\& \sum\limits_{n = 1}^N {\frac{{\exp \left( {s({{\boldsymbol{k}}_n},{\boldsymbol{q}})} \right)}}{{ \displaystyle \sum\nolimits_j {\exp \left( {s({{\boldsymbol{k}}_j},{\boldsymbol{q}})} \right)} }}} {{\boldsymbol{v}}_n} \\ \end{split} $$ (5) 其中,
$ {\alpha _n} $ 为注意力分布;$ s({{\boldsymbol{k}}_n},{\boldsymbol{q}}) $ 为基于缩放点积的打分函数,如公式(6)所示:$$ s({{\boldsymbol{k}}_n},{\boldsymbol{q}}) = \frac{{{{\boldsymbol{k}}_n}^{ \rm T}{\boldsymbol{q}}}}{{\sqrt D }}$$ (6) -
为了能够充分利用电路板红外温度序列数据集中故障特征及特征间的相互关系,文中提出了一种将特征提取网络FEN和特征关系学习网络RLN并行的故障诊断算法模型,如图1所示。其中,FEN用于挖掘温度序列中不同尺度下的局部特征信息;RLN用于学习特征重要程度以及发现数据不同位置的特征间相关性。由于并行的FEN与RLN已经充分提取了输入序列数据特征并学习了特征间的关系,因此直接在并行网络后连接简单的分类器即可完成故障分类。
-
FEN基于1D-CNN以残差结构构建,包括三个1D卷积模块(Conv1D Block,CB)、两个膨胀卷积模块(Dilated Conv1D Block, DCB)和一个自注意力模块(Self-Attention Block,SAB)。
模型中三个CB结构一致,输入与输出关系可表示为:
$$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{CB}} = {f_{Leak{Re} lu}}({f_{BN}}({f_{CNN}}({\boldsymbol{x}}))) $$ (7) 式中:
$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{CB}} $ 和$ {\boldsymbol{x}} $ 分别为CB的输入与输出;$ {f_{Leak{Re} lu}}( \cdot ) $ 、$ {f_{BN}}( \cdot ) $ 、$ {f_{CNN}}( \cdot ) $ 分别为LeakRelu激活函数、批标准化处理和1D卷积函数。为了构建更大的多层次感受野以学习温度序列不同范的空间特征且不增加模型训练参数,参考MFCN[17]构建三层的DCB,如图2所示。其输入输出关系如公式(12)~(14)所示:
$$ V{F}_{Li}={f}_{Leak{Re}lu}({f}_{BN}({f}_{Dialted}(I{F}_{L},{s}_{L},{d}_{i}))) $$ (8) $$ I{F_{L + 1}} = {f_{CONTCAT}}([V{F_{L1}},V{F_{L2}},V{F_{L3}}])$$ (9) $$ {\boldsymbol{Ou}{\boldsymbol{t}}}_{Dialted}={f}_{Leak{Re}lu}({f}_{BN}({f}_{CNN}(I{F}_{3}))) $$ (10) 式中:
${f_{CONTCAT}}( \cdot )$ 为连接函数;$ {f_{Dialted}}( \cdot ) $ 为膨胀卷积函数;$ L = 1,2,3 $ ,为DCB的层数;$ i = 1,2,3 $ ,为每层内膨胀卷积的组数;$ V{F_{Li}} $ 为第$ L $ 层第$ i $ 个膨胀卷积模块的输出;$ {d_i} $ 为第$ i $ 个膨胀卷积的膨胀因子;$ {S_L} $ 为第$ L $ 层膨胀卷积的核尺寸;$ I{F_L} $ 为第$ (L - 1) $ 层输出。依据图1的FEN结构,FEN最终输出可表示为:$$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{FEN}} = {f_{Global}}({\boldsymbol{Ou}}{{\boldsymbol{t}}_{Dialted - 2}} \oplus {\boldsymbol{Ou}}{{\boldsymbol{t}}_{CB - 3}}) $$ (11) 式中:
$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{CB - 3}} $ 为第三个CB模块的输出;$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{Dilated - 2}} $ 为第二个DCB模块的输出;$ {f_{Global}}( \cdot ) $ 为全局池化函数;$ \oplus $ 为残差连接。 -
RLN由两个具有相同结构的LSTMwAtt模块构建。第一个LSTMwAtt主要负责提取温度序列数据之间的基本关系,第二个LSTMwAtt则通过扩展先前获得的关系细节来进一步提取隐藏在数据中的更复杂的正则化关系。与直接将注意力机制和LSTM连接的网络不同,文中将LSTM网络嵌入自注意力机制中,如图3所示。将输入信号
$ {\boldsymbol{X}} $ 分别输入三个基础LSTM,依据1.2节LSTM的计算方式分别得到三个LSTM输出,通过可训练网络参数矩阵$ {{\boldsymbol{W}}_q} $ 、$ {{\boldsymbol{W}}_k} $ 和$ {{\boldsymbol{W}}_v} $ 将上述输出分别映射到三个不同的空间,得到查询向量Quary、键向量Key和值向量Value,如公式(12)~(14)所示:$$ {\boldsymbol{Q}} = {{\boldsymbol{W}}_q}{f_{LSTM - Q}}({\boldsymbol{X}}) $$ (12) $$ {\boldsymbol{K}} = {{\boldsymbol{W}}_k}{f_{LSTM - K}}({\boldsymbol{X}}) $$ (13) $$ {\boldsymbol{V}} = {{\boldsymbol{W}}_v}{f_{LSTM - V}}({\boldsymbol{X}}) $$ (14) 式中:
$ {f_{LSTM - Q}} $ 、$ {f_{LSTM - K}} $ 和$ {f_{LSTM - V}} $ 分别为基础LSTM网络;$ {\boldsymbol{Q}} $ 、$ {\boldsymbol{K}} $ 和$ {\boldsymbol{V}} $ 分别为Quary、Key以及Value构成的向量矩阵。对于每一个查询向量$ {{\boldsymbol{q}}_n} \in {\boldsymbol{Q}} $ ,依据公式(5)、(6),化简得到LSTMwAtt的输出,如公式(15)所示:$$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{RLN}} = {\boldsymbol{H}} = {\text{softmax}}\left( {\left(\frac{{{{\boldsymbol{K}}^{\rm T}}{\boldsymbol{Q}}}}{{\sqrt {{D_k}} }}\right){\boldsymbol{V}}} \right) $$ (15) 将
$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{FEN}} $ 与$ {\boldsymbol{Ou}}{{\boldsymbol{t}}_{RLN}} $ 连接,得到并行网络的最终输出,如公式(16)所示,该输出将被送入后续的简单分类器以完成故障类别分类。$$ {\boldsymbol{Out}} = {f_{CONCAT}}([{\boldsymbol{Ou}}{{\boldsymbol{t}}_{FEN}},{\boldsymbol{Ou}}{{\boldsymbol{t}}_{RLN}}]) $$ (16) -
使用德制InfraTec-R5300红外摄像仪获取尺寸
$ 320 \times 256 $ 、测量精度±1%、温度分辨率小于等于0.015 K的某航电系统电源模块红外图像,如图4所示。人为设置
$U1$ 、$U2$ 和$U3$ 为芯片故障,如表1所示。其中,F1为正常模式,F2~F18为芯片单引脚破损或接触不良引起的断路故障,F19~F27为相邻引脚短接引起的短路故障。为确保实验中获取数据的可靠性,文中设计了一种数据采集同步系统,使电源模块通电周期与红外摄像仪采集数据周期同步,只保留电源模块开始通电至稳定工作状态间的红外图像。实验在恒温环境下进行,使用采集同步系统控制摄像仪采集共计10个周期的红外图像,设置每个周期的采样时间为300 s,采样频率1 Hz。27种故障模式下共计获得有效红外图像序列81000张。参考文献[1],将红外图像序列中的$U1$ 、$U2$ 和$U3$ 三个温度区域转换为尺寸为$20 \times 20$ 的温度矩阵,并以此分别构建三个芯片的温度序列数据集。表 1 故障模式描述
Table 1. Failure mode description
Mode Description Mode Description Mode Description Mode Description F1 Normal F8 pin2 of U2-open F15 pin3 of U3-open F22 pin2 and 3 of U2 chip-short F2 pin2 of U1-open F9 pin3 of U2-open F16 pin4 of U3-open F23 pin3 and 4 of U2 chip-short F3 pin3 of U1-open F10 pin4 of U2-open F17 pin6 of U3-open F24 pin6 and 7 of U2 chip-short F4 pin4 of U1-open F11 pin5 of U2-open F18 pin7 of U3-open F25 pin2 and 3 of U3 chip-short F5 pin6 of U1-open F12 pin6 of U2-open F19 pin2 and 3 of U1 chip-short F26 pin3 and 4 of U3 chip-short F6 pin7 of U1-open F13 pin7 of U2-open F20 pin3 and 4 of U1 chip-short F27 pin6 and 7 of U3 chip-short F7 pin1 of U1-open F14 pin1 of U3-open F21 pin6 and 7 of U1 chip-short -
图5为
$U1$ 故障时$U1$ 、$U2$ 和$U3$ 温度序列数据曲线,文中将通过该曲线进一步说明数据集构建的依据。由于故障源于$U1$ ,因此$U1$ 温度曲线在不同模式下表现出较大差异;但是,无故障的$U2$ 和$U3$ 温度曲线却因$U1$ 故障也出现了温度差异。基于上述现象,文中首先选择同时使用$U1$ 、$U2$ 和$U3$ 的温度数据表征电路板工作(故障)模式。为了进一步扩充特征信息,实验中还提取了各芯片的温度变化率,并与原始温度信息组建新的温度序列,以此作电路板工作(故障)模式的第二种表征方式。
依据图6所示的数据结构,实验中共计建立四个表征电路板工作模式的温度序列数据集,如表2所示。实验中,分别取Datasets_1与Datasets_2的80%用于训练模型,20%用于验证模型的准确率;Datasets_3与Datasets_4则作为独立的数据用于测试模型的故障诊断准确率。
表 2 自制温度序列数据集参数
Table 2. Parameters of self-made temperature series datasets
Samples Sequence Features Purpose Datasets_1 19548 120 3 Train & Verify Datasets_2 19548 120 6 Train & Verify Datasets_3 4887 120 3 Test Datasets_4 4887 120 6 Test -
FEN通道中超参数设置主要遵循以下原则:
(1)为确保与FCN、MFCN及LSTM-FCN性能对比的公平性,参考文献[16]将DCB各层总通道数分别设置为128、256和128,核尺寸为8、5、3;考虑到FEN基于残差结构搭建,设置各CB通道数与DCB第一层通道数保持一致;
(2)为使网络在不增加训练参数前提下构建多层次的感受野,依据金字塔结构[24]为DCB中Layer_1与Layer_2 层的膨胀卷积设置不同的膨胀因子,通过调整各膨胀卷积通道比例以确定最优参数;
(3) CB3直接与残差节点相连,旨将输入信息在不同通道进行线性整合,因此核尺寸设置为1;
(4)以文献[25]作为理论支撑,依据工程应用经验将CB1和CB2核尺寸分别设置为11~9, 9~7, 7~5,5~3,基于文献[17]在自建数据集上验证上述核尺寸下模型的性能。实验结果显示,当核尺寸为9~7时,分类准确率最高,达到89.73%,高于第二名(核尺寸为7~5) 0.17%;但是,其计算用时为164.26 s,高于第二名19.02 s。综合考虑核尺寸与分类准确率和计算资源消耗之间的关系,设置CB1和CB2的核尺寸分别为7、5。具体FEN参数设置如表3所示。
RLN通道的中的每个LSTMwAtt模块包含两个隐藏层,设置LSTM每层为128个隐藏单元[19,26];使用Tensorflow框架中RMSprop优化器[27]自适应优化更新模型参数,动量项参数设为0.9,初始学习率设为0.01,衰减速率为0.9,一旦学习率小于0.001,保持并停止衰减以确保模型加快收敛速度的同时能够保持参数波动平稳;训练过程中使用L2正则化和Dropout层防止过拟合[28],Dropout率设置为0.5。
表 3 FEN超参数设置
Table 3. Hyperparameters settings of FEN
Parameters DCB:Layer_1 DCB:Layer_2 DCB:Layer_3 CB Dilated_1 Dilated_2 Dilated_3 Dilated_1 Dilated_2 Dilated_3 Conv1D Block1 Block2 Block3 Filters 128 256 128 128 Dilation rate 1 2 4 1 2 4 1 - Activation LeakRelu Filter size 8 5 3 7 5 1 Receptive field 1×8 2×8 4×8 1×5 2×5 4×5 1×3 7 5 1 Scales proportion 1 1 2 1 1 2 - - 1 2 1 1 2 1 2 1 1 2 1 1 模型在Datasets_1和Datasets_2数据集进行20次Epochs迭代训练与验证,当DCB模块中各膨胀卷积模块的核数量之比为
$ 1:2:1 $ 时,模型在验证集上的准确率最高,如表4所示,基于此结果,从而确定了文中后续实验中使用的最终结构模型。表 4 不同核数量配比下模型的性能
Table 4. Performance of different kernel quantity
1∶1∶2 1∶2∶1 2∶1∶1 Datasets_1 92.39% 95.15% 92.18% Datasets_2 98.19% 98.98% 97.07% 图7为诊断模型在Datasets_1和Datasets_2上的损失曲线与准确率曲线。对比图7(a)和(c),两个数据集上的训练损失与验证损失的下降趋势一致,模型没有出现过拟合;由于Datasets_2增加了温度变化率特征量,在相同算法结构与参数设定的情况下,模型在Datasets_2验证集的准确率与在Datasets_1验证集相比有所提高,最高准确率达到了98.98%,如图7(b)和(d)所示。上述实验结果表明,文中算法有效地提取温度序列信息的不同尺度的特征,并且充分利用了特征间的内在关系,因此在验证集上取得了满意的故障诊断率。实验结果同时也说明增加温度变化率而构建的温度特征可以更有效地表征各模式故障特征,有利于算法准确率的提升。
图 7 模型在Datasets_1和Datasets_2的训练与验证结果
Figure 7. Results of training and validation in Datasets_1 and Datasets_2
为了进一步对比说明文中算法模型的性能,分别使用FCN、MFCN、LSTM、LSTM-FCN在Datasets_1和Datasets_2数据集进行训练和验证,将训练好的各个模型布置于Datasets_3和Datasets_4进行测试,各模型故障诊断性能如表5所示。由于在Datasets_1和Datasets_2的训练集与验证集出于同一个数据集内,训练集与验证集具有高度相似的特征信息,因此所有的算法均在训练集中学到了有效的分类特征,在验证集上获得了较好的诊断性能,各模型间的诊断准确率差距较小,文中算法的准确率仅比LSTM-FCN模型准确率高0.32%和0.54%。
表 5 不同分类算法模型性能对比
Table 5. Performance comparison of different classi-fication algorithm models
FCN MFCN LSTM LSTM-FCN Proposed Datasets_1 93.21% 94.71% 91.78% 94.83% 95.15% Datasets_2 97.12% 96.69% 96.35% 98.44% 98.98% Datasets_3 80.71% 83.30% 80.09% 85.80% 91.15% Datasets_4 88.8% 89.56% 85.58% 91.71% 96.27% 但是在Datasets_3和Datasets_4两个独立测试集的诊断结果显示,所有模型的诊断准确率均有所下降,但仍然遵循如下趋势:多尺度MFCN分类结果优于单尺度FCN;单尺度FCN分类结果优于LSTM;LSTM-FCN融合模型的分类准确程度则高于FCN、MFCN和LSTM,但是上述算法模型的诊断准确率均低于文中模型,诊断准确率高出LSTM-FCN模型5.35%和4.56%,分类性能优势得到了体现。为了更直观地展示各模型在真实测试数据集上的性能,使用上述分类算法分别对随机选择Datasets_4的100个数据进行分类测试,模型预测的结果与实际故障结果的对比如图8所示。
图8(a)~(d)的分类错误主要集中与在F4和F27,这也是导致各个模型在测试集上分类准确率均大幅降低的主要原因。图8(e)为使用文中算法模型的故障分类结果,F4和F27样本中均只出现了一个错误分类,远远小于用于对比其他算法的模型,文中模型在测试集上依然取得了令人满意的诊断准确率。
综合上述实验结果可以发现,由于同时并行多尺度膨胀卷积与LSTMwAtt网络,文中模型不仅能够充分地挖掘训练数据中足够多的局部特征,还可以学习特征的重要性并分配权重,将不同位置特征之间关系联系起来,捕获它们的内在正则化过程。当模型应用于新的数据集时,依然能够最大化地利用局部特征与特征间的内在关系,确保较高的故障诊断准确率。但是,受限于数据集规模仍然不够,所以在独立的测试集Datasets_3和Datasets_4上,文中算法模型的分类准确率也有所降低。
-
为了科学解释模型性的能优势,文中对模型结构与参数进行消融实验。由于篇幅原因,对下述实验名称作简化替代。其中,“Relu”表示使用Relu替代文中FEN通道中的激活函数LeakRelu;“No_Dilate”表示不使用膨胀卷积结构;“No_SelfAtt”表示取消文中算法中两个DCB之间的Self-Attention层;“AttLSTM”表示仅通过简单级联LSTM与注意力机制所形成的模型;“1_LSTM”和“3_LSTM”分别表示RLN通道中使用一个和三个LSTMwAtt模块,使用更改后的模型结构在Datasets_3和Datasets_4上进行诊断测试,实验结果如表6所示。
结合表6的实验结果可以发现:FEN中采用膨胀卷积使模型可以在不增加训练参数的前提下扩展感受野尺度、提高特征覆盖面,全面学习了各模式下的温度时间序列特征;与只关注正数的Relu激活函数不同,LeakRelu同时利用了正、负数,有助于避免特征在激活过程中丢失;Self_Att层可以使FEN分配更多的权重给重要的特征,即对三个芯片提供的特征数据给予权重划分,提高真实故障芯片的特征对分类的贡献。因此,膨胀卷积、LeakRelu和Self_Att的使用均在一定程度上提高了FEN特征提取的能力和故障诊断的准确率。
表 6 消融实验性能对比
Table 6. Performance comparison of ablation experiment
Relu No_Dilate No_SelfAtt AttLSTM 1_LSTM 3_LSTM Datasets_3 89.73% 90.23% 90.21% 89.37% 88.54% 83.38% Datasets_4 95.75% 95.67% 95.07% 95.97% 93.82% 90.80% RLN通道利用LSTMwAtt模块使得通道具备了更强的关系提取能力,充分利用了温度序列数据不同位置提取的特征间内的在关系,捕获相似故障模式中的差异化数据;使用两个LSTMwAtt的故障分类准确率明显高于1_LSTM和3_LSTM,究其原因是由于第二层LSTMwAtt揭示了第一层捕获的特征之间的关系细节,已经充分挖掘并提取了数据中所忽略的复杂表示与隐藏的复杂的关系;而当使用三个LSTMwAtt时,第三层仅充当了信息传输层,当细节均在第二层中被明确揭示的前提下,第三层反而导致数据特征在传输过程中丢失,分类准确率降低,还消耗了额外计算资源。当仅使用一个AttLSTM,数据特征传输过程中则缺乏对已提取的表示间内部连接更深入的关注,导致从数据中学习的特征关系不足,降低了诊断准确率。因此,特定数量的LSTMwAtt模块的使用,在一定程度上增强了RLN的特征关系学习能力,提高了模型诊断的准确率。
通过消融实验的性能对比,对模型结构和参数的优化,可以提高对某些不易识别的故障模式的判别能力,即使在训练样本较少的前提下,依然在真实独立的测试集上取得了较为理想的诊断结果,从而也说明了文中算法具有较强的适应性和泛华性,能够为基于红外的电路板故障诊断系统提供科学合理的诊断结论。
Research on circuit board fault diagnosis based on infrared temperature series
-
摘要: 电路板红外温度序列包含了丰富的故障类别信息,充分利用其局部与全局特征可以提高电路板故障诊断的准确率。为此,文中提出了一种由特征提取网络(Features Extraction Network,FEN)与关系学习网络(Relationship Learning Network,RLN)并行构成的可综合利用温度序列局部特征及特征间关系的电路板故障诊断模型。其中,FEN基于多尺度膨胀卷积(Multi-scale Dilated CNN,MDCNN)残差结构搭建,可在不增加训练参数的前提下构建多层次感受野,学习温度序列不同范围的空间特征;RLN基于嵌入长短期记忆网络的注意力机制(Long Short-Term Memory hybridized with Attention,LSTMwAtt)结构搭建,通过控制温度序列信息传递来学习特征重要性并分配权重,挖掘不同位置特征间的相关性。实验结果显示,所提模型在两个自建电路板温度序列测试数据集上的诊断性能优于同类型的FCN、MFCN、LSTM和LSTM-FCN,故障诊断准确率分别达到91.15%和96.27%,可实现对电路板故障的高准确率诊断。Abstract:
Objective A rapid and accurate detection of the fault occurring to the airborne electronic system plays a crucial role in ensuring the safety of civil aircraft. However, due to the increase of circuit board size and component density in airborne electronic system, the traditional contact fault diagnosis method encounters various problems such as low accuracy, huge time cost and the demanding requirements on personnel competency. Therefore, this study aims to explore the solution to circuit board fault diagnosis based on non-contact infrared technology, which is essential for improving the accuracy of fault diagnosis for the airborne electronic system. Methods After the sequential thermal image of the circuit board is captured by using the infrared camera, the region of interest in the thermal image is processed as the infrared temperature series. Since the infrared temperature series of the circuit board contains various fault-related information, the accuracy of fault diagnosis can be improved by making full use of its local and global features. In this study, a fault diagnosis algorithm is proposed to achieve this purpose. Composed of the features extraction network (FEN) and the relationship learning network (RLN), it utilizes the local features of temperature series and the relationship between the features. Built on a residual structure with multi-scale dilated CNN, FEN plays the role of a local-feature extraction network to construct a multi-scale receptive field without increasing the number of training parameters and to learn the spatial features of temperature series of different ranges. Based on the embedded structure of two identical layers, attention mechanism and LSTM network, RLN is a network that can apply control on the transmission of temperature series to learn the importance of features and assign attention weights for mining the correlations between the features extracted from different positions. To develop a complete circuit board fault diagnosis algorithm, the parallel FEN and RLN networks are connected to the "Softmax" classifier. Results and Discussions The temperature series datasets representing 27 different fault categories are constructed based on the infrared thermal image of airborne power board (Tab.1, Tab.5). (1) By analyzing the temperature series datasets, it can be found that there are significant differences between the temperature curves of the chip under different fault conditions, and the temperature curves of non-faulty chips are also affected by faulty chips (Fig.5). (2) The experimental results show that the proposed algorithm achieves a better diagnostic performance than FCN, MFCN, LSTM and LSTM-FCN on the datasets of the temperature series testing on two self-built circuit boards. To be specific, its diagnostic accuracy reaches 91.15% and 96.27%, respectively (Fig.8) (Tab.5). (3) Given the identical hyperparameter setting, the increase in dimension of temperature sequence feature vector contributes to improving the diagnostic performance. That is to say, appropriate sample is one of the key influencing factors in improving the accuracy of fault diagnosis (Tab.5). (4) Ablation studies reveal that the performance of FEN in feature extraction capability can be improved by the proper setting of hyperparameters, which is conducive to enhancing the diagnostic accuracy of the algorithm (Tab.6). (5) The long Short-term Memory hybridized with Attention (LSTMwAtt) plays a role in improving the performance of the proposed algorithm in terms of relation extraction. By fully utilizing the intrinsic relationship between the characteristics of different locations of temperature series, the proposed algorithm is more likely to capture the differentiated data carried by similar faults (Tab.6). Conclusions In this study, a fault diagnosis algorithm intended for the airborne circuit board is proposed by using infrared temperature series. In this algorithm, the features extraction network is responsible for extracting local features and learning the spatial features of temperature series of different ranges, while the relationship learning network is proposed to discover the intrinsic relationships among the representations learned from infrared temperature series. According to the experimental results, the proposed diagnosis algorithm performs well on self-built testing datasets. However, it is worth noting that the small size of the self-built datasets reduces the accuracy of the algorithm when the proposed algorithm is applied to the new datasets. As the size of self-built datasets increases, it performs better in fault diagnosis. Hopefully, it would be applicable in circuit board fault systems to deal with the fault that occurs to the airborne electronic systems. -
表 1 故障模式描述
Table 1. Failure mode description
Mode Description Mode Description Mode Description Mode Description F1 Normal F8 pin2 of U2-open F15 pin3 of U3-open F22 pin2 and 3 of U2 chip-short F2 pin2 of U1-open F9 pin3 of U2-open F16 pin4 of U3-open F23 pin3 and 4 of U2 chip-short F3 pin3 of U1-open F10 pin4 of U2-open F17 pin6 of U3-open F24 pin6 and 7 of U2 chip-short F4 pin4 of U1-open F11 pin5 of U2-open F18 pin7 of U3-open F25 pin2 and 3 of U3 chip-short F5 pin6 of U1-open F12 pin6 of U2-open F19 pin2 and 3 of U1 chip-short F26 pin3 and 4 of U3 chip-short F6 pin7 of U1-open F13 pin7 of U2-open F20 pin3 and 4 of U1 chip-short F27 pin6 and 7 of U3 chip-short F7 pin1 of U1-open F14 pin1 of U3-open F21 pin6 and 7 of U1 chip-short 表 2 自制温度序列数据集参数
Table 2. Parameters of self-made temperature series datasets
Samples Sequence Features Purpose Datasets_1 19548 120 3 Train & Verify Datasets_2 19548 120 6 Train & Verify Datasets_3 4887 120 3 Test Datasets_4 4887 120 6 Test 表 3 FEN超参数设置
Table 3. Hyperparameters settings of FEN
Parameters DCB:Layer_1 DCB:Layer_2 DCB:Layer_3 CB Dilated_1 Dilated_2 Dilated_3 Dilated_1 Dilated_2 Dilated_3 Conv1D Block1 Block2 Block3 Filters 128 256 128 128 Dilation rate 1 2 4 1 2 4 1 - Activation LeakRelu Filter size 8 5 3 7 5 1 Receptive field 1×8 2×8 4×8 1×5 2×5 4×5 1×3 7 5 1 Scales proportion 1 1 2 1 1 2 - - 1 2 1 1 2 1 2 1 1 2 1 1 表 4 不同核数量配比下模型的性能
Table 4. Performance of different kernel quantity
1∶1∶2 1∶2∶1 2∶1∶1 Datasets_1 92.39% 95.15% 92.18% Datasets_2 98.19% 98.98% 97.07% 表 5 不同分类算法模型性能对比
Table 5. Performance comparison of different classi-fication algorithm models
FCN MFCN LSTM LSTM-FCN Proposed Datasets_1 93.21% 94.71% 91.78% 94.83% 95.15% Datasets_2 97.12% 96.69% 96.35% 98.44% 98.98% Datasets_3 80.71% 83.30% 80.09% 85.80% 91.15% Datasets_4 88.8% 89.56% 85.58% 91.71% 96.27% 表 6 消融实验性能对比
Table 6. Performance comparison of ablation experiment
Relu No_Dilate No_SelfAtt AttLSTM 1_LSTM 3_LSTM Datasets_3 89.73% 90.23% 90.21% 89.37% 88.54% 83.38% Datasets_4 95.75% 95.67% 95.07% 95.97% 93.82% 90.80% -
[1] Hao J X, Jia C Y. Research on fault mode diagnosis of airborne circuit board based on infrared lmages [J]. Infrared Technology, 2019, 41(3): 273-278. (in Chinese) [2] Li C L. Research on analog circuit fault diagnosis methods based on SVDD and parameter identification[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2011. (in Chinese) [3] Li G H, Lei J. Research on major technology of PCB infrared thermal detection based on information fusion [J]. Infrared Technology, 2017, 39(9): 829-834. (in Chinese) [4] Norhisham M, Bugis I, Jamaludin I, et al. Thermal analysis on PCB using Galerkin approach[C]//International Conference on Modeling. IEEE, 2011: 1-6 [5] Varghese J, Singh T, Mohan S. PCB thermal image analysis using MATLAB [J]. International Journal of Recent Advances in Engineering & Technology, 2014, 2(3): 46-52. [6] Wang M H, Chung Y K, Sung W T. Using thermal image matter-element to design a circuit board fault diagnosis system [J]. Expert Systems with Applications, 2011, 38(5): 6164-6169. doi: 10.1016/j.eswa.2010.11.055 [7] Al-Obaidy F, Yazdani F, Mohammadi F A. Fault detection using thermal image based on soft computing methods: Comparative study [J]. Microelectronics Reliability, 2017, 71(58): 88-100. [8] Cui W. Research on key technologies for printed circuit board detection by infrared thermorgraphy[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2011. (in Chinese) [9] Wang L, Wang M. Automatic image segmentation algorithm by PCNN based on mean threshold and Ostu [J]. Infrared Technology, 2015, 37(7): 553-559. (in Chinese) [10] Wang K, Zhang K, Wang L, et al. Infrared image segmentation based on MRF combined with two-algorithm game [J]. Infrared Technology, 2015, 37(2): 134-138. (in Chinese) [11] Wang L, Zeng P P, Hao J X. Infrared image registration of circuit board based on ROI gray compression [J]. Laser & Infrared, 2014, 44(3): 313-318. (in Chinese) [12] Wang L, Li S, Wu H, et al. Research of aviation circuit board fault diagnosis and prediction based on infrared technology [J]. Computer Simulation, 2017, 34(7): 89-93. (in Chinese) [13] Hao J X, Jia C Y. Circuit board fault diagnosis algorithm based SVM and improved D-S evidence theory [J]. Modern Elec-tronics Technique, 2019, 42(22): 15-20. (in Chinese) [14] Yann L C, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 147(521): 436-444. [15] Schmidhuber J. Deep learning in neural networks: An overview [J]. Neural Newworks, 2015, 61(1): 85-117. doi: https://doi.org/10.1016/j.neunet.2014.09.003 [16] Wang Z, Yan W, Oates T. Time series classification from scratch with deep neural networks: A strong baseline[C]//Inter-national Joint Conference on Neural Networks, 2017: 1578-1585. [17] Cui Z, Chen W, Chen Y. Multi-scale convolutional neural networks for time series classification[EB/OL]. (2016-03-22)[2022-07-14]. http://arxiv.org/abs/1603.06995. [18] Zou X W, Wang Z D, Li Q, et al. Integration of residual network and convolutional neural network along with various activation functions and global pooling for time series classification [J]. Neurocomputing, 2019, 367: 39-45. doi: 10.1016/j.neucom.2019.08.023 [19] Karim F, Majumdar S, Darabi H, et al. LSTM fully con-volutional networks for time series classification [J]. IEEE Access, 2018, 6: 1662-1669. doi: 10.1109/ACCESS.2017.2779939 [20] Elsayed N, Maida A S, Bayoumi M. Gated recurrent neural networks empirical utilization for time series classification [C]// 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), 2019: 1207-1210. [21] Hochreiter S, Schmidhuber J A . LSTM can solve hard long time lag problems [C]//Neural Information Processing Systems, MIT Press, 1996: 473-479. [22] Chorowski J, Bahdanau D, Serdyuk D, et al. Attention-Based Models for Speech Recognition[C]//NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015, 1: 577-585. [23] Xie G, Shangguan A, Fei R, et al. Motion trajectory prediction based on a CNN-LSTM sequential model [J]. Science China Information Sciences, 2020, 63(11): 233-253. [24] Zhao Liang, Mo Chunyang, Ma Jiajun, et al. LSTM-MFCN: A time series classifier based on multi-scale spatial-temporal features [J]. Computer Communications, 2022, 182: 52-59. doi: https://doi.org/10.1016/j.comcom.2021.10.036 [25] Tang W, Long G, Liu L, et al. Rethinking 1D-CNN for time series classification: A stronger baseline[C]//The Tenth Inter-national Conference on Learning Representations(ICLR 2022), 2022. [26] Karim F, Majumdar S, Darabi H, et al. Multivariate LSTM-FCNs for time series classification [J]. Neural Networks, 2018, 116: 237-245. [27] Xiao Z, Xu X, Xing H, et al. RTFN: A robust temporal feature network for time series classification [J]. Information Sciences, 2021, 571: 65-86. doi: 10.1016/j.ins.2021.04.053 [28] Zhu Y, Zhao C, Guo H, et al. Attention couplenet: Fully convolutional attention coupling network for object detection [C]//IEEE Transactions on Image Processing, 2019: 113-126.