-
实验硬件和软件环境如表1所示。
表 1 实验平台参数设置
Table 1. Experimental platform parameter settings
Experiment platform Model parameters Operating system Ubuntu18.4 CPU I9-12900 KF 3.2 GHz GPU RTX2070 S 8 GB Graphics card 12 G Frame Pytorch Programming language Pyhton -
由于目前并没有导线的公开数据集,故文中建立了专用于红外图像架空导线检测的标签数据集。文中利用双光无人机,采集1000张红外架空导线图片,作为数据集的来源。图像尺寸统一修改为128×128,其中,将图像中的导线标记为wire,所有图片标签均通过Labelme软件自行标注,生成json文件,并用于训练,图7为训练样本的原始图像及标签图像。按照8∶2的比例对数据集和测试集进行划分,即800张用于训练,200张用于测试。
-
为了验证文中构建的导线分割模型的性能,需选取合适的性能评价指标,在语义分割领中有多种评价精度的指标,文中选取均像素精度(Mean Pixel Accuracy,MPA)、平均交并比(Mean Intersection over Union,MIoU)这3个指标作为衡量模型分割精度的标准。其中MPA表示所有类别像素精度的均值,是语义分割领域中最普遍的指标之一,如公式(1)所示。MIoU表示的是真实值与预测值的交集比并集,如公式(2)所示。其中MPA侧重表现像素级的准确率,而MIoU更加侧重于表现模型计算分割区域的完整性以及分割位置的准确性。
$$ {{MPA}} = \dfrac{1}{{k + 1}}\displaystyle\sum\limits_{i = 0}^k {\dfrac{{{p_{ii}}}}{{\displaystyle\sum\limits_{j = 0}^k {{p_{ij}}} }}} $$ (1) $$ MIoU = \dfrac{1}{{k + 1}}\displaystyle\sum\limits_{i = 0}^k {\dfrac{{{p_{ii}}}}{{\displaystyle\sum\limits_{i = 0}^k {{p_{ij}} + \displaystyle\sum\limits_{i = 0}^k {{p_{ji}} - {p_{ii}}} } }}} $$ (2) 式中:k表示类别总数;pij表示实际像素类别i,被预测为类别j的像素数量;pii表示实际像素类别为i,被预测为类别i的像素数量;k+1表示分割类别的数量,其中含有一个背景类。
-
为说明该算法的有效性,文中先使用传统算法(Canny算子)对导线目标进行检测,进行实验对比,对30幅导线图的导线条数做了统计,人眼可看到的导线一共有132根,利用Canny算子一共检测到108根导线,检测率是81.8%,Canny算子能够检测到部分边缘,但是图像中还存在大量噪声,且会丢失部分导线信息,检测效果不佳。而后使用文中提出的算法,实验表明,该算法具有良好的检测效果。文中先使用原 Deeplabv3+的网络结构对导线目标进行检测,然后对 Deeplabv3+的特征提取主干网络ResNet50进行改进,引入注意力机制,再引入特征金字塔网络,再次进行实验,实验结果如表2所示。
表 2 实验结果度量
Table 2. Experiment results metrics
Method MPA MIOU Deeplabv3+ 92.91% 86.79% Proposed algorithm 93.52% 87.83% 3种检测算法的实验结果对比如图8所示,其中图(a)为原图;图(b)为Canny算子检测结果,从图中可以看出,Canny算子的检测存在大量噪声,且导线信息提取不完整,在道路这种具有热辐射的图像中,该算子会将道路误检为导线目标;图(c)为原 Deeplabv3+可以较好地提取到导线信息,但对于特别细小的导线目标提取不完整,这是因为原 Deeplabv3+算法在进行特征提取时,随着网络的加深,一些细小的导线目标会消失在最后一层特征图中,导致后续的网络无法检测到该目标;图(d)为文中的改进算法,结合了注意力机制及特征金字塔,改善了特征提取过程中,像素点少的细小的导线目标丢失的问题,能够完整地提取到导线信息,效果最好。
Infrared aerial image overhead wire identification algorithm based on improved Deeplabv3+
-
摘要: 随着国家电网规模的不断扩大,架空导线作为电力系统的重要组成,对它的定期巡检变得极其重要,同时,随着低空飞行领域的开放,为了保证国家电网的正常运行及低空飞行的安全,架空导线的识别也变得极其重要。文中提出了一种使用Deeplabv3+语义分割网络模型对红外航拍图像架空导线进行识别的方法,并且针对红外架空导线图像目标的特征对该算法进行了改进。首先在原Deeplabv3+算法的特征提取主干网络ResNet50中加入注意力机制,使网络突出导线目标所在区域的特征,更加关注导线目标所在的位置,进而弱化背景等非主要区域的特征。然后对Deeplabv3+的编码器部分进行改进,在ResNet50模型中加入特征金字塔网络,可以将浅层和深层的特征进行融合,增强网络对不同大小目标属性的识别能力,及导线这种小目标的检测能力,进而提高网络的整体识别效果。实验结果表明:改进后的算法检测性能良好,均像素精度为93.52%,平均交并比为87.83%。
-
关键词:
- Deeplabv3+ /
- 特征金字塔 /
- 架空导线 /
- 注意力机制 /
- ResNet50
Abstract: With the continuous expansion of the scale of the national grid, overhead conductors, as an important component of the power system, have become extremely important to its regular inspection. At the same time, with the opening of the low-altitude flight field, the identification of overhead conductors has also become extremely important to ensure the normal operation of the national grid and the safety of low-altitude flight. The method for identifying overhead wires in infrared aerial images was proposed using the Deeplabv3+ semantic segmentation network model, and the algorithm was improved according to the characteristics of infrared overhead wire image targets in this paper. Firstly, an attention mechanism was added to the feature extraction backbone network ResNet50 of the original Deeplabv3+ algorithm, so that the network highlights the characteristics of the area where the wire target was located, and paid more attention to the location of the wire target, thereby weakening the background and other non-main area features; Then, the encoder part of Deeplabv3+ was improved, and the Feature Pyramid Networks (FPN) was added to the ResNet50 model, which can fuse the shallow and deep features, enhance the network's ability to identify the attributes of targets of different sizes, and the performance of small targets such as wires, and then improve the overall recognition effect of the network. The experimental results show that the improved algorithm has good detection performance, the average pixel accuracy is 93.52%, and the average intersection ratio is 87.83%.-
Key words:
- Deeplabv3+ /
- feature pyramid /
- overhead wire /
- attention mechanism /
- ResNet50
-
表 1 实验平台参数设置
Table 1. Experimental platform parameter settings
Experiment platform Model parameters Operating system Ubuntu18.4 CPU I9-12900 KF 3.2 GHz GPU RTX2070 S 8 GB Graphics card 12 G Frame Pytorch Programming language Pyhton 表 2 实验结果度量
Table 2. Experiment results metrics
Method MPA MIOU Deeplabv3+ 92.91% 86.79% Proposed algorithm 93.52% 87.83% -
[1] Du Zhongming, Wang Xuesong. Forecast of China's electricity demand level during the "13th Five-Year Plan" [J]. China Electric Power, 2017, 50(9): 11-17. (in Chinese) doi: 10.11930/j.issn.1004-9649.201706123 [2] Pouliot N, Richard P L, Montambault S. Line scout technology opens the way to robotic inspection and maintenance of high-voltage power lines [J]. IEEE Power and Energy Technology Systems Journal, 2015, 2(1): 1-11. doi: 10.1109/JPETS.2015.2395388 [3] Zhang Congxin, Zhao Le, Wang Xianpei. A fast extraction algorithm of power lines under the background of complex ground objects [J]. Journal of Wuhan University (Engineering Edition), 2018, 51(8): 732-739. (in Chinese) [4] Zhao Haocheng, Lei Junfeng, Wang Xianpei, et al. Algorithm for power line recognition in aerial images with complex background [J]. Bulletin of Surveying and Mapping, 2019(7): 28-32. (in Chinese) [5] Han C, Liu Q. Appearance monitoring of the transmission lines based on hough transform[C]//2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 2020: 71-75. [6] Weiran C, Zhu Linlin, Jianda H. Iterable multidirectional autocorrelation approach for aerial power line image enhancement [J]. Robot, 2015, 37: 738-747. [7] Nusantika N R, Hu X G, Jin X. Improvement Canny edge detection for the UAV icing monitoring of transmission line icing[C]//2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), 2021: 1838-1843. [8] Shelhamer E, Jonathan L, Trevor D. Fully convolutional net-works for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. [9] Vo A T, Tran H S, Le T H. Advertisement image classification using convolutional neural network[C]//2017 9th International Conference on Knowledge and Systems Engineering (KSE), 2017: 197-202. [10] Ding J, Chen B, Liu H, et al. Convolutional neural network with data augmentation for SAR target recongnition [J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(3): 364-368. [11] Luo Jianjun, Liu Zhensheng, Gong Xiang, et al. State evaluation method of line insulators based on UAV images and transfer learning [J]. Electric Power Engineering Technology, 2019, 38(5): 30-36. (in Chinese) [12] Madaan R, Maturana D, Scherer S. Wire detection using synthetic data and dilated convolutional networks for unmanned aerial vehicles[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017: 3487-3494. [13] Yuan Li, Yuan Jishou, Zhang Dezheng. Remote sensing image classification based on DeepLab-v3+ [J]. Advances in Lasers and Optoelectronics, 2019, 56(15): 152801. (in Chinese) [14] Ha D, Riegler M A, Johansen D, et al. Doubleu-net: A deep convolutional neural network for medical image segmentation [C]//2020 IEEE 33rd International Symposium on Computer Based Medical Systems (CBMS), 2020: 558-564. [15] Sun L, Yang K, Hu X, et al. Real-time fusion net-work for RGB-D semantic segmentation incorporating unexpected obstacle detection for road-driving images [J]. IEEE Robotics and Automation Letters, 2020, 5(4): 5558-5565. doi: 10.1109/LRA.2020.3007457 [16] Zhao Le, Wang Xianpei, Yao Hongtai, et al. A review of power line extraction algorithms based on visible light aerial images [J]. Power System Technologey, 2021, 45(4): 1536-1546. (in Chinese) [17] Zheng Kai, Li Jiansheng. A review of image semantic segmentation based on deep neural network [J]. Surveying and Mapping and Spatial Geographic Information, 2020, 43(10): 119-125. (in Chinese) doi: 10.3969/j.issn.1672-5867.2020.10.032 [18] Xu Hui, Zhu Yuhua, Zhen Tong, et al. A review of deep neural network image semantic segmentation methods [J]. Computer Science and Exploration, 2021, 15(1): 47-59. (in Chinese) doi: 10.3778/j.issn.1673-9418.2004039 [19] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. [20] Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs[EB/OL]. (2014-12-22)[2022-04-10]. https://arxiv.org/abs/1412.7062. [21] Wang Yagang, Xi Yiyuan, Pan Xiaoying. Improved deep Labv3+ network for intestinal polyp segmentation [J]. Computer Science and Exploration, 2020, 14(7): 1243-1250. (in Chinese) [22] Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848. [23] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks[EB/OL]. (2014-09-10)[2022-04-10]. https://arxiv.org/abs/1409.3215. [24] Misra D, Nalamada T, Arasanipalai A U, et al. Rotate to attend: Convolutional triplet attention module[C]//Proceedings of the IEEE/CVF Winter Conferenc on Applications of Computer Vision, 2021: 3139-3148. [25] Wang F, Tax D M J. Survey on the attention based RNN model and its applications in computer vision[EB/OL]. (2016-01-25)[2022-04-10]. https://arxiv.org/abs/1601.06823v1. [26] Hou Ruihuan, Yang Xiwang, Wang Zhichao, et al. A real-time detection method for forestry pests based on YOLOv4-TIA [J]. Computer Engineering, 2022, 48(4): 255-261. (in Chinese) [27] Tan M, Pang R, Le Q V. Efficientdet: Scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2020: 10781-10790. [28] Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 2117-2125.