-
由于U-Net网络的特征融合只是用到contact,考虑到遥感图像数据固有的大数据的4V特性,即:快速处理速度(velocity)、高值但低密度(value)、多样(variety)、量大(volume),因此文中选择M2 DET网络中的尺度特征聚合模块SFAM与之相融合,SFAM的主要目的是将多级多尺度特征进行有效聚合,从而得到多级特征金字塔,可以获取更多的语义信息,使结果更精确[13]。SFAM的首个阶段主要顺着信道维度将目标特征进行有效衔接,此时聚合金字塔内各比例都涵盖多级深度的特征。但是,简单的连接操作不太适合。在第二部分重点采用了通道注意模块,从而使特征都涵盖于最有益的区间。在SE区块之后,可以通过全局平均池化来获得通道统计z∈RC。除此之外,为了对通道依赖性进行准确捕获,聚合模块可以进行调整。
SFAM 的首个阶段是沿着信道维度与存在等效比例的特征进行衔接,此后通过SE注意机制对目标特征进行良好的适应,如公式(1)所示:
$$ s={F}_{ex}({z},W)=s\left({W}_{2}d({W}_{1}{z})\right) $$ (1) 式中:W1∈RC/r'×C,W2∈RC×C/r',r'是减少的比例。最后通过激活函数对输入X重新加权得到输出,如公式(2)所示:
$$ {X_i} = {F_{scale}}\left(X_i^c,{s_c}\right) = {s_c}*X_i^c $$ (2) -
传统的U-Net网络是对称结构[14],左侧是编码器,右侧是解码器,在提取特征时要完成10次3×3标准卷积操作,因遥感图像存在复杂的背景,信息繁多,光谱区间广,故此基础U-Net模型提取遥感图像特征欠佳,因此此次研究经由提高基础U-Net网络深度完成对复杂度更高的光谱特征的提取,从而提升检测的精度。在原有的网络结构上加深两层编码器与解码器,右、左半部从而依然可呈中心对称,因左半部的构成为一连串上采样,故其核心对应于下采样,各层输入不仅含源自上一层的特征信息,同时含源自相应下采样层的环境信息,从而可恢复特征图细节,同时使其对应的空间信息维度恒定。
-
ASPP(atrous spatial pyramid pooling)模块是在空间维度上实现金字塔型的空洞池化[15]。针对ASPP模块而言,其能够为金字塔型的空洞池化提供良好的支持。此类设计对预设的输入以多种空洞卷积完成采样工作,其等同于通过若干比例捕捉图像的相关数据。利用shortcut将ASPP模块所产生的输入和输出实现加叠,这种方式并不会显著提升网络的计算量,但是它却能够有效增加模型的训练效率,并对实际的训练效果进行一定优化,在模型参数不断加深的过程中,该结构可以有效地对退化问题进行处理。ASPP结构如图3所示,具体计算如公式(3)所示:
$$ Y = Concat\left({I_{pooling}}(X),{H_{1.3}}(X),{H_{6.3}}(X),{H_{12.3}}(X),{H_{18.3}}(X)\right) $$ (3) 式中:Concat()为拼接操作第一维度上的特征图;Hr.n(X)表示n尺寸卷积核与r采样率的带孔卷积;Ipooling为
图3上image pooling分支的所有图像级特征,是一种将特征图输入的平均池化特征。 -
文中改进的农作物分类模型的整体架构如图4所示,其涵盖了U-Net的基本结构,一方面在拼接层中加入SFAM结构,使获取到的语义信息更加丰富;另一方面在最下层编码器与解码器彼此衔接的位置,添加ASPP结构,因此,可在多个采样率条件下产生上下文语义信息,更好地识别像素,从而保证分类效果最为理想,并且在原始U-Net结构上加深了两层,此处U-Net的对称结构又进行了一定扩增,这是由于相对较多的层数能够形成更多的语义信息,最后获得的特征就更为全面,即保证了加深之后不会出现梯度爆炸的情况,并对遥感图像农作物进行标准化的分类和识别。首先,输入图片,运用卷积进行下采样特征提取,通过SFAM结构,使得语义信息更为丰富,然后通过ASPP模块使下采样提取的特征更为丰富,防止加深的网络出现梯度爆炸,最后在右侧进行反卷积,得出结果图片。
-
文中选取实验环境搭载ENVI5.3遥感数据处理软件,与图片处理软件Matlab 2017。深度学习服务器的CPU为IntelXeon E5-2678 v3 CPU,其主频最高为2.5 GHz,显卡是NVIDIA(英伟达)GTX TITAN XP 12 G(泰坦系列),核心频率最高1582 MHz,3840个CUDA核心,服务器运行内存64 G,4 T储存空间。实验模型选择Adam作为优化器,该优化器可以在训练过程中自适应地调整学习率,设置初始学习率为0.01,设置batch size为16,迭代周期为100次,选取TensorFlow深度学习框架,模型构建完成后,将9735张训练图像和2433张验证图像存储在Numpy数组中以方便实验。
-
为更为深入地分析文中改进网络模型农作物类型提取环节的具体优势与劣势,选择现今广泛应用的U-Net、Segnet和FCN算法分别进行识别效果比较。对于研究区内一处区域,借助4类技术开展提取测试,就这4类技术提取效果的精度、完整度与准确度展开统计、对比。
-
有效且合理的分类结果评价技术能够更为全面、客观地评估分类结果,有助于验证分类方法的准确性与有效性。文中选取总体精度和交并比作为实验的分类精度评价指标。
基于遥感影像在分类对象与标准上的不同,分别通过总体精度(OA)、Accuracy (准确度A)、Precision (精确度P)与Recall (召回率R)开展量化评估[16]。
基于卷积神经网络(CNN)对遥感影像各图像块的分类结果,对错误、正确分类的目标地物像素量展开统计,将OA当作遥感地物分类评估指标,当作一类常用的精度评估指标,OA代表各像素准确分类的概率,以下为计算公式:
$$ OA = \frac{1}{{{N_{{\text{total}}}}}}\sum\nolimits_{K - 1}^K {{N_{KK}}} $$ (4) 式内:NKK、Ntotal分别代表图像内像元被准确分类的数量、图像内像元的总量;将卫星遥感影像分类提供的地块对象当做基本单元,可通过Accuracy、Precision与Recall来量化评价分类效果,以下为计算公式:
$$ A = \frac{{TP + TN}}{{TP + FN + FP + TN}} $$ (5) $$ P = \frac{{TP}}{{TP + FP}} $$ (6) $$ A = \frac{{TP}}{{TP + FN}} $$ (7) 式中:TP、FP、FN、TN依次所指为准确识别的此类田块数目、错误识别的此类田块数目、此类田块被识别成其他田块的数目、准确识别的其他田块数目;均交并比(MIoU)[17],代表的是语义分割的标准度量,它能够用于分析两集合的交集和并集之比,针对语义分割问题而言,其分别为真实值(ground truth)和预测值(predicted segmentation)。这个比例可以变形为TP (交集)比上TP、FP、FN之和(并集)。在每个类上计算IoU,然后取平均值。
$$ MIoU = \dfrac{1}{{k + 1}}\sum\nolimits_{i = 0}^k {\frac{{{p_{ii}}}}{{\displaystyle\sum\nolimits_{j = 0}^k {{p_{ij}} + \displaystyle\sum\nolimits_{j = 0}^k {{p_{ji}} - {p_{ii}}} } }}} $$ (8) -
为进一步探索文中方法在农作物分类中的适用性,将文中方法与U-Net、SegNet、FCN进行识别对比实验,通过与U-Net模型进行对比来总结文中的改进想法是否成立,同时,FCN和SegNet网络是经典的图像识别网络,具有一定的代表性,与其对比,可以使文中的结果更有说服力。模型的特点如表1所示。以总体分类精度和MIoU作为评估指标。为验证改进U-Net网络的稳定性,比较各类方法的总体分类精度结果如表2所示。
表 1 不同深度学习网络模型特点
Table 1. Characteristics of network model for different deep learning
Deep learning segmentation model Model characteristics FCN For the first time, a fully convolutional network based on the end-to-end concept is proposed, which removes the fully connected layer and samples in the deconvolutional layer SegNet The pooling layer result is used in the decoding and a large amount of coding information is introduced U-Net Based on the end-to-end standard network structure, the decoder is obtained by splicing the results of each layer on the encoder, and the result is more ideal Improve U-Net The ability of semantic recognition is enhanced, and it is more sensitive to feature extraction 表 2 不同深度学习网络农作物识别方法对比实验结果
Table 2. Experimental results of crop recognition with different methods
Experimental
networkU-Net SegNet FCN Improve U-Net OA MIoU OA MIoU OA MIoU OA MIoU Precision 85.41% 0.39 84.86% 0.39 86.44% 0.45 88.33% 0.52 -
通过深度学习模型的农作物分类识别实验以及精度评价,基于模型训练得到的FCN、SegNet、U-Net和改进的U-Net模型的农作物分类识别的模型,输入测试集图像,得到农作物分类结果;运用MIoU和总体分类精度两种精度评定指标对文中研究的分类结果进行定量的精度评价;结果表明,在相同的样本库进行模型训练的场景下,从总体分类精度来看:文中改进的U-Net模型的总体分类精度达到88.83%,从交并比来看,文中改进的U-Net模型在MIoU上达到0.52,均高于传统机器学习算法,说明文中改进的U-Net模型能够有效应用于农作物分类识别场景,从分类结果可以看出:薏仁米种植面积很集中,样本采集也较多,所以在四种模型中表现出来的分类精度很高,从而可以看出不同类别的样本的数量与样本分布情况也是可能影响农作物分类精度的重要因素。改进后U-Net是基于图像语义进行分类的深度学习模型,其特征识别能力得到了一定的强化,既运用了薏仁米表现在影像上自身的特征信息,同时也结合了围绕在其周围的像素进行识别和分类,所以其准确性和可靠性更高。
文中方法与其他算法部分实验结果对比如图5所示。
Research on the classification of typical crops in remote sensing images by improved U-Net algorithm
-
摘要: 针对传统算法提取遥感图像分类特征不全,及识别农作物分类准确率不高的问题,以无人机遥感图像为数据源,提出改进U-Net模型对研究区域薏仁米、玉米等农作物进行分类识别。实验中首先对遥感影像进行预处理,并进行数据集标注与增强;其次通过加深U-Net网络结构、引入SFAM模块和ASPP模块,多级多尺度特征聚合金字塔方法等对算进行法改进,构建改进的U-Net算法,最后进行模型训练与改进修正。实验结果表明:总体分类精度OA达到88.83%,均交并比MIoU达到0.52,较传统U-Net模型、FCN模型和SegNet模,在分类指标和精度上都有明显的提升。Abstract: Aiming at the problem of incomplete classification features of remote sensing images extracted by traditional algorithms and low accuracy of crop classification, we use drone remote sensing images as the data source and propose an improved U-Net model to classify and recognize crops such as barley, corn, etc. in the study area. In the experiment, the remote sensing image is preprocessed, and the data set is labeled and enhanced. Secondly, the algorithm is improved by deepening the U-Net network structure, introducing the SFAM module and the ASPP module, and using the multi-level and multi-scale feature aggregation pyramid method to construct an improved U-Net algorithm. Finally model training and improvement are completed. The experimental results show that the overall classification accuracy OA reaches 88.83%, and the combined ratio of MIoU reaches 0.52. Compared with the traditional U-Net model, FCN model and SegNet model, the classification index and accuracy are significantly improved.
-
Key words:
- deep learning /
- crop classification /
- drone remote sensing /
- improved U-Net model
-
表 1 不同深度学习网络模型特点
Table 1. Characteristics of network model for different deep learning
Deep learning segmentation model Model characteristics FCN For the first time, a fully convolutional network based on the end-to-end concept is proposed, which removes the fully connected layer and samples in the deconvolutional layer SegNet The pooling layer result is used in the decoding and a large amount of coding information is introduced U-Net Based on the end-to-end standard network structure, the decoder is obtained by splicing the results of each layer on the encoder, and the result is more ideal Improve U-Net The ability of semantic recognition is enhanced, and it is more sensitive to feature extraction 表 2 不同深度学习网络农作物识别方法对比实验结果
Table 2. Experimental results of crop recognition with different methods
Experimental
networkU-Net SegNet FCN Improve U-Net OA MIoU OA MIoU OA MIoU OA MIoU Precision 85.41% 0.39 84.86% 0.39 86.44% 0.45 88.33% 0.52 -
[1] Liu Yanwen, Liu Chengwu, He Zongyi, et al. Delineation of basic farmland based on local spatial autocorrelation of farmland quality at pixel scale [J]. Transactions of the Chinese Society of Agricultural Machinery, 2019, 50(5): 260-268, 319. (in Chinese) [2] 朱琳. 基于Sentinel多源遥感数据的作物分类及种植面积提取研究[D]. 西北农林科技大学, 2018. Zhu Lin. Research on crop classification and planting area extraction based on Sentinel multi-source remote sensing data[D]. Xianyang: Northwest Sci-tech University of Agriculture and Forestry, 2018. (in Chinese) [3] He Zhangzheng. Remote sensing technology and its application analysis in land and space planning [J]. Real Estate, 2019(11): 44-45. (in Chinese) [4] 元晨. 高空间分辨率遥感影像分类研究[D]. 长安大学, 2016. Yuan Chen. Research on classification of high spatial resolution remote sensing images[D]. Xi'an: Chang'an University, 2016. (in Chinese) [5] Guo Jing, Jiang Jie, Cao Shixiang. Level set hierarchical segmentation of buildings in remote sensing images [J]. Infrared and Laser Engineering, 2014, 43(4): 1332-1337. (in Chinese) [6] 张诗琪. 基于深度学习的无人机遥感影像农作物分类识别[D]. 成都理工大学, 2020. Zhang Shiqi. Crop classification and recognition based on deep learning for UAV remote sensing images [D]. Chengdu: Chengdu University of Technology, 2020. (in Chinese) [7] Li Changjun, Huang He, Li Wei. Research on cultivated land extraction technology of agricultural remote sensing image based on support vector machine [J]. Instrument Technology, 2018(11): 5-8, 48. (in Chinese) [8] 顾炼. 基于深度学习的遥感图像建筑物检测及其变化检测研究[D]. 浙江工商大学, 2018. Gu Lian. Research on remote sensing image building detection and change detection based on deep learning [D]. Hangzhou: Zhejiang Gongshang University, 2018. (in Chinese) [9] Lu Heng, Fu Xiao, He Yinan, et al. Extraction of cultivated land information from UAV images based on migration learning [J]. Transactions of the Chinese Society of Agricultural Machinery, 2015, 46(12): 274-279, 284. (in Chinese) doi: 10.6041/j.issn.1000-1298.2015.12.037 [10] Zhu Xiufang, Li Shibo, Xiao Guofeng. Extraction method of film-coated farmland area and distribution based on UAV remote sensing image [J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(4): 106-113. (in Chinese) [11] 朱明. 卷积神经网络在高分辨率卫星影像地表覆盖分类中的应用研究[D]. 中国地质大学(北京), 2020. Zhu Ming. Application of convolutional neural network in classification of high-resolution satellite image land cover[D]. Beijing: China University of Geosciences (Beijing), 2020. (in Chinese) [12] Yang Nan, Nan Lin, Zhang Dingyi, et al. Research on image description based on deep learning [J]. Infrared and Laser Engineering, 2018, 47(2): 0203002. (in Chinese) doi: 10.3788/IRLA201847.0203002 [13] 张俊. 基于图像特征和深度学习的奶牛身份识别方法的研究[D]. 吉林农业大学, 2020. Zhang Jun. Research on dairy cow identification method based on image features and deep learning [D]. Changchun: Jilin Agricultural University, 2020. (in Chinese) [14] Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184 [15] Liu Wenxiang, Shu Yuanzhong, Tang Xiaomin, et al. Semantic segmentation of remote sensing images using dual attention mechanism Deeplabv3+ algorithm [J]. Tropical Geography, 2020, 40(2): 303-313. (in Chinese) [16] Sun Yu, Han Jingye, Chen Zhibo, et al. Aerial monitoring method of greenhouse and mulching farmland by drone based on deep learning [J]. Transactions of the Chinese Society of Agricultural Machinery, 2018, 49(2): 133-140. (in Chinese) [17] Bingwen Qiu, Weijiao Li, Zhenghong Tang, et al. Mapping paddy rice areas based on vegetation phenology and surface moisture conditions [J]. Ecological Indicators, 2015, 56: 79-86. doi: https://doi.org/10.1016/j.ecolind.2015.03.039