Crop Insect Pest Detection based on Dilated Multi-scale Attention U-Net

Crop Insect Pest Detection based on Dilated Multi-scale Attention U-Net | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Crop Insect Pest Detection based on Dilated Multi-scale Attention U-Net Xuqi Wang, Shanwen Zhang, Ting Zhang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3799461/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 26 Feb, 2024 Read the published version in Plant Methods → Version 1 posted 9 You are reading this latest preprint version Abstract Early detection and identification of insect pests is the premise and basis of scientific control and accurate utilization of Insect pesticides. Aiming at the problems of low detection accuracy and slow training speed of the existing crop Insect pest detection models, a dilated multi-scale attention U-Net (DMSAU-Net) model is constructed for crop Insect pest detection. In its encoder, dilated Inception is designed to replace the convolution layer in U-Net to extract the multi-scale features of insect pest images and improve the accuracy of the model. An attention module is added to its decoder to focus on the edge of the insect pest image and reduce the upsampling noise and accelerate model convergence. The results on the crop insect pest image dataset verify that the proposed method has high segmentation accuracy and good generalization ability, and can be applied to practical crop insect pest monitoring system. insect pest detection U-Net Dilated Inception. Multi-scale convolution Dilated multi-scale attention U-Net (DMSAU-Net) Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 1. Introduction Crop insect pests seriously reduce the yield and quality of crops. Crop insect pest detection is the premise and foundation of crop insect pest identification and control [ 1 – 3 ]. However, due to the irregular shapes, postures, colors and appearances of insect pests in the field, insect pest detection is always a difficult and challenging problem [ 4 , 5 ]. insect pest detection is to segment the insect pest image from the background, which is described as the problem of insect pest image semantic segmentation. With development of computer, big data and artificial intelligence technologies, many insect pest detection methods have been presented [ 6 ]. Clustering is a powerful technique for insect pest detection. Pratheba et al. [ 7 ] introduced some insect pest detection methods using clustering algorithms, such as K-means and Fuzzy c-means (FCM) algorithms. Experimental results show that FCM can better segment images and provide flexibility for different types of pixels. Galphat et al. [ 8 ] comprehensively reviewed and analyzed the algorithms and technologies of insect pest detection in the agricultural field. Wang et al.[ 9 ] summarized the method of automatic insect pest detection and recognition in the field by integrating multi-scale context-aware feature representation. From the above methods, it can be seen that their results depend on the extracted manual features. In fact, it is difficult to extract the optimal robust features from each insect pest image due to the irregular scale, shape, pose, position, illumination and background of insect pests in the field, especially the variety of images taken by the same insect pest in the field. Some examples are shown in Fig. 1. Convolutional neural network (CNN) and its variants, such as VGGNet, AlexNet, ResNet, GoogLeNet, MobileNet and DenseNet, have been widely applied to image detection, recognition and semantic segmentation, and achieved outstanding results under natural conditions [ 10 , 11 ]. The fully connected layers of CNN have many parameters, which leads to a large amount of redundancy and slow network training. Fully convolutional network (FCN) is an end-to-end image segmentation method modified from VggNet. It realizes pixel-level classification, and overcomes the shortcomings of CNN in image segmentation through deconvolution and upsampling operations [ 12 , 13 ]. The main difference between FCN and CNN is that FCN replaces the fully-connected layers at the end of CNN with a convolutional layer, which can reduce the computational cost[ 14 ]. FCN can input images of arbitrary size and output images, but the segmentation effect of image details is not good enough for insect pest detection [ 15 ]. U-Net is a relatively simple, fast and accurate image symmetric semantic segmentation network [ 16 ]. It is an encoder-decoder network commonly used in image segmentation. Its many variants have been proposed and successfully applied to insect pest detection and recognition [ 17 ]. Yang et al.[ 18 ] proposed a modified U-Net with dilated convolution (DCU-Net) for brain tumor segmentation based on the classic U-Net. In the model, the multi-scale spatial pyramid pooling is used to replace the max pooling at the end of the down-sampling path, which can expand the feature receptive field while maintaining image resolution. The advantages of residual network (ResNet) and U-Net are used to simplify the deep training network, promote information transmission and reduce parameter transmission. The combination of ResNet and U-Net can improve the accuracy of image detection and network training[ 19 ]. Dilated convolution can enlarge the receptive filed and improve the target detection ability of the network [ 20 ]. Several modified U-Nets did not consider the influence of U-Net combined with dilated convolution module on the result of feature extraction. Dilated ResNet can improve fine segmentation and detection results [ 21 ]. Inception module can capture multi-scale context features by using multiple convolutional kernels of different sizes. It can not only capture long time-frequency context information of features, but also exploit information from multiple layers of CNN. Attention mechanism can help the network model locate the focus area, extract more useful features, and achieve high precision fusion. Attention based CNN has higher classification accuracy and significantly reduces the number of particles misclassified, which reflects the focusing effect of attention mechanism [ 22 , 23 ]. Chen et al. [ 24 ] summarized semantic image segmentation with deep convolutional nets, dilated convolution, and fully connected Conditional Random Field (CRF), highlighted the convolution with up-sampled filters and dilated convolution, and proposed dilated spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. To improve the detection accuracy of field insect pests, an improved U-Net, namely the expanded multi-scale Attention U-Net (DMSAUN-Net), is constructed by using the advantages of ResNet, dilated convolution and Inception module. The main contributions of this paper are summarized as follows: 1. Dilated Inception module with various dilation ratios is introduced to extract the multi-scale contextual features. 2. Spatial attention mechanism is added to the skip connection layers of U-Net, which can focus the attention on the edge of the insect pest and reduce the noise and computational cost. 3. The computational cost is further reduced by introducing ResNet into the skip connection layer of U-Net. The rest of this paper is organized as follows. The related works are summarized in Section 2. The proposed DMSAUN-Net based insect pest detection is illustrated in Section 3. The detail experimental analysis and comparison is provided in Section 4, and Section 5 summarizes this paper and points out the future work in. 2. Related works 2.1 U-Net U-Net consists of encoding part, decoding part and jump connection without fully-connected layers. Its architecture is shown in Fig. 2 . Encoding part follows the typical construction of convolutional networks, including two repeated applications of convolution, and one activation unit (ReLU) and one pooling operation are connected after convolution for down-sampling. The decoding part includes up-sampling of the feature map, and superposition of two convolution parts. Skip connection is used to concatenated encoding part with the corresponding feature map of the encoding part. Each convolution is basically symmetric and formed by ReLU to activate the encoding part and decoding part and form a U-shaped structure. In the down-sampling process, the number of feature channels is doubled while is shrunk after deconvolution operation in up-sampling. 2.2 Inception module Inception is a multi-branch convolution module. It can extract the multi-scale features from the input images by different convolutional kernels. Its structure is shown in Fig. 3 , including 1×1, 3×3 and 5×5 convolutional kernels, where 1×1 convolution operation is used to reduce the amount of calculation. To make the feature map have the same size, each branch adopts the same padding mode, and the stride is 1. 2.3 Dilated convolution In U-Net, the receptive field is increased by pooling and the image size is reduced, and then the image size is restored by up-sampling. This process may cause the accuracy loss. To reduce this loss, it is natural to remove the pooling layer, but as a result, the receptive field of the feature map is too small. Dilated convolution is to solve this problem by increasing the receptive field. It is to insert the value of 0 between the ordinary convolution kernels to increase the expansion coefficient of the convolution kernel, so as to increase the receptive field, reduce the computation amount and retain the nodal information, as shown in Fig. 4 with 4 dilated rates. It is seen from Fig. 4, the size of receptive field increases with the dilated rate, but the network parameters do not increase, that is, 9 parameters. 3. Dilated Multi-scale attention U-Net (DMSAU-Net) Due to the small dataset and the easy influence of complex background such as illumination and clutter, as shown in Fig. 1, the detection accuracy of crop insect pest is low, which is over-segmentation or under-segmentation. In this Section, an improved U-Net model namely dilated multi-scale attention U-Net (DMSAU-Net) is constructed for insect pest image detection. Its overall structure is shown in Fig. 5 . The numbers shown below each dilated Inception module indicate the total number of kernel used, height, width and depth of the output feature maps. 3.1 Detail of DMSAU-Net Similar to U-Net, DMSAU-Net mainly consists of the encoding part, decoding part and skip connection with attention mechanism. Encoding part is a multi-scale convolutional network, including dilated Inception module (convolution kernel of 3×3 kernel), the pooling layer (2×2 maximum pooling), and the activation function (ReLU). In encoding part, the multi-scale and multi-level features are extracted from images through dilated Inception module, and then the extracted features are downsampled to realize the correlation of multiple channels and the full decoupling of image feature space. In decoding part, the extracted features are restored by up-sampling (deconvolution of 2×2), skip connection with attention, dilated Inception module (convolution kernel of 3×3) and activation function (ReLU) classification. Finally, the binary insect pest image of the insect pest and the background is obtained by 1×1 convolution layer and Sigmoid activation function. Considering the possible mesh effect caused by deconvolution, the bilinear interpolation method in upsampling is used to restore the image, during which 1×1 convolution is used to restore the number of channels. During the up-sampling process, the feature maps corresponding to the same resolution of down-sampling are concatenated. After each concatenation, the feature maps are further refined through dilated multi-scale module, and the up-sampling is performed successively until the features extracted from the encoder are restored to the size of the input maps. Skip connection with attention is to concatenate the convolutional features and the deconvolution features. 3.2 Dilated Inception module Inspired by Inception module, multi-scale concatenation module and dilated concatenation module, a dilated multi-scale module is constructed as shown in Fig. 6 , consisting of 3 dilated convolutional kernels, 3 1×1 convolution kernels and a concatenation layer. It is a modified Inception module, which aims to extract multi-scale classification features, from low-level structural features to high-level semantic features, by increasing receptive field without increasing the training parameters. 3.3 Spatial attention module Dilated Inception module obtains multi-scale features by encoding part and decoding part, and the extracted multi-scale features are concatenated by skip connection to achieve more accurate details and location information of insect pest. However, the max-pooling and upsampling operation in the encoding and decoding parts will lose part of the location space and other information, resulting in inaccurate segmentation of insect pest in the field. To overcome this problem, the spatial attention mechanism module is added to the skip connection. By directly cascading the features of the encoding layer and the corresponding deconvolution layer, the attention module fuses their complementary features, suppressed the noise generated by upsampling, and enhanced the robustness of the model [ 25 ]. The structure of spatial attention module is shown in Fig. 7 . 3.4 Model training and evaluation The input insect pest images and their corresponding labeled images are used to train DMSAU-Net. Five - fold cross-validation (5-fold CV) scheme and stochastic gradient descent (SGD) with an adaptive moment estimator (Adam) are often used to train all models [ 26 ]. The purpose of crop insect pest detection is to determine the category of each pixel in the image, so as to clarify the scope of insect pests. Accuracy and Intersection over Union (IoU) are selected as indexes to evaluate the segmentation performance of the proposed algorithm, calculated as follows: $$ACC=\frac{{TP}}{{TP+FN}},\;\;\;IoU=\frac{{TP}}{{TP+FP+FN}}$$ 1 where TP , FP and FN are the numbers of true positives, false positives, and false negatives of the class, respectively. 4. Experiments and analysis To validate the proposed DMSAU-Net based insect pest detection method, the insect pest detection experiments are conducted on the crop common insect pest image dataset, and compared with U-Net [ 16 ] and its two improved models: U-Net with dilated convolution (DCU-Net) [ 18 ] and ResNet with U-Net (ResU-Net)[ 19 ]. Batch size = 32 for rice data subset of IP102 to reduce computation time. Number of iterations = 3000, global learning rate = 0.001, gradient decay factor = 0.9, squared gradient decay factor = 0.999, loss function = cross entropy. All models are tested on Keras and trained on a Intel Xeon E5-2643v3 @3.40GHz CPU, GTX2080Ti 11GB GPU, 64GB RAM, Windows 7 64bit, CUDA Toolkit10.0, CUDNN V7.6.5, Python 3.7 and Tensorflow-GPU 1.8.0. 4.1 insect pest image dataset IP102 ( https://github.com/xpwu95/IP102 ) is a public insect pest image dataset, containing more than 75,000 images belonging to 102 insect pest categories that exhibit a natural long-tailed distribution [ 27 ]. 19,000 of these images have be professionally annotated. There are 8415 insect pest images in the dataset belonging to 14 rice insect pest categories, as shown in Fig. 8 and Table 1 . From Table 1 , it is seen that the classes of rice insect pests are highly unbalanced, ranging in sample size between 173 and 1115. In the experiments on this data subset, five-fold-cross validation scheme is adopted to perform experiments. That is, the dataset is randomly split into 5 mutually exclusive subsets of equal or near equal size. The model is performed 5 times subsequently, where each time using 4 of the 5 splits as the training set to train the model, and the 1 of the 5 splits as the test set to evaluate the performance of the model. To verify the robustness of the proposed method, images under different conditions such as strong illumination and complex background are selected, and the same insect pest in the dataset contained different insect states. To improve the recognition accuracy, Photoshop is used to uniformly adjust the image to 128×128 pixels. Table 1 rice insect pest image dataset No. insect pest name Amount Number 1 rice leaf roller 1115 0-1115 2 3 4 5 6 7 8 9 10 11 12 13 14 rice leaf caterpillar paddy stem maggot asiatic rice borer yellow rice borer rice gall midge Rice Stemfly brown plant hopper white backed plant hopper small brown plant hopper rice water weevil rice leafhopper grain spreader thrips rice shell insect pest 485 261 1053 504 506 369 838 889 553 856 404 173 409 1116–1601 1602–1862 1863–2915 2916–3419 3420–3925 3926–4294 4293–5132 5133–6021 6022–6574 6575–7430 7431–7834 7835–8007 8008–8416 4.2 Experimental results To test the effectiveness of spatial attention module, Fig. 9 show the convolutional feature maps of the first dilated multi-scale module and the corresponding feature maps after spatial attention module. From Fig. 9, it is obvious that the convolutional feature maps after spatial attention module are more significant than that of the first dilated multi-scale module. DMSAU-Net is a modified U-Net. Figure 10A shows the loss values versus the number of iterations of U-Net and DMSAU-Net on the training set. From Fig. 10A, it is found that DMSAU-Net converges much better than U-Net. To further analyze the training performance, the same experimental parameters are used to train the different networks to ensure the reliability of the comparison results. The detection results are shown in Fig. 10B. From Fig. 10B, it can be seen that as the number of iterations increases, the detection loss of the three models decreases, and the detection performance of DMSAU-Net outperformances other models. Before 1000 iterations, their losses decrease quickly, go down very slowly and then level off. From Fig. 10, it can be seen that their training processes are relatively stable after 2000 iterations, DCU-Net and DMSAU-Net are better than ResU-Net, and DMSAU-Net is better than DCU-Net. This owes to the dilated multi-scale convolution and spatial attention mechanism. To test the detection performance of DMSAU-Net, we randomly select 4 rice insect pest images from the IP102 dataset and visualize the detection results, as shown in Fig. 11. To reflect the advantages of deep learning methods, we compare DMSAU-Net with the traditional insect pest detection algorithm. Fuzzy c-means (FCM) is one of the most widely used for insect pest detection [ 28 ]. The detection results are also in Fig. 11. As can be seen from Fig. 11, the four improved U-Net models are much better than FCM. They can effectively detect rice insect pests under complex background, and the position and shape of insect pests are good, among which DCU-Net is better than ResU-Net, and DMSAUN-Net has the best detection effect with more accurate insect pest shape and edge. To quantitatively estimate the detection performance of DMSAU-Net, 5-fold cross validation experiments are conducted on the rice insect pest image subset of IP102. As can be seen from Fig. 10, all models basically converge at the 3000th iteration. For fair, the trained models are selected at the 3000th iteration. The detection results of U-Net, DCU-Net, ResU-Net and DMSAU-Net are listed in Table 2 . Table 2 The detection results of rice insect pests by 4 insect pest detection algorithms Method Results U-Net DCU-Net ResU-Net DMSAU-Net Accuracy (%) 0.842 0.907 0.884 0.921 IoU (%) 0.814 0.892 0.865 0.917 Training time (h) 15.6 9.5 8.3 7.7 Test time (ms) 51 45 38 31 DMSAU-Net is constructed by making use of the advantage of U-Net, dilated convolution, multi-scale convolution and spatial attention mechanism. To further verify the superiority of DMSAU-Net, some ablation experiments are carried out under similar conditions. The different experimental set and rice insect pest results are shown in Table 3 , where attention is added to skip connection of U-Net. Table 3 The different experimental set and result Results Experimental set Accuracy IoU Training time (h) U-Net 0.842 0.814 15.6 U-Net + spatial attention 0.856 0.843 13.2 U-Net + Inception 0.872 0.864 13.7 U-Net + dilated multi-scale 0.881 0.875 11.0 U-Net + Inception + spatial attention 0.890 0.879 10.7 U-Net + dilated multi-scale + channel-spatial attention 0.912 0.908 8.6 Table 3 indicates that spatial attention, channel-spatial attention, Inception, dilated multi-scale can contribute to the results to some extent. 4.3 Analysis From Figs. 10 and 11 and Tables 2 and 3 , it can be seen that DMSAU-Net has best detection performance, the highest detection rate and the least training and detecting time, due to dilated convolution, multi-scale convolution and spatial attention mechanism. With the aid of dilated multi-scale, DMSAU-Net can extract the multi-scale classification features. With the aid of skip connection combined with spatial attention mechanism, DMSAU-Net can enhance the constraint on the feature maps, focus more attention to the insect pest image region, and speed up the training. That is to reduce the learning of non-important areas and enhance the learning of insect pest areas, so as to improve the detection ability of insect pest characteristics and improve the detection accuracy rate. The detection rate of U-Net is the poorest, because it is difficult to extract the robust classification features from the various insect pest images with very complex background. 5. Conclusions In modern agricultural field, insect pest detection plays an important role in timely and accurate diagnosis of crop insect pest. But it is difficult to detect crop insect pest in the field due to the various-shape-size insect pests with complex background. To solve such problem, a dilated multi-scale attention U-Net (DMSAU-Net) model is constructed for crop insect pest detection by making use of the advantage of multi-scale convolution and attention mechanism. In dilated inception module, multi-scale convolution kernels without increasing training parameters are used to extract the distributed characteristics of insect pests at different scales and to perform cascade fusion. The experiments are carried out on the rice insect pest image subset of IP102 dataset. The detection results show that DMSAU-Net is effective and feasible for crop insect pest detection in the field. This research can be used to realize the automation degree of insect pest management in agricultural field. Future work is to optimize the model to organically integrate it into an effective insect pest detection system. Declarations Conflict of Interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Ethical Approval no applicable. Funding This study was supported in part by grants from the National Natural Science Foundation of China (Nos. 62172338) and 2022 General Special Research Plan Project of Shaanxi Provincial Department of Education (No.: 22JK0596). Availability of data and materials The data that support the findings of this study are openly available in https://github.com/xpwu95/IP102. References Dattatraya VS, Sudhir KS, Ghanshyam C et al. Low cost sensor based embedded system for plant protection and pest control. International Conference on Soft Computing Techniques & Implementations, 2015. 10.1109/ICSCTI.2015.7489628 . Bhujel A, Mahonar S, Choubey M, et al. Pest and Diseases Management in Darjeeling Tea. Social Sci Electron Publishing. 2016;6(3):469–72. Xie C, Zhang J, Li R, et al. Automatic classification for field crop insects via multiple-task sparse representation and multiple-kernel learning. Computers & Electronics in Agriculture. 2015;119:123–32. Selvaraj MG, Vergara A, Ruiz H, et al. AI-powered banana diseases and pest detection. Plant Methods. 2019;15(1). 10.1186/s13007-019-0475-z . Thenmozhi K, Reddy US. Crop pest classification based on deep convolutional neural network and transfer learning. Computers and Electronics in Agriculture,2019,164,104906. 10.1016/j.compag.2019.104906 . Teng Yue Z, Jie D, Shifeng, et al. MSR-RCNN: A Multi-Class Crop Pest Detection Network Based on a Multi-Scale Super-Resolution Feature Enhancement Module. Frontiers. 2022;13. 10.3389/fpls.2022.810546 . Pratheba R, Sivasangari A, Saraswady D et al. Performance analysis of pest detection for agricultural field using clustering techniques. International Conference on Circuits, Power and Computing Technologies, 2014. 10.1109/ICCPCT.2014.7054833 . Galphat Y, Patange VR, Talreja P et al. Survey and Analysis of Pest Detection in Agricultural Field. International Conference on Computer Networks, Big Data and IOT, 2018: 976–983. 10.1007/978-3-030-24643-3_116 . Wang F, Wang R, Xie C, et al. Fusing multi-scale context-aware feature representation for automatic in-field pest detection and recognition. Comput Electron Agric. 2020;169:105222. Rikiya Y, Mizuho N, Gian D et al. Convolutional neural networks: an overview and application in radiology. Insights into Imaging, 2018:1–19. 10.1007/s13244-018-0639-9 . Jafari R, Razvarz S, Gegov A et al. Deep Learning for Pipeline Damage Detection: an Overview of the Concepts and a Survey of the State-of-the-Art. 10th IEEE International Conference on Intelligent Systems IS’20, 2020. 10.1109/IS48319.2020.9200137 . Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and, Recognition P. (CVPR), Boston, MA, USA, 2015:3431–3440. Roth HR, Shen C, Oda H et al. A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018, 11073, 417–425. 10.1007/978-3-030-00937-3_48 . Badrinarayanan V, Kendall A, Cipolla R, Segnet. A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39:2481–95. Li R, Liu W, Yang L, et al. DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation. IEEE J Sel Top Appl Earth Observations Remote Sens. 2018;11(11):3954–62. Sivagami S, Chitra P, Kailash G et al. U-Net Architecture Based Dental Panoramic Image Segmentation. International Conference on Wireless Communications Signal Processing and Networking, 2020: 187–191. Zhang ZX, Liu QJ, Wang YH. Road extraction by deep residual U-Net. IEEE Geoscience and Remote Sensing Letters 2018,15(5):749–53. Yang T, Zhou Y, Li L, et al. DCU-Net: Multi-scale U-Net for brain tumor segmentation. J X-Ray Sci Technol. 2020;28(4):709–26. Najeeb RS, Dahl IO. Brain Tumor Segmentation Utilizing Generative Adversarial, Resnet and U-net Deep Learning. 8th International Conference on Contemporary Information Technology and Mathematics, Mosul, Iraq, 2022:85–89. 10.1109/ICCITM56309.2022.10031760 . Shi B, Su Y, Lian C et al. Obstacle type recognition in visual images via dilated convolutional neural network for unmanned surface vehicles. J Navig, 202275(2): 437–54. Wu Y, Guo C, Gao H, et al. Dilated residual networks with multi-level attention for speaker verification. Neurocomputing. 2020;412(4):177–86. Wang H, Yang J, Wang R, et al. Remaining Useful Life Prediction of Bearings Based on Convolution Attention Mechanism and Temporal Convolution Network. IEEE Access. 2023;11:24407–19. 10.1109/ACCESS.2023.3255891 . Wei H, Zhang Q, Gu Y. Remaining Useful Life Prediction of Bearings Based on Self-attention Mechanism, Multi-scale Dilated Causal Convolution, and Temporal Convolution Network. Meas Sci Technol. 2023;34(4):045107. 10.1088/1361-6501/acb0e9 . Chen LC, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Dilated Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell. 2018;40(4):834–48. He H, Yang DF, Wang SC, et al. Road extraction by using atrous spatial pyramid pooling integrated encoder-decoder network and structural similarity loss. Remote Sens. 2019;11(9):1015. Liu M, Yao D, Liu Z, et al. An Improved Adam Optimization Algorithm Combining Adaptive Coefficients and Composite Gradients Based on Randomized Block Coordinate Descent. Comput Intell Neurosci. 2023;10(4765891). 10.1155/2023/4765891 . Wu X, Zhan C, Lai YK et al. IP102: a large-scale benchmark dataset for insect pest recognition. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),2019: 8779–8788. Pratheba R, Sivasangari A, Saraswady D. Performance analysis of pest detection for agricultural field using clustering techniques. International Conference on Circuits, Power and Computing Technologies, 2014, 14968106. 10.1109/ICCPCT.2014.7054833 . Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 26 Feb, 2024 Read the published version in Plant Methods → Version 1 posted Editorial decision: Revision requested 29 Jan, 2024 Reviews received at journal 11 Jan, 2024 Reviewers agreed at journal 03 Jan, 2024 Reviewers agreed at journal 29 Dec, 2023 Reviewers agreed at journal 28 Dec, 2023 Reviewers invited by journal 28 Dec, 2023 Editor assigned by journal 28 Dec, 2023 Submission checks completed at journal 28 Dec, 2023 First submitted to journal 24 Dec, 2023 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3799461","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":264647979,"identity":"f7efed44-c023-4df7-a872-133cf1d78792","order_by":0,"name":"Xuqi Wang","email":"","orcid":"","institution":"Xijing University","correspondingAuthor":false,"prefix":"","firstName":"Xuqi","middleName":"","lastName":"Wang","suffix":""},{"id":264647980,"identity":"096b50ba-70fb-4e39-9c8a-12b350ebd69b","order_by":1,"name":"Shanwen Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1ElEQVRIiWNgGAWjYBACNvbmAwcSDGzkGNsbGx8kVNQQ1sLHcyzxwIOCNGPmnsPNBg/OHCOsRU4ix/jggw+HEtlnpLdJPmxhJsJhPMcSgA47kMDbkNhWkdjAxsDf3p1AjF/u5Ek2HGy7kbhDhkHizNkNxNjyrNiwsRGo5Qwbg4FELgEtEjlAVxkcTtx/mLGtILGNmQQtjW2MbQzEaYE4LM2YsYexWSLhzDEegn6Rb28+/PHHH2BUzn/+8OOPiho5/vZe/FowAA9pykfBKBgFo2AUYAUADpNWBZvmLRkAAAAASUVORK5CYII=","orcid":"","institution":"Xijing University","correspondingAuthor":true,"prefix":"","firstName":"Shanwen","middleName":"","lastName":"Zhang","suffix":""},{"id":264647981,"identity":"9b599537-f52d-4863-b17f-cbe67c94bfb7","order_by":2,"name":"Ting Zhang","email":"","orcid":"","institution":"Xijing University","correspondingAuthor":false,"prefix":"","firstName":"Ting","middleName":"","lastName":"Zhang","suffix":""}],"badges":[],"createdAt":"2023-12-24 07:59:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3799461/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3799461/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13007-024-01163-w","type":"published","date":"2024-02-26T15:00:34+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":49091259,"identity":"16efd03c-7c08-429c-b58e-78625530ec95","added_by":"auto","created_at":"2024-01-03 01:52:09","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":103993,"visible":true,"origin":"","legend":"\u003cp\u003eInsect pests in the field with various shapes, poses, sizes, colors, illumination and background\u003c/p\u003e","description":"","filename":"F1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/2b40775fe7b5164245500080.jpg"},{"id":49090518,"identity":"ba9a96bc-c98a-4d4d-95be-fac05780f209","added_by":"auto","created_at":"2024-01-03 01:44:09","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":41308,"visible":true,"origin":"","legend":"\u003cp\u003eU-Net architecture\u003c/p\u003e","description":"","filename":"F2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/267c36fedd79d231a1433a9a.jpg"},{"id":49090520,"identity":"830f6f74-3f30-4def-a921-5fb61c2f777d","added_by":"auto","created_at":"2024-01-03 01:44:09","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":31633,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure of Inception\u003c/p\u003e","description":"","filename":"F3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/386c6289e7d1d77c5c9acfdc.jpg"},{"id":49091260,"identity":"3cf577e8-4de7-458c-8650-ef2fd6f8a394","added_by":"auto","created_at":"2024-01-03 01:52:09","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":36977,"visible":true,"origin":"","legend":"\u003cp\u003eDilated convolution with 4 dilation rates\u003c/p\u003e","description":"","filename":"F4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/b51afd4200aacc76923f44a2.jpg"},{"id":49091261,"identity":"8137b832-9136-4c31-93bd-22e81fabb011","added_by":"auto","created_at":"2024-01-03 01:52:09","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":92678,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure of DMSAU-Net\u003c/p\u003e","description":"","filename":"F5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/9bb3af93a30b1acf1e77fc8b.jpg"},{"id":49090528,"identity":"9d943343-c63f-49ce-9761-549df607c435","added_by":"auto","created_at":"2024-01-03 01:44:09","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":38197,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure of dilated multi-scale module\u003c/p\u003e","description":"","filename":"F6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/6894b5990a970798820cc038.jpg"},{"id":49092031,"identity":"21a4b003-dc0b-4104-b8c0-17d5e63a4b65","added_by":"auto","created_at":"2024-01-03 02:00:09","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":40090,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure of spatial attention module\u003c/p\u003e","description":"","filename":"F7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/a4deb7fa0065d2f761c28655.jpg"},{"id":49092700,"identity":"4514b424-7f3e-40a4-b734-1cff3136857c","added_by":"auto","created_at":"2024-01-03 02:08:09","extension":"jpg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":65148,"visible":true,"origin":"","legend":"\u003cp\u003e14 original insect pest images, one per species\u003c/p\u003e","description":"","filename":"F8.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/6df7f000bb5f775f587b4189.jpg"},{"id":49091263,"identity":"a9f7edc0-7f46-4ce0-9f40-db8034a541d0","added_by":"auto","created_at":"2024-01-03 01:52:09","extension":"jpg","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":45832,"visible":true,"origin":"","legend":"\u003cp\u003eConvolutional feature maps\u003c/p\u003e","description":"","filename":"F9.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/9369a5083ca1fc45796e3886.jpg"},{"id":49090522,"identity":"017fb60c-4724-4537-989e-9b57f2b4712f","added_by":"auto","created_at":"2024-01-03 01:44:09","extension":"jpg","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":113318,"visible":true,"origin":"","legend":"\u003cp\u003eLoss versus iterations\u003c/p\u003e","description":"","filename":"F10.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/d5831bd1c7e1c200cb40a7c2.jpg"},{"id":49090527,"identity":"0307290b-f806-4fb7-acfe-cf402f0cd04f","added_by":"auto","created_at":"2024-01-03 01:44:09","extension":"jpg","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":69729,"visible":true,"origin":"","legend":"\u003cp\u003eRice insect pest detection of IP102\u003c/p\u003e","description":"","filename":"F11.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/2eb6042831e48374c5d0dc19.jpg"},{"id":51958003,"identity":"d5d90be2-f74a-4c67-8cfa-214a960201da","added_by":"auto","created_at":"2024-03-04 15:08:45","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":873620,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3799461/v1/4c089375-1dd3-4350-9b61-0c136c5450bd.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Crop Insect Pest Detection based on Dilated Multi-scale Attention U-Net","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eCrop insect pests seriously reduce the yield and quality of crops. Crop insect pest detection is the premise and foundation of crop insect pest identification and control [\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e]. However, due to the irregular shapes, postures, colors and appearances of insect pests in the field, insect pest detection is always a difficult and challenging problem [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]. insect pest detection is to segment the insect pest image from the background, which is described as the problem of insect pest image semantic segmentation. With development of computer, big data and artificial intelligence technologies, many insect pest detection methods have been presented [\u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e]. Clustering is a powerful technique for insect pest detection. Pratheba et al. [\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e] introduced some insect pest detection methods using clustering algorithms, such as K-means and Fuzzy c-means (FCM) algorithms. Experimental results show that FCM can better segment images and provide flexibility for different types of pixels. Galphat et al. [\u003cspan class=\"CitationRef\"\u003e8\u003c/span\u003e] comprehensively reviewed and analyzed the algorithms and technologies of insect pest detection in the agricultural field. Wang et al.[\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e] summarized the method of automatic insect pest detection and recognition in the field by integrating multi-scale context-aware feature representation.\u003c/p\u003e\n\u003cp\u003eFrom the above methods, it can be seen that their results depend on the extracted manual features. In fact, it is difficult to extract the optimal robust features from each insect pest image due to the irregular scale, shape, pose, position, illumination and background of insect pests in the field, especially the variety of images taken by the same insect pest in the field. Some examples are shown in Fig. 1.\u003c/p\u003e\n\u003cp\u003eConvolutional neural network (CNN) and its variants, such as VGGNet, AlexNet, ResNet, GoogLeNet, MobileNet and DenseNet, have been widely applied to image detection, recognition and semantic segmentation, and achieved outstanding results under natural conditions [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e]. The fully connected layers of CNN have many parameters, which leads to a large amount of redundancy and slow network training. Fully convolutional network (FCN) is an end-to-end image segmentation method modified from VggNet. It realizes pixel-level classification, and overcomes the shortcomings of CNN in image segmentation through deconvolution and upsampling operations [\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e]. The main difference between FCN and CNN is that FCN replaces the fully-connected layers at the end of CNN with a convolutional layer, which can reduce the computational cost[\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e]. FCN can input images of arbitrary size and output images, but the segmentation effect of image details is not good enough for insect pest detection [\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e]. U-Net is a relatively simple, fast and accurate image symmetric semantic segmentation network [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e]. It is an encoder-decoder network commonly used in image segmentation. Its many variants have been proposed and successfully applied to insect pest detection and recognition [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e]. Yang et al.[\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e] proposed a modified U-Net with dilated convolution (DCU-Net) for brain tumor segmentation based on the classic U-Net. In the model, the multi-scale spatial pyramid pooling is used to replace the max pooling at the end of the down-sampling path, which can expand the feature receptive field while maintaining image resolution. The advantages of residual network (ResNet) and U-Net are used to simplify the deep training network, promote information transmission and reduce parameter transmission. The combination of ResNet and U-Net can improve the accuracy of image detection and network training[\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. Dilated convolution can enlarge the receptive filed and improve the target detection ability of the network [\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e]. Several modified U-Nets did not consider the influence of U-Net combined with dilated convolution module on the result of feature extraction. Dilated ResNet can improve fine segmentation and detection results [\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e]. Inception module can capture multi-scale context features by using multiple convolutional kernels of different sizes. It can not only capture long time-frequency context information of features, but also exploit information from multiple layers of CNN. Attention mechanism can help the network model locate the focus area, extract more useful features, and achieve high precision fusion. Attention based CNN has higher classification accuracy and significantly reduces the number of particles misclassified, which reflects the focusing effect of attention mechanism [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. Chen et al. [\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e] summarized semantic image segmentation with deep convolutional nets, dilated convolution, and fully connected Conditional Random Field (CRF), highlighted the convolution with up-sampled filters and dilated convolution, and proposed dilated spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales.\u003c/p\u003e\n\u003cp\u003eTo improve the detection accuracy of field insect pests, an improved U-Net, namely the expanded multi-scale Attention U-Net (DMSAUN-Net), is constructed by using the advantages of ResNet, dilated convolution and Inception module. The main contributions of this paper are summarized as follows:\u003c/p\u003e\n\u003cp\u003e\u003cspan\u003e\u003c/span\u003e\u003c/p\u003e\n\u003cp\u003e1. Dilated Inception module with various dilation ratios is introduced to extract the multi-scale contextual features.\u003c/p\u003e\u003cspan\u003e\n \u003cp\u003e2. Spatial attention mechanism is added to the skip connection layers of U-Net, which can focus the attention on the edge of the insect pest and reduce the noise and computational cost.\u003c/p\u003e\n\u003c/span\u003e\u003cspan\u003e\n \u003cp\u003e3. The computational cost is further reduced by introducing ResNet into the skip connection layer of U-Net.\u003c/p\u003e\n\u003c/span\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003cp\u003eThe rest of this paper is organized as follows. The related works are summarized in Section 2. The proposed DMSAUN-Net based insect pest detection is illustrated in Section 3. The detail experimental analysis and comparison is provided in Section 4, and Section 5 summarizes this paper and points out the future work in.\u003c/p\u003e"},{"header":"2. Related works","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n \u003ch2\u003e2.1 U-Net\u003c/h2\u003e\n \u003cp\u003eU-Net consists of encoding part, decoding part and jump connection without fully-connected layers. Its architecture is shown in Fig. \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e. Encoding part follows the typical construction of convolutional networks, including two repeated applications of convolution, and one activation unit (ReLU) and one pooling operation are connected after convolution for down-sampling. The decoding part includes up-sampling of the feature map, and superposition of two convolution parts. Skip connection is used to concatenated encoding part with the corresponding feature map of the encoding part. Each convolution is basically symmetric and formed by ReLU to activate the encoding part and decoding part and form a U-shaped structure. In the down-sampling process, the number of feature channels is doubled while is shrunk after deconvolution operation in up-sampling.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n \u003ch2\u003e2.2 Inception module\u003c/h2\u003e\n \u003cp\u003eInception is a multi-branch convolution module. It can extract the multi-scale features from the input images by different convolutional kernels. Its structure is shown in Fig. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, including 1\u0026times;1, 3\u0026times;3 and 5\u0026times;5 convolutional kernels, where 1\u0026times;1 convolution operation is used to reduce the amount of calculation. To make the feature map have the same size, each branch adopts the same padding mode, and the stride is 1.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003e2.3 Dilated convolution\u003c/h2\u003e\n \u003cp\u003eIn U-Net, the receptive field is increased by pooling and the image size is reduced, and then the image size is restored by up-sampling. This process may cause the accuracy loss. To reduce this loss, it is natural to remove the pooling layer, but as a result, the receptive field of the feature map is too small. Dilated convolution is to solve this problem by increasing the receptive field. It is to insert the value of 0 between the ordinary convolution kernels to increase the expansion coefficient of the convolution kernel, so as to increase the receptive field, reduce the computation amount and retain the nodal information, as shown in Fig. 4 with 4 dilated rates. It is seen from Fig. 4, the size of receptive field increases with the dilated rate, but the network parameters do not increase, that is, 9 parameters.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"3. Dilated Multi-scale attention U-Net (DMSAU-Net)","content":"\u003cp\u003eDue to the small dataset and the easy influence of complex background such as illumination and clutter, as shown in Fig.\u0026nbsp;1, the detection accuracy of crop insect pest is low, which is over-segmentation or under-segmentation. In this Section, an improved U-Net model namely dilated multi-scale attention U-Net (DMSAU-Net) is constructed for insect pest image detection. Its overall structure is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e5\u003c/span\u003e. The numbers shown below each dilated Inception module indicate the total number of kernel used, height, width and depth of the output feature maps.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Detail of DMSAU-Net\u003c/h2\u003e \u003cp\u003eSimilar to U-Net, DMSAU-Net mainly consists of the encoding part, decoding part and skip connection with attention mechanism. Encoding part is a multi-scale convolutional network, including dilated Inception module (convolution kernel of 3\u0026times;3 kernel), the pooling layer (2\u0026times;2 maximum pooling), and the activation function (ReLU). In encoding part, the multi-scale and multi-level features are extracted from images through dilated Inception module, and then the extracted features are downsampled to realize the correlation of multiple channels and the full decoupling of image feature space. In decoding part, the extracted features are restored by up-sampling (deconvolution of 2\u0026times;2), skip connection with attention, dilated Inception module (convolution kernel of 3\u0026times;3) and activation function (ReLU) classification. Finally, the binary insect pest image of the insect pest and the background is obtained by 1\u0026times;1 convolution layer and Sigmoid activation function. Considering the possible mesh effect caused by deconvolution, the bilinear interpolation method in upsampling is used to restore the image, during which 1\u0026times;1 convolution is used to restore the number of channels. During the up-sampling process, the feature maps corresponding to the same resolution of down-sampling are concatenated. After each concatenation, the feature maps are further refined through dilated multi-scale module, and the up-sampling is performed successively until the features extracted from the encoder are restored to the size of the input maps. Skip connection with attention is to concatenate the convolutional features and the deconvolution features.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Dilated Inception module\u003c/h2\u003e \u003cp\u003eInspired by Inception module, multi-scale concatenation module and dilated concatenation module, a dilated multi-scale module is constructed as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e6\u003c/span\u003e, consisting of 3 dilated convolutional kernels, 3 1\u0026times;1 convolution kernels and a concatenation layer. It is a modified Inception module, which aims to extract multi-scale classification features, from low-level structural features to high-level semantic features, by increasing receptive field without increasing the training parameters.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Spatial attention module\u003c/h2\u003e \u003cp\u003eDilated Inception module obtains multi-scale features by encoding part and decoding part, and the extracted multi-scale features are concatenated by skip connection to achieve more accurate details and location information of insect pest. However, the max-pooling and upsampling operation in the encoding and decoding parts will lose part of the location space and other information, resulting in inaccurate segmentation of insect pest in the field. To overcome this problem, the spatial attention mechanism module is added to the skip connection. By directly cascading the features of the encoding layer and the corresponding deconvolution layer, the attention module fuses their complementary features, suppressed the noise generated by upsampling, and enhanced the robustness of the model [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. The structure of spatial attention module is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e7\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Model training and evaluation\u003c/h2\u003e \u003cp\u003eThe input insect pest images and their corresponding labeled images are used to train DMSAU-Net. Five\u003cem\u003e-\u003c/em\u003efold cross-validation (5-fold CV) scheme and stochastic gradient descent (SGD) with an adaptive moment estimator (Adam) are often used to train all models [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe purpose of crop insect pest detection is to determine the category of each pixel in the image, so as to clarify the scope of insect pests. Accuracy and Intersection over Union (IoU) are selected as indexes to evaluate the segmentation performance of the proposed algorithm, calculated as follows:\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$ACC=\\frac{{TP}}{{TP+FN}},\\;\\;\\;IoU=\\frac{{TP}}{{TP+FP+FN}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cem\u003eTP\u003c/em\u003e, \u003cem\u003eFP\u003c/em\u003e and \u003cem\u003eFN\u003c/em\u003e are the numbers of true positives, false positives, and false negatives of the class, respectively.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Experiments and analysis","content":"\u003cp\u003eTo validate the proposed DMSAU-Net based insect pest detection method, the insect pest detection experiments are conducted on the crop common insect pest image dataset, and compared with U-Net [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e] and its two improved models: U-Net with dilated convolution (DCU-Net) [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e] and ResNet with U-Net (ResU-Net)[\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. Batch size\u0026thinsp;=\u0026thinsp;32 for rice data subset of IP102 to reduce computation time. Number of iterations\u0026thinsp;=\u0026thinsp;3000, global learning rate\u0026thinsp;=\u0026thinsp;0.001, gradient decay factor\u0026thinsp;=\u0026thinsp;0.9, squared gradient decay factor\u0026thinsp;=\u0026thinsp;0.999, loss function\u0026thinsp;=\u0026thinsp;cross entropy. All models are tested on Keras and trained on a Intel Xeon E5-2643v3 @3.40GHz CPU, GTX2080Ti 11GB GPU, 64GB RAM, Windows 7 64bit, CUDA Toolkit10.0, CUDNN V7.6.5, Python 3.7 and Tensorflow-GPU 1.8.0.\u003c/p\u003e\n\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\n \u003ch2\u003e4.1 insect pest image dataset\u003c/h2\u003e\n \u003cp\u003eIP102 (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/xpwu95/IP102\u003c/span\u003e\u003c/span\u003e) is a public insect pest image dataset, containing more than 75,000 images belonging to 102 insect pest categories that exhibit a natural long-tailed distribution [\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e]. 19,000 of these images have be professionally annotated. There are 8415 insect pest images in the dataset belonging to 14 rice insect pest categories, as shown in Fig. \u003cspan class=\"InternalRef\"\u003e8\u003c/span\u003e and Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. From Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, it is seen that the classes of rice insect pests are highly unbalanced, ranging in sample size between 173 and 1115. In the experiments on this data subset, five-fold-cross validation scheme is adopted to perform experiments. That is, the dataset is randomly split into\u0026nbsp;\u003cem\u003e5\u003c/em\u003e mutually exclusive subsets of equal or near equal size. The model is performed 5 times subsequently, where each time using 4 of the 5 splits as the training set to train the model, and the 1 of the 5 splits as the test set to evaluate the performance of the model. To verify the robustness of the proposed method, images under different conditions such as strong illumination and complex background are selected, and the same insect pest in the dataset contained different insect states. To improve the recognition accuracy, Photoshop is used to uniformly adjust the image to 128\u0026times;128 pixels.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003erice insect pest image dataset\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eNo.\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003einsect pest name\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAmount\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eNumber\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003erice leaf roller\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1115\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0-1115\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003cp\u003e14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003erice leaf caterpillar\u003c/p\u003e\n \u003cp\u003epaddy stem maggot\u003c/p\u003e\n \u003cp\u003easiatic rice borer\u003c/p\u003e\n \u003cp\u003eyellow rice borer\u003c/p\u003e\n \u003cp\u003erice gall midge\u003c/p\u003e\n \u003cp\u003eRice Stemfly\u003c/p\u003e\n \u003cp\u003ebrown plant hopper\u003c/p\u003e\n \u003cp\u003ewhite backed plant hopper\u003c/p\u003e\n \u003cp\u003esmall brown plant hopper\u003c/p\u003e\n \u003cp\u003erice water weevil\u003c/p\u003e\n \u003cp\u003erice leafhopper\u003c/p\u003e\n \u003cp\u003egrain spreader thrips\u003c/p\u003e\n \u003cp\u003erice shell insect pest\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e485\u003c/p\u003e\n \u003cp\u003e261\u003c/p\u003e\n \u003cp\u003e1053\u003c/p\u003e\n \u003cp\u003e504\u003c/p\u003e\n \u003cp\u003e506\u003c/p\u003e\n \u003cp\u003e369\u003c/p\u003e\n \u003cp\u003e838\u003c/p\u003e\n \u003cp\u003e889\u003c/p\u003e\n \u003cp\u003e553\u003c/p\u003e\n \u003cp\u003e856\u003c/p\u003e\n \u003cp\u003e404\u003c/p\u003e\n \u003cp\u003e173\u003c/p\u003e\n \u003cp\u003e409\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1116\u0026ndash;1601\u003c/p\u003e\n \u003cp\u003e1602\u0026ndash;1862\u003c/p\u003e\n \u003cp\u003e1863\u0026ndash;2915\u003c/p\u003e\n \u003cp\u003e2916\u0026ndash;3419\u003c/p\u003e\n \u003cp\u003e3420\u0026ndash;3925\u003c/p\u003e\n \u003cp\u003e3926\u0026ndash;4294\u003c/p\u003e\n \u003cp\u003e4293\u0026ndash;5132\u003c/p\u003e\n \u003cp\u003e5133\u0026ndash;6021\u003c/p\u003e\n \u003cp\u003e6022\u0026ndash;6574\u003c/p\u003e\n \u003cp\u003e6575\u0026ndash;7430\u003c/p\u003e\n \u003cp\u003e7431\u0026ndash;7834\u003c/p\u003e\n \u003cp\u003e7835\u0026ndash;8007\u003c/p\u003e\n \u003cp\u003e8008\u0026ndash;8416\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\n \u003ch2\u003e4.2 Experimental results\u003c/h2\u003e\n \u003cp\u003eTo test the effectiveness of spatial attention module, Fig. 9 show the convolutional feature maps of the first dilated multi-scale module and the corresponding feature maps after spatial attention module. From Fig. 9, it is obvious that the convolutional feature maps after spatial attention module are more significant than that of the first dilated multi-scale module.\u003c/p\u003e\n \u003cp\u003eDMSAU-Net is a modified U-Net. Figure 10A shows the loss values versus the number of iterations of U-Net and DMSAU-Net on the training set. From Fig. 10A, it is found that DMSAU-Net converges much better than U-Net.\u003c/p\u003e\n \u003cp\u003eTo further analyze the training performance, the same experimental parameters are used to train the different networks to ensure the reliability of the comparison results. The detection results are shown in Fig.\u0026nbsp;10B. From Fig.\u0026nbsp;10B, it can be seen that as the number of iterations increases, the detection loss of the three models decreases, and the detection performance of DMSAU-Net outperformances other models. Before 1000 iterations, their losses decrease quickly, go down very slowly and then level off. From Fig.\u0026nbsp;10, it can be seen that their training processes are relatively stable after 2000 iterations, DCU-Net and DMSAU-Net are better than ResU-Net, and DMSAU-Net is better than DCU-Net. This owes to the dilated multi-scale convolution and spatial attention mechanism.\u003c/p\u003e\n \u003cp\u003eTo test the detection performance of DMSAU-Net, we randomly select 4 rice insect pest images from the IP102 dataset and visualize the detection results, as shown in Fig.\u0026nbsp;11. To reflect the advantages of deep learning methods, we compare DMSAU-Net with the traditional insect pest detection algorithm. Fuzzy c-means (FCM) is one of the most widely used for insect pest detection [\u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e]. The detection results are also in Fig. 11. As can be seen from Fig. 11, the four improved U-Net models are much better than FCM. They can effectively detect rice insect pests under complex background, and the position and shape of insect pests are good, among which DCU-Net is better than ResU-Net, and DMSAUN-Net has the best detection effect with more accurate insect pest shape and edge.\u003c/p\u003e\n \u003cp\u003eTo quantitatively estimate the detection performance of DMSAU-Net, 5-fold cross validation experiments are conducted on the rice insect pest image subset of IP102. As can be seen from Fig. 10, all models basically converge at the 3000th iteration. For fair, the trained models are selected at the 3000th iteration. The detection results of U-Net, DCU-Net, ResU-Net and DMSAU-Net are listed in Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u0026nbsp;\u003c/p\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eThe detection results of rice insect pests by 4 insect pest detection algorithms\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMethod\u003c/p\u003e\n \u003cp\u003eResults\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eU-Net\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDCU-Net\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResU-Net\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDMSAU-Net\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAccuracy (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.842\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.907\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.884\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.921\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIoU (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.814\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.892\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.865\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.917\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTraining time (h)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTest time (ms)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e45\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e31\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003eDMSAU-Net is constructed by making use of the advantage of U-Net, dilated convolution, multi-scale convolution and spatial attention mechanism. To further verify the superiority of DMSAU-Net, some ablation experiments are carried out under similar conditions. The different experimental set and rice insect pest results are shown in Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, where attention is added to skip connection of U-Net.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eThe different experimental set and result\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResults\u003c/p\u003e\n \u003cp\u003eExperimental set\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eIoU\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTraining time (h)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.842\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.814\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e15.6\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u0026thinsp;+\u0026thinsp;spatial attention\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.856\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.843\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e13.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u0026thinsp;+\u0026thinsp;Inception\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.872\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.864\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e13.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u0026thinsp;+\u0026thinsp;dilated multi-scale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.881\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.875\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e11.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u0026thinsp;+\u0026thinsp;Inception\u0026thinsp;+\u0026thinsp;spatial attention\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.890\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.879\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e10.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eU-Net\u0026thinsp;+\u0026thinsp;dilated multi-scale\u0026thinsp;+\u0026thinsp;channel-spatial attention\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.912\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.908\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e8.6\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eTable \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e indicates that spatial attention, channel-spatial attention, Inception, dilated multi-scale can contribute to the results to some extent.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n \u003ch2\u003e4.3 Analysis\u003c/h2\u003e\n \u003cp\u003eFrom Figs. 10 and 11 and Tables \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, it can be seen that DMSAU-Net has best detection performance, the highest detection rate and the least training and detecting time, due to dilated convolution, multi-scale convolution and spatial attention mechanism. With the aid of dilated multi-scale, DMSAU-Net can extract the multi-scale classification features. With the aid of skip connection combined with spatial attention mechanism, DMSAU-Net can enhance the constraint on the feature maps, focus more attention to the insect pest image region, and speed up the training. That is to reduce the learning of non-important areas and enhance the learning of insect pest areas, so as to improve the detection ability of insect pest characteristics and improve the detection accuracy rate. The detection rate of U-Net is the poorest, because it is difficult to extract the robust classification features from the various insect pest images with very complex background.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"5. Conclusions","content":"\u003cp\u003eIn modern agricultural field, insect pest detection plays an important role in timely and accurate diagnosis of crop insect pest. But it is difficult to detect crop insect pest in the field due to the various-shape-size insect pests with complex background. To solve such problem, a dilated multi-scale attention U-Net (DMSAU-Net) model is constructed for crop insect pest detection by making use of the advantage of multi-scale convolution and attention mechanism. In dilated inception module, multi-scale convolution kernels without increasing training parameters are used to extract the distributed characteristics of insect pests at different scales and to perform cascade fusion. The experiments are carried out on the rice insect pest image subset of IP102 dataset. The detection results show that DMSAU-Net is effective and feasible for crop insect pest detection in the field. This research can be used to realize the automation degree of insect pest management in agricultural field. Future work is to optimize the model to organically integrate it into an effective insect pest detection system.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eConflict of Interest\u003c/h2\u003e\n\u003cp\u003eThe authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.\u003c/p\u003e\n\u003ch3\u003eEthical Approval\u003c/h3\u003e\n\u003cp\u003eno applicable.\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThis study was supported in part by grants from the National Natural Science Foundation of China (Nos. 62172338) and 2022 General Special Research Plan Project of Shaanxi Provincial Department of Education (No.: 22JK0596).\u003c/p\u003e\n\u003ch3\u003eAvailability of data and materials\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eThe data that support the findings of this study are openly available in https://github.com/xpwu95/IP102.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eDattatraya VS, Sudhir KS, Ghanshyam C et al. Low cost sensor based embedded system for plant protection and pest control. International Conference on Soft Computing Techniques \u0026amp; Implementations, 2015. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICSCTI.2015.7489628\u003c/span\u003e\u003cspan address=\"10.1109/ICSCTI.2015.7489628\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBhujel A, Mahonar S, Choubey M, et al. Pest and Diseases Management in Darjeeling Tea. Social Sci Electron Publishing. 2016;6(3):469\u0026ndash;72.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXie C, Zhang J, Li R, et al. Automatic classification for field crop insects via multiple-task sparse representation and multiple-kernel learning. Computers \u0026amp; Electronics in Agriculture. 2015;119:123\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSelvaraj MG, Vergara A, Ruiz H, et al. AI-powered banana diseases and pest detection. Plant Methods. 2019;15(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s13007-019-0475-z\u003c/span\u003e\u003cspan address=\"10.1186/s13007-019-0475-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThenmozhi K, Reddy US. Crop pest classification based on deep convolutional neural network and transfer learning. Computers and Electronics in Agriculture,2019,164,104906. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.compag.2019.104906\u003c/span\u003e\u003cspan address=\"10.1016/j.compag.2019.104906\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTeng Yue Z, Jie D, Shifeng, et al. MSR-RCNN: A Multi-Class Crop Pest Detection Network Based on a Multi-Scale Super-Resolution Feature Enhancement Module. Frontiers. 2022;13. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/fpls.2022.810546\u003c/span\u003e\u003cspan address=\"10.3389/fpls.2022.810546\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePratheba R, Sivasangari A, Saraswady D et al. Performance analysis of pest detection for agricultural field using clustering techniques. International Conference on Circuits, Power and Computing Technologies, 2014. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICCPCT.2014.7054833\u003c/span\u003e\u003cspan address=\"10.1109/ICCPCT.2014.7054833\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGalphat Y, Patange VR, Talreja P et al. Survey and Analysis of Pest Detection in Agricultural Field. International Conference on Computer Networks, Big Data and IOT, 2018: 976\u0026ndash;983. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-3-030-24643-3_116\u003c/span\u003e\u003cspan address=\"10.1007/978-3-030-24643-3_116\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang F, Wang R, Xie C, et al. Fusing multi-scale context-aware feature representation for automatic in-field pest detection and recognition. Comput Electron Agric. 2020;169:105222.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRikiya Y, Mizuho N, Gian D et al. Convolutional neural networks: an overview and application in radiology. Insights into Imaging, 2018:1\u0026ndash;19. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s13244-018-0639-9\u003c/span\u003e\u003cspan address=\"10.1007/s13244-018-0639-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJafari R, Razvarz S, Gegov A et al. Deep Learning for Pipeline Damage Detection: an Overview of the Concepts and a Survey of the State-of-the-Art. 10th IEEE International Conference on Intelligent Systems IS\u0026rsquo;20, 2020. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/IS48319.2020.9200137\u003c/span\u003e\u003cspan address=\"10.1109/IS48319.2020.9200137\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLong J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and, Recognition P. (CVPR), Boston, MA, USA, 2015:3431\u0026ndash;3440.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRoth HR, Shen C, Oda H et al. A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018, 11073, 417\u0026ndash;425. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-3-030-00937-3_48\u003c/span\u003e\u003cspan address=\"10.1007/978-3-030-00937-3_48\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBadrinarayanan V, Kendall A, Cipolla R, Segnet. A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39:2481\u0026ndash;95.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi R, Liu W, Yang L, et al. DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation. IEEE J Sel Top Appl Earth Observations Remote Sens. 2018;11(11):3954\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSivagami S, Chitra P, Kailash G et al. U-Net Architecture Based Dental Panoramic Image Segmentation. International Conference on Wireless Communications Signal Processing and Networking, 2020: 187\u0026ndash;191.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang ZX, Liu QJ, Wang YH. Road extraction by deep residual U-Net. IEEE Geoscience and Remote Sensing Letters 2018,15(5):749\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang T, Zhou Y, Li L, et al. DCU-Net: Multi-scale U-Net for brain tumor segmentation. J X-Ray Sci Technol. 2020;28(4):709\u0026ndash;26.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNajeeb RS, Dahl IO. Brain Tumor Segmentation Utilizing Generative Adversarial, Resnet and U-net Deep Learning. 8th International Conference on Contemporary Information Technology and Mathematics, Mosul, Iraq, 2022:85\u0026ndash;89. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICCITM56309.2022.10031760\u003c/span\u003e\u003cspan address=\"10.1109/ICCITM56309.2022.10031760\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShi B, Su Y, Lian C et al. Obstacle type recognition in visual images via dilated convolutional neural network for unmanned surface vehicles. J Navig, 202275(2): 437\u0026ndash;54.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu Y, Guo C, Gao H, et al. Dilated residual networks with multi-level attention for speaker verification. Neurocomputing. 2020;412(4):177\u0026ndash;86.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang H, Yang J, Wang R, et al. Remaining Useful Life Prediction of Bearings Based on Convolution Attention Mechanism and Temporal Convolution Network. IEEE Access. 2023;11:24407\u0026ndash;19. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2023.3255891\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2023.3255891\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWei H, Zhang Q, Gu Y. Remaining Useful Life Prediction of Bearings Based on Self-attention Mechanism, Multi-scale Dilated Causal Convolution, and Temporal Convolution Network. Meas Sci Technol. 2023;34(4):045107. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1088/1361-6501/acb0e9\u003c/span\u003e\u003cspan address=\"10.1088/1361-6501/acb0e9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen LC, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Dilated Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell. 2018;40(4):834\u0026ndash;48.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe H, Yang DF, Wang SC, et al. Road extraction by using atrous spatial pyramid pooling integrated encoder-decoder network and structural similarity loss. Remote Sens. 2019;11(9):1015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu M, Yao D, Liu Z, et al. An Improved Adam Optimization Algorithm Combining Adaptive Coefficients and Composite Gradients Based on Randomized Block Coordinate Descent. Comput Intell Neurosci. 2023;10(4765891). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1155/2023/4765891\u003c/span\u003e\u003cspan address=\"10.1155/2023/4765891\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu X, Zhan C, Lai YK et al. IP102: a large-scale benchmark dataset for insect pest recognition. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),2019: 8779\u0026ndash;8788.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePratheba R, Sivasangari A, Saraswady D. Performance analysis of pest detection for agricultural field using clustering techniques. International Conference on Circuits, Power and Computing Technologies, 2014, 14968106. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICCPCT.2014.7054833\u003c/span\u003e\u003cspan address=\"10.1109/ICCPCT.2014.7054833\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"plant-methods","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"plme","sideBox":"Learn more about [Plant Methods](http://plantmethods.biomedcentral.com/)","snPcode":"13007","submissionUrl":"https://submission.nature.com/new-submission/13007/3","title":"Plant Methods","twitterHandle":"@PlantMethods","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"insect pest detection, U-Net, Dilated Inception. Multi-scale convolution, Dilated multi-scale attention U-Net (DMSAU-Net)","lastPublishedDoi":"10.21203/rs.3.rs-3799461/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3799461/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEarly detection and identification of insect pests is the premise and basis of scientific control and accurate utilization of Insect pesticides. Aiming at the problems of low detection accuracy and slow training speed of the existing crop Insect pest detection models, a dilated multi-scale attention U-Net (DMSAU-Net) model is constructed for crop Insect pest detection. In its encoder, dilated Inception is designed to replace the convolution layer in U-Net to extract the multi-scale features of insect pest images and improve the accuracy of the model. An attention module is added to its decoder to focus on the edge of the insect pest image and reduce the upsampling noise and accelerate model convergence. The results on the crop insect pest image dataset verify that the proposed method has high segmentation accuracy and good generalization ability, and can be applied to practical crop insect pest monitoring system.\u003c/p\u003e","manuscriptTitle":"Crop Insect Pest Detection based on Dilated Multi-scale Attention U-Net","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-03 01:44:04","doi":"10.21203/rs.3.rs-3799461/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-01-29T21:00:40+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-01-11T07:24:18+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"0c949dad-2966-43a6-9805-9577cb4a0690","date":"2024-01-03T14:16:59+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"b9f2ca9e-3da7-4881-b564-d223b79215ff","date":"2023-12-29T11:41:43+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"86bbb7db-215b-410d-aeda-ed03977aa8f0","date":"2023-12-29T01:06:27+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2023-12-28T23:26:56+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2023-12-28T05:28:30+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2023-12-28T05:28:30+00:00","index":"","fulltext":""},{"type":"submitted","content":"Plant Methods","date":"2023-12-24T07:54:59+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"plant-methods","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"plme","sideBox":"Learn more about [Plant Methods](http://plantmethods.biomedcentral.com/)","snPcode":"13007","submissionUrl":"https://submission.nature.com/new-submission/13007/3","title":"Plant Methods","twitterHandle":"@PlantMethods","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"16ca2f64-209d-4349-a3ce-f220b6802e2f","owner":[],"postedDate":"January 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-03-04T15:02:27+00:00","versionOfRecord":{"articleIdentity":"rs-3799461","link":"https://doi.org/10.1186/s13007-024-01163-w","journal":{"identity":"plant-methods","isVorOnly":false,"title":"Plant Methods"},"publishedOn":"2024-02-26 15:00:34","publishedOnDateReadable":"February 26th, 2024"},"versionCreatedAt":"2024-01-03 01:44:04","video":"","vorDoi":"10.1186/s13007-024-01163-w","vorDoiUrl":"https://doi.org/10.1186/s13007-024-01163-w","workflowStages":[]},"version":"v1","identity":"rs-3799461","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3799461","identity":"rs-3799461","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}