Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification

Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification Linan Wang, Hongmin Zhao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4691672/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract In recent years, the output of China's four major crops has declined due to pests and diseases. This situation poses a serious challenge to food security. Therefore, timely detection and prevention of diseases is essential. First, we use data enhancement techniques to augment the data to improve the generalization ability of the model. Secondly, to reduce the model parameters and facilitate the deployment at the terminal, we use the knowledge distillation method. Finally, a method of dynamically adjusting the parameter T according to the loss value (DYTKD) is proposed to improve the performance of the model further. The experiment shows that knowledge distillation can reduce the number of parameters while making the accuracy of the student model as close as possible to the teacher model 98.94%. Meanwhile, data augmentation can also improve the accuracy of the model by 6.83%. Compared with the basic knowledge distillation method, the accuracy of DYTKD was increased by 1.3% without changing the student network and other parameters, and the accuracy of pest identification and classification was effectively improved. Among 1342 pest pictures, 1221 were correctly identified and accurately classified. Our codes are available at https://github.com/wln130221/DYTKD . Data augmentation knowledge distillation Dynamic temperature Image recognition and classification Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 1. Introduction In agricultural production, pest control is essential to ensure stable crop yields. In recent years, especially in the planting process of China's four major staple grains - rice, wheat, corn, and potato, the frequent occurrence of major pests and diseases such as wheat scab and stripe rust, rice sheath blight, and corn spot disease has brought great challenges to agricultural production. These diseases not only directly threaten the safety of grain and oil production, but also may lead to a significant decline in crop yield and serious damage to quality, which in turn hurts the overall development of the agricultural industry. It is predicted that by 2024, the pest and disease situation of these food crops will be even more severe. It is estimated that the area of major pests and diseases of wheat, rice, corn, potatoes, and other food crops will reach a staggering 2.04 billion mu nationwide, an increase of 15% over the average level in 2023. This means that more than 70% of food crop-producing areas will be threatened by these pests and diseases, and agricultural production is facing unprecedented risks and challenges. To cope with this critical situation and improve agricultural production efficiency, timely detection and early prevention of plant diseases have become crucial. With the rapid development of artificial intelligence and computer technology, deep learning algorithms have shown strong potential in many fields, such as agriculture and forestry, medicine, finance, and other fields. Deep learning has become a popular research direction and has achieved excellent results in the fields of natural language processing (NLP), image processing, and object detection. In the field of pest identification, we try to use deep learning algorithms and computer vision technology to realize the automatic identification and classification of pest and disease images, which greatly improves the accuracy and efficiency of pest and disease identification. Among the many deep learning models, convolutional neural networks (CNNs) have performed well in the field of pest and disease identification due to their unique advantages. At present, a variety of CNN-based pest identification methods have been proposed, such as AlexNet, VGGNet, ResNet [ 1 – 3 ], etc., and these models have achieved excellent results in the field of pest and disease identification. For example, Cheng et al.[ 4 ]achieved remarkable results with the application of improved AlexNet in the classification of strawberry pests and diseases, with an accuracy rate of 98%. At the same time, the work of Alatawi et al.[ 5 ]also revealed the great potential of the VGGNet model in plant disease identification. With the deepening of research, it is found that the expressive ability of the model can be further improved by designing deeper neural network structures, to extract more discriminating depth features. However, this also poses a problem: a deeper network structure means more model parameters and higher computational complexity, as well as higher requirements for storage devices and computing resources. This makes existing deep learning models face certain challenges in practical applications, especially when deployed on end-user mobile devices with limited resources. To solve this problem, researchers have begun to explore how to reduce the complexity and computational cost of the model while ensuring the accuracy of the model[ 6 – 7 ]. Among them, the proposal of a knowledge distillation (KD) method[ 8 ] has attracted extensive attention. This approach guides the learning process of a smaller, less parameter-rich model (the student model) by utilizing a larger, parameter-rich model (the teacher model). During the training process, the teacher model "distills" the knowledge and feature information it has learned into the student model so that the student model has lower complexity and computational cost while maintaining high accuracy. To let the students better imitate the teacher, we also set the temperature parameter T to be dynamically adjusted according to the predicted value output by the student model and the loss value of the true label. This method can not only simplify the model structure on the premise of ensuring the accuracy of the model but also improve the feasibility and generalizability of the model in practical application. Therefore, the pest classification method based on knowledge distillation provides a new idea and method for solving practical problems in the identification of crop diseases and pests. The rest of this article is organized as follows: First, we reviewed the work in Section 2 . Then, in the third section, we will elaborate on the details of the ideas and models presented. We then present the experimental results and comparisons in Section 4 and finally summarize our work in Section 5 . 2. Related Work 2.1 The development process of crop pest and disease identification In the early stages of crop pest identification, manual on-site inspection, measurement, and identification are mainly relied upon. This method is highly dependent on the experience and expertise of the observer, so it is highly subjective. Different experts may come to different conclusions on the same issue. In addition, manual identification is inefficient, time-consuming, and labor-intensive, making it difficult to cope with large-scale agricultural production needs. To improve the efficiency and accuracy of identification, researchers have gradually introduced various detection instruments. For example, Li et al. [ 9 ]developed a citrus pest detector. However, these testing instruments still need to be operated manually, the degree of automation and intelligence is generally low, and the detection results are easily affected by external environmental factors (such as temperature, light, etc.), which limits their wide application. In recent years, with the rapid development of image processing and pattern recognition technologies, experts and scholars have begun to apply these technologies to the identification of crop diseases and pests[ 10 ]. At this stage, the color, texture morphology, and other features extracted from the image, combined with recognition technologies such as linear classifiers, are used to classify and identify various crop diseases and pests, which promotes the development of agricultural informatization and precision. Duarte-Carvajalino et al.[ 11 ] aimed to identify potato diseases and used the collected datasets to train machine learning algorithms such as multilayer perceptrons, convolutional neural networks (CNNs), support vector machines, and random forests. The results show that the performance of the convolutional neural network is slightly better than that of other machine learning algorithms. There are many commonly used CNNs, each with its unique advantages[ 12 ], such as VGG, ResNet, GoogleNet, and DenseNet. [ 13 – 16 ] provides detailed research and experimental results on the role of CNN in pest and disease identification, which provides an important reference for further research and application. With the continuous progress of science and technology, more and more scholars at home and abroad have begun to use deep learning technology to identify crop diseases and pests.[ 17 ] Through automated and efficient image processing technology, target features can be accurately extracted from a large number of crop pest and disease images, thereby improving the accuracy and efficiency of identification. These technologies not only reduce the dependence on manual experience but also greatly improve the speed and reliability of pest and disease identification, providing strong support for agricultural production. In general, crop pest and disease identification technology has undergone a significant development process from early manual detection to the use of image processing and pattern recognition technology, and then to the widely used deep learning technology. With the continuous maturity of deep learning technology, the identification of crop diseases and pests will become more intelligent and efficient, providing strong technical support for the modernization and precision of agriculture. 3. Methods 3.1 Data preprocessing Data augmentation, as a key technology in the field of deep learning, is a method of extending a finite dataset by generating more equivalent data, aiming to increase the number and diversity of training samples, thereby improving the generalization ability and performance of the model. Li et al.[ 18 ] demonstrated through research that data augmentation significantly improves knowledge distillation performance on image classification and object detection tasks even though the teacher model lacks comprehensive information about augmented samples. With data augmentation, the over-reliance of the model on specific features can be reduced, thus avoiding overfitting. In deep learning, a network often has millions of parameters that require large amounts of data to train effectively. However, in some application scenarios, such as crop identification, the difficulty, and complexity of data collection are relatively high due to the influence of various factors on the growing environment of crops, and it is usually difficult to obtain enough data to support network training. Using raw data directly can lead to unbalanced sample classification and affect the training process of the model. Since the training goal is to improve the classification accuracy on the dataset, the category with a large sample size tends to dominate the training process due to its large cumulative training error, resulting in the algorithm biasing towards this category on unbalanced data. To improve the accuracy of the model and avoid overfitting the model, we can use data augmentation techniques to augment the sample[ 19 ]. In the application of crop identification, the growth environment of crops is affected by many factors, such as temperature and humidity, light, rainfall, soil, etc., and these objective conditions make data collection more difficult and complex. Therefore, data augmentation becomes an effective means to solve this problem. Common data augmentation methods include: Add noise: Introduce random noise into the input data to simulate real-world noise interference, making the model more robust to random changes in the data. Modify the intensity and contrast of the image: Enhance the adaptability of the model by adjusting the brightness, contrast, and color of the image to generate different viewing angles. Geometric transformation: including rotation, clipping, and translation, by changing the geometric properties of the image, more training samples are generated. [ 20 – 21 ] For example, an example of a data augmentation method is shown in Fig. 1 , showing the leaves of four crop diseases, and the original image undergoes a series of transformations, from left to right, after 45° rotation, adjusting the brightness of the image, Gaussian blurring, color enhancement, and saturation increase. These newly generated images are treated as different samples in the network, which improves the recognition accuracy of the model and effectively mitigates the overfitting phenomenon. Through these methods, the model can better learn and identify diseases in complex and variable crop growth environments, and improve the overall classification performance and robustness. 3.2 knowledge distillation In 2015, Hinton et al. first proposed the Knowledge Distillation technique[ 6 ] to solve the problem of increasing parameters as models become more complex. Knowledge distillation involves training the student network to mimic the output of the teacher's network and mimic its internal characteristics or decision-making processes. The purpose of this technique is to enhance the performance of smaller models while making them behave close to larger models while reducing computational costs and memory requirements. The knowledge distillation system consists of three key components: knowledge, refining algorithms, and teacher-student architecture. Figure 2 illustrates the flowchart of knowledge distillation, with a "teacher-student" structure at its core. There are three different forms of knowledge in knowledge distillation: response-based knowledge, feature-based knowledge, and relationship-based knowledge. In this paper, we will use the logits of the last output layer of the teacher model as teacher knowledge[ 22 ]. This method is known as response-based knowledge distillation, and its main idea is to directly mimic the final predictions of the teacher model. Response-based knowledge distillation is a simple and effective model compression method that has been widely used in different tasks and applications. The Logits of the last output of the teacher model are used as the basis for comparing the output of the student model and the teacher model. By testing Logits, the model can measure the certainty or confidence of its predictions before applying the Softmax function to obtain the final probability. Specifically, the probability of the Softmax function is calculated as follows: $$\:\begin{array}{c}{p}_{i}=\frac{exp\left(\frac{zi}{T}\right)}{\sum\:_{j}exp\left(\frac{zj}{T}\right)}\left(1\right)\end{array}$$ Here, zi is the logit of class i, and the importance of the hyperparameter T is introduced to control the importance of each soft target, where T represents the "temperature" of knowledge distillation. When T = 1, it corresponds to the normalized exponential function. As the temperature parameter T increases, the probability distribution of the Softmax function becomes smoother, conveying finer details about the relationship between the different classes in the teacher model. 3.3 Dynamically adjust the temperature The soft tags described by Hinton et al. contain valuable dark knowledge from the teacher model. Thus, the loss function is extended to include the standard cross-entropy loss between the predictions of the student model and the true labels, as well as the difference between measuring the softening probability of the teacher's model (obtained by a higher temperature Softmax) and the corresponding predictions of the student model. The loss value is used to measure the difference between the model prediction and the true label. The smaller the loss value, the more accurate the model prediction. At the same time, the T-parameter is an important hyperparameter in the model, which may affect the complexity, learning speed, or generalization ability of the model. Figure 3 is a flow chart of the network for training students. In the process of traditional knowledge distillation training, we use static T parameters, which cannot adjust the training state in time. [ 23 ] It has been shown that the traditional fixed temperature setting may not be sufficient to capture the full knowledge of the teacher model during distillation. So this paper proposes a method to dynamically adjust the temperature parameter T according to the value of the student's predicted value and the cross-entropy loss of the real label, to expect the model performance to achieve better training results. To implement this method, we only need to complete the following steps in the training process: first, set an initial parameter value T, secondly, calculate the cross-entropy loss of the prediction of the student model and the real label, and then set a threshold, and when the loss value is greater than the threshold, increase the parameter T appropriately; When the loss value is less than the threshold, the parameter T is lowered appropriately, and finally, we recalculate the distillation loss value with the T parameter. The combined loss function for distillation loss and student loss is determined as follows: $$\:\begin{array}{c}L\left(x;W\right)=\alpha\:\times\:CE\left(y,\sigma\:\left({z}_{s};T=1\right)\right)+\left(1-\alpha\:\right)\times\:KL\left(\sigma\:\left({z}_{t};T=\tau\:\right),\sigma\:\left({z}_{s},T=\tau\:\right)\right)\left(2\right)\end{array}$$ In this case, x represents the input data, W is the set of weight parameters for the student model, and y is the corresponding true class label. We introduce cross-entropy loss (CE) as part of supervised learning, which aims to optimize the accuracy of the student model for direct label prediction. Meanwhile, using a Softmax function σ regulated by a "temperature" parameter T, we transform the logits (Zt) of the teacher model into a smoother probability distribution, a process called "softening" the objective. The student model's logits (Zs) try to match these softened targets, which goes beyond simple label prediction to learn the teacher model's internal understanding and representation of the data. To further improve the performance of the student model, we design a comprehensive loss function to minimize the comprehensive loss function, which not only encourages the student model to learn to classify correctly, but also encourages it to learn the deep feature representation and generalization ability of the teacher model when dealing with complex data. 4. Experiment 4.1 Dataset and preprocessing The Rice Leaf Disease Images dataset was used in the experiment, including Bacterial blight, Blast, Brown spot, and Tungro. There were 4700 images in JPEG format. The size is 224x224 pixels. Table 1 shows our experimental condition configuration. To further enhance the generalization ability of the model, reduce the risk of overfitting, and improve the robustness of the model in the face of complex and variable environments, we introduce a data augmentation strategy. This strategy includes not only basic brightness and contrast adjustments, but also advanced techniques such as noise addition and image rotation to simulate real-world disturbances such as illumination changes, shooting Angle differences, and natural noise. Through these measures, the size of the original dataset was successfully doubled to 9400 images, and each image was carefully scaled and normalized, which laid a solid foundation for the subsequent training process. Table 1 Experimental Configuration Experimental Environment Value Programming language Deep learning framework GPU GPU acceleration tool Python 3.8 PyTorch 1.11.0 RTX 3080 CUDA:11.3 To quantitatively evaluate the actual effect of data augmentation strategies, we design a series of rigorous comparison experiments. While keeping the key elements of the network architecture (ResNet34 as the teacher model and ResNet18 as the student model for knowledge distillation) and the optimization algorithm unchanged, we use the original dataset and the enhanced dataset for training respectively. The experimental results show that after using the enhanced dataset, the recognition accuracy of the model is significantly improved, from 90.98–97.81%, with an increase of 6.83 percentage points. Figure 4 visually shows this significant performance improvement, which further confirms the excellent contribution of data augmentation in improving the robustness and accuracy of the model. In addition, we use more detailed evaluation metrics, including accuracy, precision, and recall, to fully understand the performance of the model. $$\:\begin{array}{c}Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\left(3\right)\end{array}$$ $$\:\begin{array}{c}\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}=\frac{TP}{TP+FP}\left(4\right)\end{array}$$ $$\:\begin{array}{c}Recall=\frac{TP}{TP+FN}\left(5\right)\end{array}$$ By comparing the TP (true examples), FP (false positive examples), TN (true negative examples), and FN (false negative examples) data of the four pest categories before and after data augmentation in Table 2 , we found that the data augmentation strategy significantly reduced the misclassification (for example, the FP of Bacterial blight decreased from 87 to 4, and the FP of Bacterial blight decreased from 87 to 4. Tungro's TP increased from 154 to 307), which greatly improved the accuracy of the model. Table 2 Performance comparison before and after data enhancement (after knowledge distillation) Original data Enhance data Class Bacterial blight Blast Brown spot Tungro Bacterial blight Blast Brown spot Tungro TP 546 306 212 154 359 237 296 307 FP 87 23 14 0 4 10 19 0 FN 30 54 40 0 25 3 4 1 TN 679 959 1076 1188 844 982 913 924 Accuracy(%) 91.28 94.26 95.98 100.0 97.65 98.94 98.13 99.92 Precision(%) 86.26 93.01 93.81 100.0 98.90 95.95 93.97 100.0 Recall(%) 94.79 85.00 84.13 100.0 93.49 98.75 98.67 99.68 4.2 Experimental Results Taking the Resnet model as an example, we use the pre-trained Resnet34 as the teacher model on the rice dataset, and the accuracy can reach 98.94%. To further compress the model, we use knowledge distillation technology to pass the logits of the teacher output layer as knowledge to the lightweight student model Resnet18, and the accuracy is 97.81%. Compared with the Resnet18 model without distillation, the performance has been significantly improved. Figure 5(a) shows the comparison chart of teacher-student model accuracy. Table 3 Comparison of accuracy, precision, and recall of DYTKD Model Accuracy Precision Recall Parameters(M) Teacher: ResNet34 Student: ResNet18 98.94% 93.83% 99.09 95.12 98.79 92.46 21.8 11.2 KD DYTKD 97.81% 99.11% 97.7 99.15 97.94 99.04 11.2 11.2 To verify the effectiveness of dynamically adjusting the parameter T, the module of dynamically adjusting T is introduced without changing the network structure, and other hyperparameters, Table 3 , Table 4 and Fig. 5(b) are the comparison of the experimental results. The experimental results show that our proposed DYTKD method can fully let students learn the knowledge in the teacher network, and generalize well to the test data set. Not only the accuracy and precision are improved, but the number of true examples (TP) also increased from 367 to 382. Figure 6(a) is the confusion matrix of the student model on the test set after knowledge distillation. It performs well in classifying these pests and diseases, but there is still room for improvement, especially in reducing misjudgment, among which there are 9 times that are Bacterial blight but predicted as Blast. Figure 6(b) is the confusion matrix of our proposed DYTKD method. Compared with the previous confusion matrix, the performance of the model has been further improved with higher accuracy and fewer misjudgments. Table 4 Comparison of TP, FP, TN, and FN of DYTKD teacher student Class Bacterial blight Blast Brown spot Tungro Bacterial blight Blast Brown spot Tungro TP 384 234 294 307 382 180 288 306 FP 11 2 0 0 60 3 7 6 FN 0 6 6 1 2 60 12 2 TN 837 990 932 924 788 989 925 918 Accuracy(%) 99.11 99.35 99.51 99.92 94.97 94.89 98.46 99.35 Precision(%) 97.22 99.15 100.0 100.0 86.43 98.36 97.63 98.08 Recall(%) 100.0 97.50 98.0 99.68 99.48 75.0 96.0 99.35 KD DYTKD Class Bacterial blight Blast Brown spot Tungro Bacterial blight Blast Brown spot Tungro TP 367 234 298 306 382 236 296 307 FP 6 9 12 0 7 3 1 0 FN 17 6 2 2 2 4 4 1 TN 842 983 920 924 841 989 931 924 Accuracy(%) 98.13 98.78 98.86 99.84 99.27 99.43 99.59 99.92 Precision(%) 98.39 96.30 96.13 100.0 98.20 98.74 99.66 100.0 Recall(%) 95.57 97.50 99.33 99.35 99.48 98.33 98.67 99.68 Table 5 Teachers and students share the same architectures. ∆represents the performance improvement over the KD. Teacher(%) Student(%) Resnet34 95.14 Resnet18 91.77 Resnet50 94.84 Resnet18 91.60 Resnet101 94.72 Resnet34 90.67 Resnet152 94.77 Resnet34 89.93 KD(%) 94.88 93.76 93.07 93.17 DYTKD(%) 95.65 94.93 93.90 93.96 △(%) + 0.77 + 1.17 + 0.83 + 0.79 To further verify the effectiveness of the DYTKD method, we also carried out comparison tests of different networks and replaced the teacher-student network with different network structures, as shown in Table 5 . The experimental results show that the performance of DYTKD is improved compared with KD in each time. 5. Conclusions In this paper, the recognition and classification performance of crop pests and diseases is significantly improved by effectively transferring the knowledge of large models to small models and combining data augmentation techniques. Taking the rice dataset as an example, our experimental results show that the loss threshold is set to dynamically adjust the temperature parameter during the training process. When the student predicted and true label cross-entropy loss values are large, it means that the student's student ability is weak, and we appropriately adjust to the appropriate temperature. Compared with the static temperature parameter in the traditional knowledge distillation method, the performance of the model is improved. Through this method, it provides strong technical support for the accurate identification and classification of crop pests and diseases. Declarations Author Contribution Wang participated in the experiments, data analysis visualization and manuscript writing;Zhao gives final approval for what will be published.All authors reviewed the manuscript Data Availability The datasets generated during and/or analysed during the current study are available in the [GitHub] repository, [https://github.com/wln130221/DYTKD] References Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012, 25. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arxiv preprint arxiv:1409.1556, 2014. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770–778. Dong C, Zhang Z, Yue J, et al. Classification of strawberry diseases and pests by improved AlexNet deep learning networks[C]//2021 13th International Conference on Advanced Computational Intelligence (ICACI). IEEE, 2021: 359–364. Alatawi A A, Alomani S M, Alhawiti N I, et al. Plant disease detection using AI based vgg-16 model[J]. International Journal of Advanced Computer Science and Applications, 2022, 13(4). Guan H, Fu C, Zhang G, et al. A lightweight model for efficient identification of plant diseases and pests based on deep learning[J]. Frontiers in Plant Science, 2023, 14: 1227011. Xu Y, Gao Z, Zhai Y, et al. A CNNA-Based Lightweight Multi-Scale Tomato Pest and Disease Classification Method[J]. Sustainability, 2023, 15(11): 8813. Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arxiv preprint arxiv:1503.02531, 2015. Li Z, Hong T, Wang J, et al. Development and experiment of Panonychus citri infestation fast detector[J]. Transactions of the Chinese Society of Agricultural Engineering, 2014, 30(14): 49–56. Guan Z X, Yao Q, Yang B J, et al. Application of digital image processing technology in recognizing the diseases, pests, and weeds from crops[J]. Scientia Agricultura Sinica, 2009, 42(7): 2349–2358. Duarte-Carvajalino J M, Alzate D F, Ramirez A A, et al. Evaluating late blight severity in potato crops using unmanned aerial vehicles and machine learning algorithms[J]. Remote Sensing, 2018, 10(10): 1513. Hasan R I, Yusuf S M, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: A broad analysis and discussion[J]. Plants, 2020, 9(10): 1302. Liu B, Zhang Y, He D J, et al. Identification of apple leaf diseases based on deep convolutional neural networks[J]. Symmetry, 2017, 10(1): 11. **e X, Ma Y, Liu B, et al. A deep-learning-based real-time detector for grape leaf diseases using improved convolutional neural networks[J]. Frontiers in plant science, 2020, 11: 751. Saleem M H, Khanchi S, Potgieter J, et al. Image-based plant disease identification by deep learning meta-architectures[J]. Plants, 2020, 9(11): 1451. Prashanthi V, Srinivas K. Plant disease detection using Convolutional neural networks[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2020, 9(3). Sourav M S U, Wang H. Intelligent identification of jute pests based on transfer learning and deep convolutional neural networks[J]. Neural Processing Letters, 2023, 55(3): 2193–2210. Li W, Shao S, Qiu Z, et al. Multi-perspective analysis on data augmentation in knowledge distillation[J]. Neurocomputing, 2024, 583: 127516. Zhao S, Sun X, Gai L. Data enhancement and multi-feature learning model for pest classification[J]. Journal of Intelligent & Fuzzy Systems, 2023 (Preprint): 1–13. Liu G, Nouaze J C, Touko Mbouembe P L, et al. YOLO-tomato: A robust algorithm for tomato detection based on YOLOv3[J]. Sensors, 2020, 20(7): 2145. Wu H, Wiesner-Hanks T, Stewart E L, et al. Autonomous detection of plant disease symptoms directly from aerial imagery[J]. The plant phenome journal, 2019, 2(1): 1–9. Mirzadeh S I, Farajtabar M, Li A, et al. Improved knowledge distillation via teacher assistant[C]//Proceedings of the AAAI conference on artificial intelligence. 2020, 34(04): 5191–5198. Chi Z, Zheng T, Li H, et al. Normkd: Normalized logits for knowledge distillation[J]. arxiv preprint arxiv:2308.00520, 2023. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 15 Jul, 2024 Reviews received at journal 15 Jul, 2024 Reviews received at journal 10 Jul, 2024 Reviewers agreed at journal 10 Jul, 2024 Reviewers agreed at journal 10 Jul, 2024 Reviewers invited by journal 10 Jul, 2024 Editor assigned by journal 10 Jul, 2024 Submission checks completed at journal 09 Jul, 2024 First submitted to journal 05 Jul, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4691672","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":326976463,"identity":"9b30de22-4f68-4a59-bb0c-7798d600f42c","order_by":0,"name":"Linan Wang","email":"","orcid":"","institution":"Central South University of Forestry and Technology","correspondingAuthor":false,"prefix":"","firstName":"Linan","middleName":"","lastName":"Wang","suffix":""},{"id":326976464,"identity":"0c7c67ea-580d-4b94-8ce7-b81be7ec2ea4","order_by":1,"name":"Hongmin Zhao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAUlEQVRIiWNgGAWjYDAC9saGAx8M/snxszcAeTYgoQT8Onh4Dh98OKPigLFkzwEgN82ACC0SacnGPGcOJG6YkUCkFnuGHDMJ3rY7iRskHz/+zJPwh4GfPceA4ecOPLYwnDGTkGx7ZrxdOs1MmifBgEGy540BY+8ZPFoYe8wkDNuYZXfOzmFj5v1hwGBwI8eAmbENjxZmHjOJxDZmxg03zzB/BtliT1ALG1uywYEzhxU33OBhADvMQIKQljPMBx82VKQBAznNTHJOgjGPxJlnBQd78Whhn/+w4fAfAxtgVB5+/OFNgpwcf3vyxgc/8WjBtBZEHCBBwygYBaNgFIwCLAAACENRPTXyssIAAAAASUVORK5CYII=","orcid":"","institution":"Central South University of Forestry and Technology","correspondingAuthor":true,"prefix":"","firstName":"Hongmin","middleName":"","lastName":"Zhao","suffix":""}],"badges":[],"createdAt":"2024-07-05 10:41:59","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4691672/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4691672/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61581098,"identity":"be7e7cf3-5d62-4ebc-9f9b-033b3776577e","added_by":"auto","created_at":"2024-08-01 13:28:01","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":914211,"visible":true,"origin":"","legend":"\u003cp\u003eData enhancement example\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/ccbf6a936a3535b99562164e.png"},{"id":61581102,"identity":"fa3ffeb3-78e0-4717-a0d3-fc5634834da6","added_by":"auto","created_at":"2024-08-01 13:28:01","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":272376,"visible":true,"origin":"","legend":"\u003cp\u003eA generic framework for knowledge distillation\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/3bb3ccc167c2f6596b7260e0.png"},{"id":61581099,"identity":"ed1c7335-b3f8-4611-9318-efb62fab42f0","added_by":"auto","created_at":"2024-08-01 13:28:01","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":16596,"visible":true,"origin":"","legend":"\u003cp\u003eTraining process after adding dynamic T module\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/046068e1f40e1e91623af140.png"},{"id":61581606,"identity":"23b1dc89-840d-467c-a41b-80661d12a263","added_by":"auto","created_at":"2024-08-01 13:36:01","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":24509,"visible":true,"origin":"","legend":"\u003cp\u003eAccuracy comparison before and after data augmentation (epoch=10)\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/82d47ab57e5426e7ba56479f.png"},{"id":61581607,"identity":"08e91da4-9441-4157-85f6-9ad5e58e1659","added_by":"auto","created_at":"2024-08-01 13:36:01","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":39297,"visible":true,"origin":"","legend":"\u003cp\u003e(a) Comparison of accuracy between teacher-student network and distilled student network (b) Comparison of DYTKD and KD results\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/2f84ae7d99c32ebb1b512d85.png"},{"id":61581608,"identity":"d124a36c-fa3d-4a26-b621-7e625daff597","added_by":"auto","created_at":"2024-08-01 13:36:01","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":40988,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix for classification of a: KD,b: DYTKD\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/86642e978967d23df0f35b06.png"},{"id":61582477,"identity":"ba2c4850-c184-43b9-8460-6dcda545d300","added_by":"auto","created_at":"2024-08-01 13:44:03","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2323198,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4691672/v1/b3ccf26a-c1b3-4ace-b3ec-c95f23ac01d7.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eIn agricultural production, pest control is essential to ensure stable crop yields. In recent years, especially in the planting process of China's four major staple grains - rice, wheat, corn, and potato, the frequent occurrence of major pests and diseases such as wheat scab and stripe rust, rice sheath blight, and corn spot disease has brought great challenges to agricultural production. These diseases not only directly threaten the safety of grain and oil production, but also may lead to a significant decline in crop yield and serious damage to quality, which in turn hurts the overall development of the agricultural industry. It is predicted that by 2024, the pest and disease situation of these food crops will be even more severe. It is estimated that the area of major pests and diseases of wheat, rice, corn, potatoes, and other food crops will reach a staggering 2.04\u0026nbsp;billion mu nationwide, an increase of 15% over the average level in 2023. This means that more than 70% of food crop-producing areas will be threatened by these pests and diseases, and agricultural production is facing unprecedented risks and challenges. To cope with this critical situation and improve agricultural production efficiency, timely detection and early prevention of plant diseases have become crucial.\u003c/p\u003e \u003cp\u003eWith the rapid development of artificial intelligence and computer technology, deep learning algorithms have shown strong potential in many fields, such as agriculture and forestry, medicine, finance, and other fields. Deep learning has become a popular research direction and has achieved excellent results in the fields of natural language processing (NLP), image processing, and object detection. In the field of pest identification, we try to use deep learning algorithms and computer vision technology to realize the automatic identification and classification of pest and disease images, which greatly improves the accuracy and efficiency of pest and disease identification.\u003c/p\u003e \u003cp\u003eAmong the many deep learning models, convolutional neural networks (CNNs) have performed well in the field of pest and disease identification due to their unique advantages. At present, a variety of CNN-based pest identification methods have been proposed, such as AlexNet, VGGNet, ResNet [\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e], etc., and these models have achieved excellent results in the field of pest and disease identification. For example, Cheng et al.[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]achieved remarkable results with the application of improved AlexNet in the classification of strawberry pests and diseases, with an accuracy rate of 98%. At the same time, the work of Alatawi et al.[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]also revealed the great potential of the VGGNet model in plant disease identification.\u003c/p\u003e \u003cp\u003eWith the deepening of research, it is found that the expressive ability of the model can be further improved by designing deeper neural network structures, to extract more discriminating depth features. However, this also poses a problem: a deeper network structure means more model parameters and higher computational complexity, as well as higher requirements for storage devices and computing resources. This makes existing deep learning models face certain challenges in practical applications, especially when deployed on end-user mobile devices with limited resources.\u003c/p\u003e \u003cp\u003eTo solve this problem, researchers have begun to explore how to reduce the complexity and computational cost of the model while ensuring the accuracy of the model[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Among them, the proposal of a knowledge distillation (KD) method[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] has attracted extensive attention. This approach guides the learning process of a smaller, less parameter-rich model (the student model) by utilizing a larger, parameter-rich model (the teacher model). During the training process, the teacher model \"distills\" the knowledge and feature information it has learned into the student model so that the student model has lower complexity and computational cost while maintaining high accuracy. To let the students better imitate the teacher, we also set the temperature parameter T to be dynamically adjusted according to the predicted value output by the student model and the loss value of the true label. This method can not only simplify the model structure on the premise of ensuring the accuracy of the model but also improve the feasibility and generalizability of the model in practical application. Therefore, the pest classification method based on knowledge distillation provides a new idea and method for solving practical problems in the identification of crop diseases and pests.\u003c/p\u003e \u003cp\u003eThe rest of this article is organized as follows: First, we reviewed the work in Section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. Then, in the third section, we will elaborate on the details of the ideas and models presented. We then present the experimental results and comparisons in Section \u003cspan refid=\"Sec8\" class=\"InternalRef\"\u003e4\u003c/span\u003e and finally summarize our work in Section \u003cspan refid=\"Sec11\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e"},{"header":"2. Related Work","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 The development process of crop pest and disease identification\u003c/h2\u003e \u003cp\u003eIn the early stages of crop pest identification, manual on-site inspection, measurement, and identification are mainly relied upon. This method is highly dependent on the experience and expertise of the observer, so it is highly subjective. Different experts may come to different conclusions on the same issue. In addition, manual identification is inefficient, time-consuming, and labor-intensive, making it difficult to cope with large-scale agricultural production needs.\u003c/p\u003e \u003cp\u003eTo improve the efficiency and accuracy of identification, researchers have gradually introduced various detection instruments. For example, Li et al. [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]developed a citrus pest detector. However, these testing instruments still need to be operated manually, the degree of automation and intelligence is generally low, and the detection results are easily affected by external environmental factors (such as temperature, light, etc.), which limits their wide application.\u003c/p\u003e \u003cp\u003eIn recent years, with the rapid development of image processing and pattern recognition technologies, experts and scholars have begun to apply these technologies to the identification of crop diseases and pests[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. At this stage, the color, texture morphology, and other features extracted from the image, combined with recognition technologies such as linear classifiers, are used to classify and identify various crop diseases and pests, which promotes the development of agricultural informatization and precision.\u003c/p\u003e \u003cp\u003eDuarte-Carvajalino et al.[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] aimed to identify potato diseases and used the collected datasets to train machine learning algorithms such as multilayer perceptrons, convolutional neural networks (CNNs), support vector machines, and random forests. The results show that the performance of the convolutional neural network is slightly better than that of other machine learning algorithms. There are many commonly used CNNs, each with its unique advantages[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e], such as VGG, ResNet, GoogleNet, and DenseNet. [\u003cspan additionalcitationids=\"CR14 CR15\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] provides detailed research and experimental results on the role of CNN in pest and disease identification, which provides an important reference for further research and application.\u003c/p\u003e \u003cp\u003eWith the continuous progress of science and technology, more and more scholars at home and abroad have begun to use deep learning technology to identify crop diseases and pests.[\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] Through automated and efficient image processing technology, target features can be accurately extracted from a large number of crop pest and disease images, thereby improving the accuracy and efficiency of identification. These technologies not only reduce the dependence on manual experience but also greatly improve the speed and reliability of pest and disease identification, providing strong support for agricultural production.\u003c/p\u003e \u003cp\u003eIn general, crop pest and disease identification technology has undergone a significant development process from early manual detection to the use of image processing and pattern recognition technology, and then to the widely used deep learning technology. With the continuous maturity of deep learning technology, the identification of crop diseases and pests will become more intelligent and efficient, providing strong technical support for the modernization and precision of agriculture.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Methods","content":"\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003e3.1 Data preprocessing\u003c/h2\u003e\n \u003cp\u003eData augmentation, as a key technology in the field of deep learning, is a method of extending a finite dataset by generating more equivalent data, aiming to increase the number and diversity of training samples, thereby improving the generalization ability and performance of the model. Li et al.[\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e] demonstrated through research that data augmentation significantly improves knowledge distillation performance on image classification and object detection tasks even though the teacher model lacks comprehensive information about augmented samples. With data augmentation, the over-reliance of the model on specific features can be reduced, thus avoiding overfitting. In deep learning, a network often has millions of parameters that require large amounts of data to train effectively. However, in some application scenarios, such as crop identification, the difficulty, and complexity of data collection are relatively high due to the influence of various factors on the growing environment of crops, and it is usually difficult to obtain enough data to support network training. Using raw data directly can lead to unbalanced sample classification and affect the training process of the model. Since the training goal is to improve the classification accuracy on the dataset, the category with a large sample size tends to dominate the training process due to its large cumulative training error, resulting in the algorithm biasing towards this category on unbalanced data. To improve the accuracy of the model and avoid overfitting the model, we can use data augmentation techniques to augment the sample[\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. In the application of crop identification, the growth environment of crops is affected by many factors, such as temperature and humidity, light, rainfall, soil, etc., and these objective conditions make data collection more difficult and complex. Therefore, data augmentation becomes an effective means to solve this problem.\u003c/p\u003e\n \u003cp\u003eCommon data augmentation methods include:\u003c/p\u003e\n \u003cp\u003eAdd noise: Introduce random noise into the input data to simulate real-world noise interference, making the model more robust to random changes in the data.\u003c/p\u003e\n \u003cp\u003eModify the intensity and contrast of the image: Enhance the adaptability of the model by adjusting the brightness, contrast, and color of the image to generate different viewing angles.\u003c/p\u003e\n \u003cp\u003eGeometric transformation: including rotation, clipping, and translation, by changing the geometric properties of the image, more training samples are generated. [\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e]\u003c/p\u003e\n \u003cp\u003eFor example, an example of a data augmentation method is shown in Fig. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, showing the leaves of four crop diseases, and the original image undergoes a series of transformations, from left to right, after 45\u0026deg; rotation, adjusting the brightness of the image, Gaussian blurring, color enhancement, and saturation increase. These newly generated images are treated as different samples in the network, which improves the recognition accuracy of the model and effectively mitigates the overfitting phenomenon. Through these methods, the model can better learn and identify diseases in complex and variable crop growth environments, and improve the overall classification performance and robustness.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n \u003ch2\u003e3.2 knowledge distillation\u003c/h2\u003e\n \u003cp\u003eIn 2015, Hinton et al. first proposed the Knowledge Distillation technique[\u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e] to solve the problem of increasing parameters as models become more complex. Knowledge distillation involves training the student network to mimic the output of the teacher\u0026apos;s network and mimic its internal characteristics or decision-making processes. The purpose of this technique is to enhance the performance of smaller models while making them behave close to larger models while reducing computational costs and memory requirements. The knowledge distillation system consists of three key components: knowledge, refining algorithms, and teacher-student architecture. Figure \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e illustrates the flowchart of knowledge distillation, with a \u0026quot;teacher-student\u0026quot; structure at its core.\u003c/p\u003e\n \u003cp\u003eThere are three different forms of knowledge in knowledge distillation: response-based knowledge, feature-based knowledge, and relationship-based knowledge. In this paper, we will use the logits of the last output layer of the teacher model as teacher knowledge[\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. This method is known as response-based knowledge distillation, and its main idea is to directly mimic the final predictions of the teacher model. Response-based knowledge distillation is a simple and effective model compression method that has been widely used in different tasks and applications.\u003c/p\u003e\n \u003cp\u003eThe Logits of the last output of the teacher model are used as the basis for comparing the output of the student model and the teacher model. By testing Logits, the model can measure the certainty or confidence of its predictions before applying the Softmax function to obtain the final probability. Specifically, the probability of the Softmax function is calculated as follows:\u003c/p\u003e\n \u003cdiv id=\"Equa\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e$$\\:\\begin{array}{c}{p}_{i}=\\frac{exp\\left(\\frac{zi}{T}\\right)}{\\sum\\:_{j}exp\\left(\\frac{zj}{T}\\right)}\\left(1\\right)\\end{array}$$\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003eHere, zi is the logit of class i, and the importance of the hyperparameter T is introduced to control the importance of each soft target, where T represents the \u0026quot;temperature\u0026quot; of knowledge distillation. When T\u0026thinsp;=\u0026thinsp;1, it corresponds to the normalized exponential function. As the temperature parameter T increases, the probability distribution of the Softmax function becomes smoother, conveying finer details about the relationship between the different classes in the teacher model.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003e3.3 Dynamically adjust the temperature\u003c/h2\u003e\n \u003cp\u003eThe soft tags described by Hinton et al. contain valuable dark knowledge from the teacher model. Thus, the loss function is extended to include the standard cross-entropy loss between the predictions of the student model and the true labels, as well as the difference between measuring the softening probability of the teacher\u0026apos;s model (obtained by a higher temperature Softmax) and the corresponding predictions of the student model.\u003c/p\u003e\n \u003cp\u003eThe loss value is used to measure the difference between the model prediction and the true label. The smaller the loss value, the more accurate the model prediction. At the same time, the T-parameter is an important hyperparameter in the model, which may affect the complexity, learning speed, or generalization ability of the model. Figure \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e is a flow chart of the network for training students. In the process of traditional knowledge distillation training, we use static T parameters, which cannot adjust the training state in time. [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e] It has been shown that the traditional fixed temperature setting may not be sufficient to capture the full knowledge of the teacher model during distillation. So this paper proposes a method to dynamically adjust the temperature parameter T according to the value of the student\u0026apos;s predicted value and the cross-entropy loss of the real label, to expect the model performance to achieve better training results.\u003c/p\u003e\n \u003cp\u003eTo implement this method, we only need to complete the following steps in the training process: first, set an initial parameter value T, secondly, calculate the cross-entropy loss of the prediction of the student model and the real label, and then set a threshold, and when the loss value is greater than the threshold, increase the parameter T appropriately; When the loss value is less than the threshold, the parameter T is lowered appropriately, and finally, we recalculate the distillation loss value with the T parameter. The combined loss function for distillation loss and student loss is determined as follows:\u003c/p\u003e\n \u003cdiv id=\"Equb\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e$$\\:\\begin{array}{c}L\\left(x;W\\right)=\\alpha\\:\\times\\:CE\\left(y,\\sigma\\:\\left({z}_{s};T=1\\right)\\right)+\\left(1-\\alpha\\:\\right)\\times\\:KL\\left(\\sigma\\:\\left({z}_{t};T=\\tau\\:\\right),\\sigma\\:\\left({z}_{s},T=\\tau\\:\\right)\\right)\\left(2\\right)\\end{array}$$\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003eIn this case, x represents the input data, W is the set of weight parameters for the student model, and y is the corresponding true class label. We introduce cross-entropy loss (CE) as part of supervised learning, which aims to optimize the accuracy of the student model for direct label prediction. Meanwhile, using a Softmax function \u0026sigma; regulated by a \u0026quot;temperature\u0026quot; parameter T, we transform the logits (Zt) of the teacher model into a smoother probability distribution, a process called \u0026quot;softening\u0026quot; the objective. The student model\u0026apos;s logits (Zs) try to match these softened targets, which goes beyond simple label prediction to learn the teacher model\u0026apos;s internal understanding and representation of the data.\u003c/p\u003e\n \u003cp\u003eTo further improve the performance of the student model, we design a comprehensive loss function to minimize the comprehensive loss function, which not only encourages the student model to learn to classify correctly, but also encourages it to learn the deep feature representation and generalization ability of the teacher model when dealing with complex data.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"4. Experiment","content":"\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n \u003ch2\u003e4.1 Dataset and preprocessing\u003c/h2\u003e\n \u003cp\u003eThe Rice Leaf Disease Images dataset was used in the experiment, including Bacterial blight, Blast, Brown spot, and Tungro. There were 4700 images in JPEG format. The size is 224x224 pixels. Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e shows our experimental condition configuration.\u003c/p\u003e\n \u003cp\u003eTo further enhance the generalization ability of the model, reduce the risk of overfitting, and improve the robustness of the model in the face of complex and variable environments, we introduce a data augmentation strategy. This strategy includes not only basic brightness and contrast adjustments, but also advanced techniques such as noise addition and image rotation to simulate real-world disturbances such as illumination changes, shooting Angle differences, and natural noise. Through these measures, the size of the original dataset was successfully doubled to 9400 images, and each image was carefully scaled and normalized, which laid a solid foundation for the subsequent training process.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eExperimental Configuration\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eExperimental Environment\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eValue\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eProgramming language\u003c/p\u003e\n \u003cp\u003eDeep learning framework\u003c/p\u003e\n \u003cp\u003eGPU\u003c/p\u003e\n \u003cp\u003eGPU acceleration tool\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePython 3.8\u003c/p\u003e\n \u003cp\u003ePyTorch 1.11.0\u003c/p\u003e\n \u003cp\u003eRTX 3080\u003c/p\u003e\n \u003cp\u003eCUDA:11.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eTo quantitatively evaluate the actual effect of data augmentation strategies, we design a series of rigorous comparison experiments. While keeping the key elements of the network architecture (ResNet34 as the teacher model and ResNet18 as the student model for knowledge distillation) and the optimization algorithm unchanged, we use the original dataset and the enhanced dataset for training respectively. The experimental results show that after using the enhanced dataset, the recognition accuracy of the model is significantly improved, from 90.98\u0026ndash;97.81%, with an increase of 6.83 percentage points. Figure \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e visually shows this significant performance improvement, which further confirms the excellent contribution of data augmentation in improving the robustness and accuracy of the model.\u003c/p\u003e\n \u003cp\u003eIn addition, we use more detailed evaluation metrics, including accuracy, precision, and recall, to fully understand the performance of the model.\u003c/p\u003e\n \u003cdiv id=\"Equc\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e$$\\:\\begin{array}{c}Accuracy=\\frac{TP+TN}{TP+TN+FP+FN}\\left(3\\right)\\end{array}$$\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Equd\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e$$\\:\\begin{array}{c}\\text{P}\\text{r}\\text{e}\\text{c}\\text{i}\\text{s}\\text{i}\\text{o}\\text{n}=\\frac{TP}{TP+FP}\\left(4\\right)\\end{array}$$\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Eque\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e$$\\:\\begin{array}{c}Recall=\\frac{TP}{TP+FN}\\left(5\\right)\\end{array}$$\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003eBy comparing the TP (true examples), FP (false positive examples), TN (true negative examples), and FN (false negative examples) data of the four pest categories before and after data augmentation in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, we found that the data augmentation strategy significantly reduced the misclassification (for example, the FP of Bacterial blight decreased from 87 to 4, and the FP of Bacterial blight decreased from 87 to 4. Tungro\u0026apos;s TP increased from 154 to 307), which greatly improved the accuracy of the model.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003ePerformance comparison before and after data enhancement (after knowledge distillation)\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colspan=\"5\"\u003e\n \u003cp\u003eOriginal data\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"4\"\u003e\n \u003cp\u003eEnhance data\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eClass\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e546\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e306\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e212\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e154\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e359\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e237\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e296\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e307\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e87\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e679\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e959\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1076\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1188\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e844\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e982\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e913\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e924\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAccuracy(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e91.28\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94.26\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.98\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e97.65\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrecision(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e86.26\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.81\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRecall(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94.79\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e85.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e84.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.75\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.67\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.68\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\n \u003ch2\u003e4.2 Experimental Results\u003c/h2\u003e\n \u003cp\u003eTaking the Resnet model as an example, we use the pre-trained Resnet34 as the teacher model on the rice dataset, and the accuracy can reach 98.94%. To further compress the model, we use knowledge distillation technology to pass the logits of the teacher output layer as knowledge to the lightweight student model Resnet18, and the accuracy is 97.81%. Compared with the Resnet18 model without distillation, the performance has been significantly improved. Figure 5(a) shows the comparison chart of teacher-student model accuracy.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eComparison of accuracy, precision, and recall of DYTKD\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eParameters(M)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTeacher: ResNet34\u003c/p\u003e\n \u003cp\u003eStudent: ResNet18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.94%\u003c/p\u003e\n \u003cp\u003e93.83%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.09\u003c/p\u003e\n \u003cp\u003e95.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.79\u003c/p\u003e\n \u003cp\u003e92.46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21.8\u003c/p\u003e\n \u003cp\u003e11.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eKD\u003c/p\u003e\n \u003cp\u003eDYTKD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e97.81%\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e99.11%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e97.7\u003c/p\u003e\n \u003cp\u003e99.15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e97.94\u003c/p\u003e\n \u003cp\u003e99.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11.2\u003c/p\u003e\n \u003cp\u003e11.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eTo verify the effectiveness of dynamically adjusting the parameter T, the module of dynamically adjusting T is introduced without changing the network structure, and other hyperparameters, Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, Table \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e and Fig. 5(b) are the comparison of the experimental results. The experimental results show that our proposed DYTKD method can fully let students learn the knowledge in the teacher network, and generalize well to the test data set. Not only the accuracy and precision are improved, but the number of true examples (TP) also increased from 367 to 382.\u003c/p\u003e\n \u003cp\u003eFigure 6(a) is the confusion matrix of the student model on the test set after knowledge distillation. It performs well in classifying these pests and diseases, but there is still room for improvement, especially in reducing misjudgment, among which there are 9 times that are Bacterial blight but predicted as Blast. Figure 6(b) is the confusion matrix of our proposed DYTKD method. Compared with the previous confusion matrix, the performance of the model has been further improved with higher accuracy and fewer misjudgments.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003cbr\u003e\n \u003ctable id=\"Tab4\" border=\"1\" style=\"margin-right: calc(15%); width: 85%;\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eComparison of TP, FP, TN, and FN of DYTKD\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colspan=\"8\" style=\"width: 60.5673%;\"\u003e\n \u003cp\u003eteacher\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"5\" style=\"width: 38.1885%;\"\u003e\n \u003cp\u003estudent\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"1\" style=\"width: 1.1016%;\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eClass\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eTP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e384\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e234\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e294\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e307\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e382\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e180\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e288\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e306\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eFP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e60\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eFN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e60\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eTN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e837\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e990\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e932\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e924\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e788\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e989\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e925\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e918\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eAccuracy(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e99.11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e99.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e94.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e99.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003ePrecision(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e97.22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e99.15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e86.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e98.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e97.63\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e98.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eRecall(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 5.5319%;\"\u003e\n \u003cp\u003e97.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.68\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e75.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e96.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 7.0992%;\"\u003e\n \u003cp\u003e99.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"1\" style=\"width: 1.1015%;\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"8\" style=\"width: 60.5673%;\"\u003e\n \u003cp\u003eKD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"6\" style=\"width: 39.149%;\"\u003e\n \u003cp\u003eDYTKD\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eClass\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBacterial blight\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBrown spot\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003eTungro\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e367\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e234\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e298\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e306\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e382\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e236\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e296\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e307\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e842\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e983\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e920\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e924\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e841\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e989\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e931\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e924\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAccuracy(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e98.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e98.78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e98.86\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.84\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e99.27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e99.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrecision(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e98.39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e96.30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e96.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e98.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.74\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.66\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRecall(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e95.57\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.3687%;\"\u003e\n \u003cp\u003e97.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e99.33\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003e99.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.33\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.67\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colspan=\"2\" style=\"width: 8.227%;\"\u003e\n \u003cp\u003e99.68\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u0026nbsp;\u003ctable id=\"Tab5\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eTeachers and students share the same architectures. ∆represents the performance improvement over the KD.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTeacher(%)\u003c/p\u003e\n \u003cp\u003eStudent(%)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResnet34\u003c/p\u003e\n \u003cp\u003e95.14\u003c/p\u003e\n \u003cp\u003eResnet18\u003c/p\u003e\n \u003cp\u003e91.77\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResnet50\u003c/p\u003e\n \u003cp\u003e94.84\u003c/p\u003e\n \u003cp\u003eResnet18\u003c/p\u003e\n \u003cp\u003e91.60\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResnet101\u003c/p\u003e\n \u003cp\u003e94.72\u003c/p\u003e\n \u003cp\u003eResnet34\u003c/p\u003e\n \u003cp\u003e90.67\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResnet152\u003c/p\u003e\n \u003cp\u003e94.77\u003c/p\u003e\n \u003cp\u003eResnet34\u003c/p\u003e\n \u003cp\u003e89.93\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eKD(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.76\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.17\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDYTKD(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.65\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94.93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e△(%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+\u0026thinsp;0.77\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+\u0026thinsp;1.17\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+\u0026thinsp;0.83\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+\u0026thinsp;0.79\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eTo further verify the effectiveness of the DYTKD method, we also carried out comparison tests of different networks and replaced the teacher-student network with different network structures, as shown in Table \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e. The experimental results show that the performance of DYTKD is improved compared with KD in each time.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"5. Conclusions","content":"\u003cp\u003eIn this paper, the recognition and classification performance of crop pests and diseases is significantly improved by effectively transferring the knowledge of large models to small models and combining data augmentation techniques. Taking the rice dataset as an example, our experimental results show that the loss threshold is set to dynamically adjust the temperature parameter during the training process. When the student predicted and true label cross-entropy loss values are large, it means that the student's student ability is weak, and we appropriately adjust to the appropriate temperature. Compared with the static temperature parameter in the traditional knowledge distillation method, the performance of the model is improved. Through this method, it provides strong technical support for the accurate identification and classification of crop pests and diseases.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eWang participated in the experiments, data analysis visualization and manuscript writing;Zhao gives final approval for what will be published.All authors reviewed the manuscript\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe datasets generated during and/or analysed during the current study are available in the [GitHub] repository, [https://github.com/wln130221/DYTKD]\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eKrizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012, 25.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSimonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arxiv preprint arxiv:1409.1556, 2014.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770\u0026ndash;778.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDong C, Zhang Z, Yue J, et al. Classification of strawberry diseases and pests by improved AlexNet deep learning networks[C]//2021 13th International Conference on Advanced Computational Intelligence (ICACI). IEEE, 2021: 359\u0026ndash;364.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlatawi A A, Alomani S M, Alhawiti N I, et al. Plant disease detection using AI based vgg-16 model[J]. International Journal of Advanced Computer Science and Applications, 2022, 13(4).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuan H, Fu C, Zhang G, et al. A lightweight model for efficient identification of plant diseases and pests based on deep learning[J]. Frontiers in Plant Science, 2023, 14: 1227011.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu Y, Gao Z, Zhai Y, et al. A CNNA-Based Lightweight Multi-Scale Tomato Pest and Disease Classification Method[J]. Sustainability, 2023, 15(11): 8813.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arxiv preprint arxiv:1503.02531, 2015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi Z, Hong T, Wang J, et al. Development and experiment of Panonychus citri infestation fast detector[J]. Transactions of the Chinese Society of Agricultural Engineering, 2014, 30(14): 49\u0026ndash;56.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuan Z X, Yao Q, Yang B J, et al. Application of digital image processing technology in recognizing the diseases, pests, and weeds from crops[J]. Scientia Agricultura Sinica, 2009, 42(7): 2349\u0026ndash;2358.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDuarte-Carvajalino J M, Alzate D F, Ramirez A A, et al. Evaluating late blight severity in potato crops using unmanned aerial vehicles and machine learning algorithms[J]. Remote Sensing, 2018, 10(10): 1513.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHasan R I, Yusuf S M, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: A broad analysis and discussion[J]. Plants, 2020, 9(10): 1302.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu B, Zhang Y, He D J, et al. Identification of apple leaf diseases based on deep convolutional neural networks[J]. Symmetry, 2017, 10(1): 11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003e**e X, Ma Y, Liu B, et al. A deep-learning-based real-time detector for grape leaf diseases using improved convolutional neural networks[J]. Frontiers in plant science, 2020, 11: 751.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSaleem M H, Khanchi S, Potgieter J, et al. Image-based plant disease identification by deep learning meta-architectures[J]. Plants, 2020, 9(11): 1451.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrashanthi V, Srinivas K. Plant disease detection using Convolutional neural networks[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2020, 9(3).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSourav M S U, Wang H. Intelligent identification of jute pests based on transfer learning and deep convolutional neural networks[J]. Neural Processing Letters, 2023, 55(3): 2193\u0026ndash;2210.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi W, Shao S, Qiu Z, et al. Multi-perspective analysis on data augmentation in knowledge distillation[J]. Neurocomputing, 2024, 583: 127516.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao S, Sun X, Gai L. Data enhancement and multi-feature learning model for pest classification[J]. Journal of Intelligent \u0026amp; Fuzzy Systems, 2023 (Preprint): 1\u0026ndash;13.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu G, Nouaze J C, Touko Mbouembe P L, et al. YOLO-tomato: A robust algorithm for tomato detection based on YOLOv3[J]. Sensors, 2020, 20(7): 2145.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu H, Wiesner-Hanks T, Stewart E L, et al. Autonomous detection of plant disease symptoms directly from aerial imagery[J]. The plant phenome journal, 2019, 2(1): 1\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMirzadeh S I, Farajtabar M, Li A, et al. Improved knowledge distillation via teacher assistant[C]//Proceedings of the AAAI conference on artificial intelligence. 2020, 34(04): 5191\u0026ndash;5198.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChi Z, Zheng T, Li H, et al. Normkd: Normalized logits for knowledge distillation[J]. arxiv preprint arxiv:2308.00520, 2023.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"discover-applied-sciences","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Discover Applied Sciences](https://link.springer.com/journal/42452)","snPcode":"42452","submissionUrl":"https://submission.springernature.com/new-submission/42452/3","title":"Discover Applied Sciences","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Data augmentation, knowledge distillation, Dynamic temperature, Image recognition and classification","lastPublishedDoi":"10.21203/rs.3.rs-4691672/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4691672/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn recent years, the output of China's four major crops has declined due to pests and diseases. This situation poses a serious challenge to food security. Therefore, timely detection and prevention of diseases is essential. First, we use data enhancement techniques to augment the data to improve the generalization ability of the model. Secondly, to reduce the model parameters and facilitate the deployment at the terminal, we use the knowledge distillation method. Finally, a method of dynamically adjusting the parameter T according to the loss value (DYTKD) is proposed to improve the performance of the model further. The experiment shows that knowledge distillation can reduce the number of parameters while making the accuracy of the student model as close as possible to the teacher model 98.94%. Meanwhile, data augmentation can also improve the accuracy of the model by 6.83%. Compared with the basic knowledge distillation method, the accuracy of DYTKD was increased by 1.3% without changing the student network and other parameters, and the accuracy of pest identification and classification was effectively improved. Among 1342 pest pictures, 1221 were correctly identified and accurately classified. Our codes are available at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/wln130221/DYTKD\u003c/span\u003e\u003cspan address=\"https://github.com/wln130221/DYTKD\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e","manuscriptTitle":"Application of knowledge distillation method with dynamic adjustment of temperature parameters in pest classification","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-01 13:27:56","doi":"10.21203/rs.3.rs-4691672/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-07-15T07:30:36+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-15T07:10:34+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-10T11:55:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"55737928503105411052591614522947694052","date":"2024-07-10T11:42:31+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"123352257302922032200052331352383160817","date":"2024-07-10T11:31:43+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-07-10T11:09:28+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-10T10:53:22+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-07-09T12:39:25+00:00","index":"","fulltext":""},{"type":"submitted","content":"Discover Applied Sciences","date":"2024-07-05T10:40:39+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"discover-applied-sciences","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Discover Applied Sciences](https://link.springer.com/journal/42452)","snPcode":"42452","submissionUrl":"https://submission.springernature.com/new-submission/42452/3","title":"Discover Applied Sciences","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e03d1e8b-5e95-4617-a8d5-3c8329e1912b","owner":[],"postedDate":"August 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2024-08-06T05:42:19+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-01 13:27:56","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4691672","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4691672","identity":"rs-4691672","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}