An novel circuit board fault diagnosis network using infrared thermal image

An novel circuit board fault diagnosis network using infrared thermal image | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An novel circuit board fault diagnosis network using infrared thermal image Song Xudong, chao liu, Weiguo Yi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3932781/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In practical industrial applications, infrared thermography fault diagnosis has received widespread attention. In order to detect the internal component faults of circuit boards, this paper proposes an novel circuit board fault diagnosis network using infrared thermal image.Firstly, a network LGC-Net network is proposed to improve ResNet50, and a two-layer attention module and a channel feature extraction module are introduced into the ResNet50 network, which strengthens the network's ability to extract global features and improves the model's extraction of features such as multi-scale key detail features of the input feature maps and the target contour, etc. Optimization of the residual structure and model pre-training plus fine-tuning are used for the Diagnostic model construction, so that the accuracy of fault diagnosis has been greatly improved. Finally, the acquired board infrared thermography dataset is preprocessed and experimented with a diagnostic fault network Experiments show that the LGC-Net network proposed in this paper has a fault diagnosis accuracy of 98.92%, which is 1.9% higher than the optimal accuracy of the classical network fault diagnosis, while the diagnosis time of a single infrared thermography is 246ms, which is 299ms lower than the shortest time consumed by other models for diagnosis.The experimental results show that the method proposed in this paper is able to efficiently identify infrared thermography faults of circuit board components, to improve the accuracy of circuit board fault diagnosis. Infrared thermal imaging ResNet50 residual structure attention mechanism Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 1 Introduction With the rapid development of electronic technology, the application range of circuit boards is more and more extensive [ 1 ] . As the core of hardware system, circuit board components will inevitably have various failures. These faults will affect the normal operation of the equipment, and at the same time, due to the increasingly complex function of the circuit board, its structure is developing in the direction of multi-layer, large component density and large scale [ 2 ] , resulting in difficulties in fault diagnosis. Therefore, a fast, intuitive and accurate circuit board fault detection method with low threshold is urgently needed. As a non-destructive testing technology, infrared thermal imaging detection technology has been successful in many fields, and has been gradually applied to the fault detection of circuit boards. In recent years, in the field of infrared image fault diagnosis, Li Changwen et al built a fault diagnosis network of shearer rocker arm gear based on ResNet, which has better performance in fault classification accuracy and network training speed [ 3 ] . Zhao Yan et al. constructed an image enhancement method combining weighted guided filtering layering and improved SSR, used weighted guided filtering to separate the base layer of the image, and enhanced it by improved SSR algorithm [ 4 ] . It can achieve the purpose of noise reduction. The emergence of ResNet network makes the fault diagnosis network layer deeper, enhances the complex computing power of the network, and realizes a great breakthrough in the field of fault identification. The main contributions of this paper are as follows: A)The infrared thermal imaging of the circuit boards that reach the moment of stable operation after powering up is acquired, and the data is expanded using horizontal flipping, random vertical flipping, and the addition of Gaussian noise to enhance the model generalization. B)For the problems of low pixel density, unclear boundaries, and low signal-to-noise ratio in the chip area of the original infrared circuit board acquired, an image enhancement algorithm is utilized to improve the quality of the infrared image in order to enhance the accuracy of fault diagnosis. C)By introducing a multi-head attention mechanism, the 3×3 convolutional layer of the residual block in the original network model is substituted with MHSA to construct a dual-attention feature extraction network to improve the feature extraction capability of infrared thermal imaging. D) The 3×3 maximum pooling is introduced into the shortcut connection of the bottleneck block of Layer3 downsampling to reduce the loss of information in the downsampling process and enhance the transform invariance of the input features. Softmax function is used to predict the belonging state of circuit board infrared thermal imaging. 2 Proposed circuit board fault diagnosis method 2.1 Infrared thermal imaging circuit board fault diagnosis overall process The flow of the proposed infrared thermal imaging circuit board fault diagnosis method based on improved ResNet50 is shown in Fig. 1 . First of all, an infrared thermal imager is used to capture thermal images of the infrared circuit board required for the experiment, complete the production of the infrared circuit board thermal imaging data set and data enhancement of the data set, and complete the task of data set division. Due to the noise and blurred edge details in the captured infrared images, image enhancement is also required to enhance the brightness and contrast, which provides high-quality images for later image fault diagnosis. Provide model selection and performance comparison, implement ResNet50 base network improvement, design a global feature extraction network LGC-Net model by introducing channel feature extraction mechanism (SE) and multi-head attention mechanism (MHSA),and improve the downsampling operation of Bottleneck down shortcut connection in Layer3 of its network model, and reduce the downsampling operation of shortcut connection in the downsampling operation of the downsampling operation of the downsampling operation of the downsampling operation of the shortcut connection in the downsampling operation of the downsampling operation in the downsampling operation in the network model by introducing Maximum pooling reduces the information loss of shortcut connections in the downsampling process and improves the network's ability to capture local features. The introduction of migration learning can make up for the shortcomings of the self-built dataset in this paper, first use a large number of high-quality infrared thermal imaging circuit boards for the first training, so that the model obtains the feature extraction capability of infrared thermal imaging circuit board faults, and then migrate the model to the self-built dataset to carry on the migration learning, and finally get the optimal fault diagnostic accuracy by selecting the appropriate learning rate. 2.2 Network structure improvement ResNet, a deep convolutional neural network architecture proposed by Microsoft Research in 2015 [ 5 ] , introduces a residual module into the network, in which a jump connection is added between the input and output to add the input directly to the output. This jump connection design allows the network to learn the residuals more easily, thus solving the problem of vanishing gradients and model degradation. Because the overall parameters of the ResNet50 model are moderate, and it has good network performance ability, this paper specially selects ResNet50 as the base model and carries out improvement operations on it.For the improvement of the base network, LGC-Net, an infrared thermal imaging fault diagnosis network, is designed by introducing the SE attention module and the MHSA attention module [ 6 ] into the base network. LGC-Net retains the first layer of the base network, adds the SE attention module to the second and third feature extraction layers, and introduces the MHSA attention module to the fourth and fifth feature extraction layers, so as to improve the key point extraction capability. The Softmax function is used to predict the belonging state of the circuit board infrared thermal imaging [ 7 ] . 2.3 Optimizing residuals The original base model in Bottleneck down shortcut connection, the down sampling operation of the input feature map with 1×1 convolution with a step size of 2 will have the problem that 3/4 of the input feature mapping is ignored, which may lead to the loss of part of the important information, affecting the extraction of fault features [ 8 ] . In contrast, maximum pooling can retain more effective information, alleviate the over-sensitivity of the convolutional layer to the position, and enhance the transform invariance of the features by taking the maximum value of the feature points in the neighborhood [ 9 ] . Therefore, this paper improves on the residual structure of the original model, and improves the downsampling operation of Bottleneck down shortcut connection in its Layer3, which can greatly reduce the information loss of the shortcut connection in the downsampling process by introducing the maximum pooling, and improves the network's ability to capture the local features [ 10 ] , so as to pass more useful information downwards. The improved ResNet50 bottleneck block structure is shown in Fig. 2 . 3 Experimental data 3.1 Data Collection Constructing a dataset is a necessary part of deep learning. Because there is no publicly available infrared thermal imaging dataset for circuit boards, this paper uses a homemade dataset for fault recognition experiments. Under the condition of ensuring that the ambient temperature is basically unchanged, this experiment uses InfiRay Xtherm II T2L infrared thermal camera to photograph the circuit boards respectively, in order to ensure the validity of the experimentally acquired data, in the same working environment, the same excitation signal is applied to all the boards, and the infrared thermal imaging of the full-load state is achieved after a period of time of power on, including six states: five fault states are set up by human beings and 1 normal state each state are collected 100 saturated state of infrared thermal imaging, a total of 600 collected. The acquisition system consists of an infrared thermal imaging camera, a microcontroller timing control device, a relay, a computer, and a computer. Infrared thermal imaging data acquisition is shown in Fig. 3 . 3.2 Data Enhancement Data augmentation is a technique to expand the training dataset by generating samples with the same labels from the same source data for the purpose of improving the robustness of the deep learning model [ 11 ] .The data augmentation methods used include random horizontal flipping, random vertical flipping, and adding Gaussian noise. The overall number of data samples is brought to 2681 and the dataset is divided as shown in Table 1 . The training data during model training is divided into 70% training set and 30% validation set. 3.3 Image Enhancement Acquired infrared images have the disadvantages of low imaging resolution, poor uniformity and stereoscopic sense, low signal-to-noise ratio, low contrast, easy to carry noise, blurred and missing edge details. Image enhancement is a process of enhancing a low definition infrared thermography image into a high definition infrared thermography image [ 12 ] . Through the acquisition of the infrared thermal imaging circuit board after the feature enhancement to highlight the characteristics of the components, the pixel points of infrared thermal imaging (x, y) horizontally and numerically in the direction of the operation, in order to use its effective image for feature extraction, choose the first-order infrared thermal imaging circuit boards, and finally obtain the gradient of the information on the gradient. For infrared thermal imaging circuit board first order derivatives are processed as shown in Eqs. 1 and 2 . $${G}_{X}\left(x,y\right)=\frac{\left(f\left(x,y+1\right)-f\left(x,y\right)\right)+\left(f\left(x+1,y+1\right)-f\left(x+1,y\right)\right)}{2}$$ 1 $${G}_{y}(x,y)=\frac{\left(f\left(x,y\right)-f\left(x+1,y\right)\right)+(f\left(x,y+1\right)-f\left(x+1,y+1\right))}{2}$$ 2 where \({\text{G}}_{\text{x}}\) (x,y) and \({\text{G}}_{\text{y}}\) (x,y) are used to denote the first order derivatives in the x and y directions. The magnitude G(x,y) and \({\theta }(\text{x},\text{y})\) direction results of the edge gradient are shown in Eq. 3 and Eq. 4 . $$G(x,y)=\sqrt{{{G}_{X}}^{2}+{{G}_{y}}^{2}}$$ 3 $$\theta (x,y)=\text{a}\text{r}\text{c}\text{t}\text{a}\text{n}\left(\frac{{G}_{X}}{{G}_{y}}\right)$$ 4 The image brightness and contrast enhancement after image enhancement is obvious, and the details of the image obtained through image enhancement are clearer. The results show that the algorithm in this section can effectively enhance the pixel density, improve the detail information and enhance the visual effect. Finally, in order to improve the training speed of the network, the pixel values of infrared thermal imaging are normalized. The image preprocessing provides high quality images for later image fault diagnosis. The original image is shown in Fig. 4,the image of the IR board after image enhancement is shown in 5. 4Experiment and Analysis 4.1 Experimental environment and parameter settings The experimental environment is based on Python3.8 and PyTorch1.10 deep learning framework, the experimental model is trained on NVIDIA GeForce GTX 3060 GPU, the optimizer is set to Adam, the loss function is set to cross-entropy loss, the Batchsize is set to 16, the number of iterations is 70, and the learning rate is set to 0.001. The formula for cross-entropy loss function is shown in Eq. 5 . loss function is calculated as shown in Eq. 5 . $$L=\frac{1}{N}\sum _{i=1}^{N}-\left[{y}_{i}\text{log}{(p}_{i}\right)+(1-{y}_{i})\text{log}{(p}_{i}\left)\right]$$ 5 With Pi representing the size of the probability that the model predicts class i, N represents the total number of samples, and yi denotes the categorical label of sample i. 4.2 Proposed diagnosis method ablation experiment In order to verify the effectiveness of the fault diagnosis method in this paper, and at the same time to explain scientifically the role of the improved structure in the model performance improvement, this chapter takes ResNet50 as the base network to be improved and conducts structural ablation experiments on the improved model, and then tests it on a test set to obtain the various evaluation indexes. Image enhancement can effectively improve the model performance, in order to explore the effect of the above approach on the performance of the improved ResNet50 network model, set the same experimental conditions, the improved ResNet50 model with image enhancement achieves a detection time of 246ms per image, an accuracy of 98.92% on the validation set, and an accuracy of 98.66% on the test set, which is 1.66% higher than that of the one without image enhancement, respectively. Fault diagnosis accuracy is 1.27% and 1.51% higher,thus proving that image enhancement has a significant effect on the model performance. The results of the ablation experiment are shown in Table 2 . From the table, it can be seen that all parts of the improvement have some effect on the results of the base experiment. By adding SE-Net to the base network and MHSA for analysis and comparison, it is finally concluded that the fault diagnosis accuracy of LGC-Net network in the test set is 98.92%, which is higher than that of the base ResNet50 model by 4.86%, which proves that the improvement of the circuit board fault diagnosis method of ResNet50 infrared thermal imaging is practicable. Table 2 Results of network ablation experiments mould Accuracy /% Precsion /% Recall /% F1-score/% ResNet50 94.06 94.24 94.06 94.12 SE-ResNet50 95.07 95.67 95.07 95.11 MHSA-ResNet50 97.02 97.16 97.02 97.02 LGC-Net 98.92 98.97 98.92 98.92 4.3 Comparison Experiments of the Proposed Diagnostic Methods In order to verify the superiority of the proposed fault diagnosis method in dealing with the classification task of infrared circuit board image dataset, under the same experimental conditions, three traditional classical network models, VGG16, GoogLeNet, and ResNet18, are selected for performance comparison.The model parameters are set up with the number of training rounds as 70, the initial value of the learning rate is set as 0.001, and the cosine annealing decay strategy is utilized during the training process For updating, it can be seen from Table 3 that the verification accuracy of the proposed improved ResNet50 network is significantly improved over the traditional network model, and compared to the other networks, the improved ResNet50 network has less number of parameters, and its infrared thermography diagnosis time is 246ms, which is much lower than the other models. Therefore, the improved ResNet50 network proposed in this paper is able to identify infrared thermal imaging circuit board faults more accurately and has a strong generalization ability, which provides excellent performance compared to other methods. Table 3 Comparison of the performance of different models methodologies Precision /% Recall /% Accuracy /% Time(ms) ResNet18 92.06 96.89 92.06 856 VGG16 97.02 97.16 97.02 545 GoogLeNet 94.12 94.02 94.02 703 LGC-Net 98.97 98.92 98.92 246 4.4 Model training evaluation During the training process the validation set curve has a larger oscillation in the pre-experiment, by fine-tuning the hyper-parameters, so that the model maintains the optimal state, as shown in Fig. 6 Loss curve diagram, the horizontal coordinate is the number of rounds of model training, and the vertical coordinate is the Loss value of the corresponding rounds, and the model gradually reaches the optimal performance in the late stage of training. As shown in Fig. 7 , the accuracy curve shows that the accuracy fluctuates greatly in the first 40 epochs of the validation set during the model training process, and the accuracy tends to stabilize in the last 30 epochs and is very close to the accuracy curve of the training set, which reflects the validity of the method of this chapter in infrared thermal imaging-oriented circuit board fault diagnosis, and the performance of the fault diagnostic model is more stable, with Robustness. 5 Conclusion In this paper, by improving the ResNet50 network and embedding the SE attention module and the MHSA attention module into it, we design the infrared image fault diagnosis recognition network LGC-Net, which consists of a feature extraction part and a classification part to complete the tasks of image feature extraction and infrared image fault diagnosis and recognition, respectively, and introduces the maximal pooling to introduce Layer3. By introducing maximum pooling, the Layer3 bottleneck block is quickly connected to reduce the loss of information in the process of downsampling, enhance the transform invariance of the input features, and effectively improve the fault diagnosis accuracy of infrared thermal images. Under the same experimental conditions, by comparing ResNet18/50, VGG16 and GoogLeNet, the improved network has the best diagnostic accuracy, and the average diagnostic time is only 246ms, which is much lower than other models. However, infrared thermography, as one of the many diagnostic methods, can not completely diagnose all circuit board faults, and in the future, we can explore the use of a variety of data forms for comprehensive diagnosis of circuit board faults, the experiments in this paper used a self-built dataset, and the amount of data collected is relatively small, and future work needs to obtain more fault types of data to improve the quality of infrared thermography. Declarations Author Contribution a,Significant revisions to improve articles and guide the completion of the overall workb,Perform dataset collection, network construction, and experimental collation processc,Guided the overall flow of the experiment smoothly, with important modifications for network improvement References Sida. Research on circuit board fault detection method based on infrared thermal imaging technology [D]. Taiyuan University of Science and Technology, 2016. Cui Haohao,Zhang Zhipeng,Liu Yanfeng et al. Fault detection method for circuit boards based on infrared images [J]. Information Technology and Informatization, 2023, (09): 139-142. Li Changwen, Cheng Zeyin, Zhang Xiaogang et al. Fault diagnosis of coal mining machine rocker gear based on deep residual network [J]. Industrial and Mining Automation, 2021, 47 (03): 71-78. DOI:10.13272/j.issn.1671-251x.2020110043 Zhao Yan. Deep learning based infrared image recognition and thermal fault diagnosis of electrical equipment [D]. North China Electric Power University, 2022. doi:10.27139/d.cnki.ghbdu.2022.000089 He K,Zhang X,Ren S, et al. Deep Residual Learning for Image Recognition.[J]. CoRR,2015, abs /1512.03385. SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021: 16519-16529. SUNKARA R，LUO T.No more strided convolutions or pooling：a new CNN building block for low- resolution images and small objects[J].arXiv：2208.03641，2022. R. Zhong,B. Jiang,N. Li et al. Face expression recognition method embedded with attention mechanism residual network [J]. Computer Engineering and Applications, 2023, 59 (11): 88-97. Heavy rain Xuan, Lu Tianliang, Du Yanhui et al. Deep forgery video detection method based on i_ResNet34 model and data enhancement [J]. Computer Science, 2021, 48 (07): 77-85. JIA Shuaikang, SUN Hairong, SU Zifan. Study on hot spot state classification of infrared images by improved residual network[J]. Industrial Control Computer,2021,34(2):79-82. ZHANG Qian,YANG Ying,LIU Gang,et al. Fusion of data enhancement and improved ResNet34 for mastitis detection in thermal infrared images of dairy cows[J]. Spectroscopy and Spectral Analysis, 2023, 43(1): 9. Jeon M, Yoo S, Kim S W. A contactless PCBA defect detection method: Convolutional neural networkswith thermographic images[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2022, 12(3): 489-501. Table Table 1 is available in the Supplementary Files section. Additional Declarations No competing interests reported. Supplementary Files Table1Resultsofdatadivision.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3932781","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":271360996,"identity":"b5194da7-3d42-4006-8c30-d16466e23172","order_by":0,"name":"Song Xudong","email":"","orcid":"","institution":"Dalian Jiaotong University, Computer Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Song","middleName":"","lastName":"Xudong","suffix":""},{"id":271360997,"identity":"e16a32b3-c46b-4a34-9cc9-c82a4919bc2f","order_by":1,"name":"chao liu","email":"","orcid":"","institution":"Dalian Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"chao","middleName":"","lastName":"liu","suffix":""},{"id":271360998,"identity":"7c7eb47b-6ed4-4fe2-87c2-462ec8863f3c","order_by":2,"name":"Weiguo Yi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0UlEQVRIiWNgGAWjYBACxmYGZjCDn735wIEPP0jRItlzLPHgzB7iLIJoMbiRY3yYg40Y9e3Mhw0+th3OMzhz5sNhBh4GeX6xA4QcxpacOOPM4WLJ470bDhdYMBjOnJ1ASAuP8WGeisOJfWfObjg8g4chweA2QS38nw//MTic2HAj58FhHjaitPAwJzMAbZlwI4eBWC1sxoY9Z9ITZ/YcMwAGsgRhvxj2H34s8bPNOrGfvfnxhw8/bOT5pQlpaUDlS+BXDgLyhJWMglEwCkbBiAcAmMNLZbGafLUAAAAASUVORK5CYII=","orcid":"","institution":"Dalian Jiaotong University","correspondingAuthor":true,"prefix":"","firstName":"Weiguo","middleName":"","lastName":"Yi","suffix":""}],"badges":[],"createdAt":"2024-02-06 04:29:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3932781/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3932781/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":50877084,"identity":"bf904fe9-27e7-46eb-b1e0-3d509074faa7","added_by":"auto","created_at":"2024-02-08 19:17:00","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":197738,"visible":true,"origin":"","legend":"\u003cp\u003efault diagnosis flow of infrared thermal imaging circuit boards\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/98f84b25d7875453bc1ad2b3.jpg"},{"id":50877081,"identity":"f74df99c-0e07-474c-94ea-ad2038e3febc","added_by":"auto","created_at":"2024-02-08 19:17:00","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":44541,"visible":true,"origin":"","legend":"\u003cp\u003eImprovement of ResNet50 bottleneck block architecture\u003c/p\u003e","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/2c52baea47a8fa317c5e009c.jpg"},{"id":50877542,"identity":"fc39452a-fd48-4a19-be86-32910094777c","added_by":"auto","created_at":"2024-02-08 19:25:00","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":65509,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic diagram of infrared thermal imaging data acquisition\u003c/p\u003e","description":"","filename":"Figure3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/c83d743470ed39d770d8b790.jpg"},{"id":50877082,"identity":"9e6abe67-0019-4136-a01c-1a747b270704","added_by":"auto","created_at":"2024-02-08 19:17:00","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":15111,"visible":true,"origin":"","legend":"\u003cp\u003eOriginal diagram\u003c/p\u003e","description":"","filename":"Figure4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/164db25a475fbfc4b7fd5697.jpg"},{"id":50877086,"identity":"1b50e887-efc1-4f2e-99e5-8a98a7192a5a","added_by":"auto","created_at":"2024-02-08 19:17:00","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":15294,"visible":true,"origin":"","legend":"\u003cp\u003eAfter image enhancement\u003c/p\u003e","description":"","filename":"Figure5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/330d421cb89beb70febb8deb.jpg"},{"id":50877544,"identity":"1bda3a75-53a9-4d2d-bb8f-1b6692d0abaa","added_by":"auto","created_at":"2024-02-08 19:25:00","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":33279,"visible":true,"origin":"","legend":"\u003cp\u003eLoss Curve\u003c/p\u003e","description":"","filename":"Figure6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/0cf70e7a707768a50a33e335.jpg"},{"id":50877543,"identity":"790b30f3-7d39-487f-93ff-da34c34e29cb","added_by":"auto","created_at":"2024-02-08 19:25:00","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":37724,"visible":true,"origin":"","legend":"\u003cp\u003eAccuracy curve\u003c/p\u003e","description":"","filename":"Figure7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/92b77de300bbd40ee47f622c.jpg"},{"id":51151660,"identity":"0e581fc3-993c-4e0a-b36c-5cc5c7e848c0","added_by":"auto","created_at":"2024-02-15 04:08:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":583108,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/c1c5617f-7dcc-4346-b855-58cfbbe0c165.pdf"},{"id":50877080,"identity":"3e3fe08d-27c4-4093-a479-b04d811d66fd","added_by":"auto","created_at":"2024-02-08 19:17:00","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":132674,"visible":true,"origin":"","legend":"","description":"","filename":"Table1Resultsofdatadivision.docx","url":"https://assets-eu.researchsquare.com/files/rs-3932781/v1/c57f72b4c31abe2d1fce6745.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"An novel circuit board fault diagnosis network using infrared thermal image","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eWith the rapid development of electronic technology, the application range of circuit boards is more and more extensive \u003csup\u003e[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]\u003c/sup\u003e. As the core of hardware system, circuit board components will inevitably have various failures. These faults will affect the normal operation of the equipment, and at the same time, due to the increasingly complex function of the circuit board, its structure is developing in the direction of multi-layer, large component density and large scale \u003csup\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/sup\u003e, resulting in difficulties in fault diagnosis. Therefore, a fast, intuitive and accurate circuit board fault detection method with low threshold is urgently needed. As a non-destructive testing technology, infrared thermal imaging detection technology has been successful in many fields, and has been gradually applied to the fault detection of circuit boards. In recent years, in the field of infrared image fault diagnosis, Li Changwen et al built a fault diagnosis network of shearer rocker arm gear based on ResNet, which has better performance in fault classification accuracy and network training speed\u003csup\u003e[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/sup\u003e. Zhao Yan et al. constructed an image enhancement method combining weighted guided filtering layering and improved SSR, used weighted guided filtering to separate the base layer of the image, and enhanced it by improved SSR algorithm\u003csup\u003e[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]\u003c/sup\u003e. It can achieve the purpose of noise reduction. The emergence of ResNet network makes the fault diagnosis network layer deeper, enhances the complex computing power of the network, and realizes a great breakthrough in the field of fault identification.\u003c/p\u003e \u003cp\u003eThe main contributions of this paper are as follows:\u003c/p\u003e \u003cp\u003eA)The infrared thermal imaging of the circuit boards that reach the moment of stable operation after powering up is acquired, and the data is expanded using horizontal flipping, random vertical flipping, and the addition of Gaussian noise to enhance the model generalization.\u003c/p\u003e \u003cp\u003eB)For the problems of low pixel density, unclear boundaries, and low signal-to-noise ratio in the chip area of the original infrared circuit board acquired, an image enhancement algorithm is utilized to improve the quality of the infrared image in order to enhance the accuracy of fault diagnosis.\u003c/p\u003e \u003cp\u003eC)By introducing a multi-head attention mechanism, the 3\u0026times;3 convolutional layer of the residual block in the original network model is substituted with MHSA to construct a dual-attention feature extraction network to improve the feature extraction capability of infrared thermal imaging.\u003c/p\u003e \u003cp\u003eD) The 3\u0026times;3 maximum pooling is introduced into the shortcut connection of the bottleneck block of Layer3 downsampling to reduce the loss of information in the downsampling process and enhance the transform invariance of the input features. Softmax function is used to predict the belonging state of circuit board infrared thermal imaging.\u003c/p\u003e"},{"header":"2 Proposed circuit board fault diagnosis method","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Infrared thermal imaging circuit board fault diagnosis overall process\u003c/h2\u003e \u003cp\u003eThe flow of the proposed infrared thermal imaging circuit board fault diagnosis method based on improved ResNet50 is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. First of all, an infrared thermal imager is used to capture thermal images of the infrared circuit board required for the experiment, complete the production of the infrared circuit board thermal imaging data set and data enhancement of the data set, and complete the task of data set division. Due to the noise and blurred edge details in the captured infrared images, image enhancement is also required to enhance the brightness and contrast, which provides high-quality images for later image fault diagnosis. Provide model selection and performance comparison, implement ResNet50 base network improvement, design a global feature extraction network LGC-Net model by introducing channel feature extraction mechanism (SE) and multi-head attention mechanism (MHSA),and improve the downsampling operation of Bottleneck down shortcut connection in Layer3 of its network model, and reduce the downsampling operation of shortcut connection in the downsampling operation of the downsampling operation of the downsampling operation of the downsampling operation of the shortcut connection in the downsampling operation of the downsampling operation in the downsampling operation in the network model by introducing Maximum pooling reduces the information loss of shortcut connections in the downsampling process and improves the network's ability to capture local features. The introduction of migration learning can make up for the shortcomings of the self-built dataset in this paper, first use a large number of high-quality infrared thermal imaging circuit boards for the first training, so that the model obtains the feature extraction capability of infrared thermal imaging circuit board faults, and then migrate the model to the self-built dataset to carry on the migration learning, and finally get the optimal fault diagnostic accuracy by selecting the appropriate learning rate.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Network structure improvement\u003c/h2\u003e \u003cp\u003eResNet, a deep convolutional neural network architecture proposed by Microsoft Research in 2015 \u003csup\u003e[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/sup\u003e, introduces a residual module into the network, in which a jump connection is added between the input and output to add the input directly to the output. This jump connection design allows the network to learn the residuals more easily, thus solving the problem of vanishing gradients and model degradation. Because the overall parameters of the ResNet50 model are moderate, and it has good network performance ability, this paper specially selects ResNet50 as the base model and carries out improvement operations on it.For the improvement of the base network, LGC-Net, an infrared thermal imaging fault diagnosis network, is designed by introducing the SE attention module and the MHSA attention module\u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/sup\u003e into the base network. LGC-Net retains the first layer of the base network, adds the SE attention module to the second and third feature extraction layers, and introduces the MHSA attention module to the fourth and fifth feature extraction layers, so as to improve the key point extraction capability. The Softmax function is used to predict the belonging state of the circuit board infrared thermal imaging\u003csup\u003e[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Optimizing residuals\u003c/h2\u003e \u003cp\u003eThe original base model in Bottleneck down shortcut connection, the down sampling operation of the input feature map with 1\u0026times;1 convolution with a step size of 2 will have the problem that 3/4 of the input feature mapping is ignored, which may lead to the loss of part of the important information, affecting the extraction of fault features\u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/sup\u003e. In contrast, maximum pooling can retain more effective information, alleviate the over-sensitivity of the convolutional layer to the position, and enhance the transform invariance of the features by taking the maximum value of the feature points in the neighborhood\u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e. Therefore, this paper improves on the residual structure of the original model, and improves the downsampling operation of Bottleneck down shortcut connection in its Layer3, which can greatly reduce the information loss of the shortcut connection in the downsampling process by introducing the maximum pooling, and improves the network's ability to capture the local features\u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]\u003c/sup\u003e, so as to pass more useful information downwards. The improved ResNet50 bottleneck block structure is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"3 Experimental data","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003e3.1 Data Collection\u003c/h2\u003e\n \u003cp\u003eConstructing a dataset is a necessary part of deep learning. Because there is no publicly available infrared thermal imaging dataset for circuit boards, this paper uses a homemade dataset for fault recognition experiments. Under the condition of ensuring that the ambient temperature is basically unchanged, this experiment uses InfiRay Xtherm II T2L infrared thermal camera to photograph the circuit boards respectively, in order to ensure the validity of the experimentally acquired data, in the same working environment, the same excitation signal is applied to all the boards, and the infrared thermal imaging of the full-load state is achieved after a period of time of power on, including six states: five fault states are set up by human beings and 1 normal state each state are collected 100 saturated state of infrared thermal imaging, a total of 600 collected. The acquisition system consists of an infrared thermal imaging camera, a microcontroller timing control device, a relay, a computer, and a computer. Infrared thermal imaging data acquisition is shown in Fig. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003e3.2 Data Enhancement\u003c/h2\u003e\n \u003cp\u003eData augmentation is a technique to expand the training dataset by generating samples with the same labels from the same source data for the purpose of improving the robustness of the deep learning model\u003csup\u003e[\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e.The data augmentation methods used include random horizontal flipping, random vertical flipping, and adding Gaussian noise. The overall number of data samples is brought to 2681 and the dataset is divided as shown in Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. The training data during model training is divided into 70% training set and 30% validation set.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n \u003ch2\u003e3.3 Image Enhancement\u003c/h2\u003e\n \u003cp\u003eAcquired infrared images have the disadvantages of low imaging resolution, poor uniformity and stereoscopic sense, low signal-to-noise ratio, low contrast, easy to carry noise, blurred and missing edge details. Image enhancement is a process of enhancing a low definition infrared thermography image into a high definition infrared thermography image\u003csup\u003e[\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/sup\u003e. Through the acquisition of the infrared thermal imaging circuit board after the feature enhancement to highlight the characteristics of the components, the pixel points of infrared thermal imaging (x, y) horizontally and numerically in the direction of the operation, in order to use its effective image for feature extraction, choose the first-order infrared thermal imaging circuit boards, and finally obtain the gradient of the information on the gradient. For infrared thermal imaging circuit board first order derivatives are processed as shown in Eqs. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e$${G}_{X}\\left(x,y\\right)=\\frac{\\left(f\\left(x,y+1\\right)-f\\left(x,y\\right)\\right)+\\left(f\\left(x+1,y+1\\right)-f\\left(x+1,y\\right)\\right)}{2}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e$${G}_{y}(x,y)=\\frac{\\left(f\\left(x,y\\right)-f\\left(x+1,y\\right)\\right)+(f\\left(x,y+1\\right)-f\\left(x+1,y+1\\right))}{2}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{G}}_{\\text{x}}\\)\u003c/span\u003e\u003c/span\u003e(x,y) and\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\text{G}}_{\\text{y}}\\)\u003c/span\u003e\u003c/span\u003e(x,y) are used to denote the first order derivatives in the x and y directions. The magnitude G(x,y) and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\theta }(\\text{x},\\text{y})\\)\u003c/span\u003e\u003c/span\u003edirection results of the edge gradient are shown in Eq. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e and Eq. \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e$$G(x,y)=\\sqrt{{{G}_{X}}^{2}+{{G}_{y}}^{2}}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e$$\\theta (x,y)=\\text{a}\\text{r}\\text{c}\\text{t}\\text{a}\\text{n}\\left(\\frac{{G}_{X}}{{G}_{y}}\\right)$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003eThe image brightness and contrast enhancement after image enhancement is obvious, and the details of the image obtained through image enhancement are clearer. The results show that the algorithm in this section can effectively enhance the pixel density, improve the detail information and enhance the visual effect. Finally, in order to improve the training speed of the network, the pixel values of infrared thermal imaging are normalized. The image preprocessing provides high quality images for later image fault diagnosis. The original image is shown in Fig. 4,the image of the IR board after image enhancement is shown in 5.\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"4Experiment and Analysis","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Experimental environment and parameter settings\u003c/h2\u003e \u003cp\u003eThe experimental environment is based on Python3.8 and PyTorch1.10 deep learning framework, the experimental model is trained on NVIDIA GeForce GTX 3060 GPU, the optimizer is set to Adam, the loss function is set to cross-entropy loss, the Batchsize is set to 16, the number of iterations is 70, and the learning rate is set to 0.001. The formula for cross-entropy loss function is shown in Eq.\u0026nbsp;\u003cspan refid=\"Equ5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. loss function is calculated as shown in Eq.\u0026nbsp;\u003cspan refid=\"Equ5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003cdiv id=\"Equ5\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e\n$$L=\\frac{1}{N}\\sum _{i=1}^{N}-\\left[{y}_{i}\\text{log}{(p}_{i}\\right)+(1-{y}_{i})\\text{log}{(p}_{i}\\left)\\right]$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWith Pi representing the size of the probability that the model predicts class i, N represents the total number of samples, and yi denotes the categorical label of sample i.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Proposed diagnosis method ablation experiment\u003c/h2\u003e \u003cp\u003eIn order to verify the effectiveness of the fault diagnosis method in this paper, and at the same time to explain scientifically the role of the improved structure in the model performance improvement, this chapter takes ResNet50 as the base network to be improved and conducts structural ablation experiments on the improved model, and then tests it on a test set to obtain the various evaluation indexes. Image enhancement can effectively improve the model performance, in order to explore the effect of the above approach on the performance of the improved ResNet50 network model, set the same experimental conditions, the improved ResNet50 model with image enhancement achieves a detection time of 246ms per image, an accuracy of 98.92% on the validation set, and an accuracy of 98.66% on the test set, which is 1.66% higher than that of the one without image enhancement, respectively. Fault diagnosis accuracy is 1.27% and 1.51% higher,thus proving that image enhancement has a significant effect on the model performance. The results of the ablation experiment are shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. From the table, it can be seen that all parts of the improvement have some effect on the results of the base experiment. By adding SE-Net to the base network and MHSA for analysis and comparison, it is finally concluded that the fault diagnosis accuracy of LGC-Net network in the test set is 98.92%, which is higher than that of the base ResNet50 model by 4.86%, which proves that the improvement of the circuit board fault diagnosis method of ResNet50 infrared thermal imaging is practicable.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eResults of network ablation experiments\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003emould\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePrecsion\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eF1-score/%\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e94.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e94.24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e94.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e94.12\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSE-ResNet50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e95.07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e95.67\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e95.07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e95.11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMHSA-ResNet50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e97.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e97.16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e97.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e97.02\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLGC-Net\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e98.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e98.97\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e98.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e98.92\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Comparison Experiments of the Proposed Diagnostic Methods\u003c/h2\u003e \u003cp\u003eIn order to verify the superiority of the proposed fault diagnosis method in dealing with the classification task of infrared circuit board image dataset, under the same experimental conditions, three traditional classical network models, VGG16, GoogLeNet, and ResNet18, are selected for performance comparison.The model parameters are set up with the number of training rounds as 70, the initial value of the learning rate is set as 0.001, and the cosine annealing decay strategy is utilized during the training process For updating, it can be seen from Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e that the verification accuracy of the proposed improved ResNet50 network is significantly improved over the traditional network model, and compared to the other networks, the improved ResNet50 network has less number of parameters, and its infrared thermography diagnosis time is 246ms, which is much lower than the other models. Therefore, the improved ResNet50 network proposed in this paper is able to identify infrared thermal imaging circuit board faults more accurately and has a strong generalization ability, which provides excellent performance compared to other methods.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of the performance of different models\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003emethodologies\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003cp\u003e/%\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTime(ms)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e92.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e96.89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e92.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e856\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVGG16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e97.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e97.16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e97.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e545\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGoogLeNet\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e94.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e94.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e94.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e703\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLGC-Net\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e98.97\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e98.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e98.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e246\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Model training evaluation\u003c/h2\u003e \u003cp\u003eDuring the training process the validation set curve has a larger oscillation in the pre-experiment, by fine-tuning the hyper-parameters, so that the model maintains the optimal state, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e6\u003c/span\u003e Loss curve diagram, the horizontal coordinate is the number of rounds of model training, and the vertical coordinate is the Loss value of the corresponding rounds, and the model gradually reaches the optimal performance in the late stage of training. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e7\u003c/span\u003e, the accuracy curve shows that the accuracy fluctuates greatly in the first 40 epochs of the validation set during the model training process, and the accuracy tends to stabilize in the last 30 epochs and is very close to the accuracy curve of the training set, which reflects the validity of the method of this chapter in infrared thermal imaging-oriented circuit board fault diagnosis, and the performance of the fault diagnostic model is more stable, with Robustness.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"5 Conclusion","content":"\u003cp\u003eIn this paper, by improving the ResNet50 network and embedding the SE attention module and the MHSA attention module into it, we design the infrared image fault diagnosis recognition network LGC-Net, which consists of a feature extraction part and a classification part to complete the tasks of image feature extraction and infrared image fault diagnosis and recognition, respectively, and introduces the maximal pooling to introduce Layer3. By introducing maximum pooling, the Layer3 bottleneck block is quickly connected to reduce the loss of information in the process of downsampling, enhance the transform invariance of the input features, and effectively improve the fault diagnosis accuracy of infrared thermal images. Under the same experimental conditions, by comparing ResNet18/50, VGG16 and GoogLeNet, the improved network has the best diagnostic accuracy, and the average diagnostic time is only 246ms, which is much lower than other models. However, infrared thermography, as one of the many diagnostic methods, can not completely diagnose all circuit board faults, and in the future, we can explore the use of a variety of data forms for comprehensive diagnosis of circuit board faults, the experiments in this paper used a self-built dataset, and the amount of data collected is relatively small, and future work needs to obtain more fault types of data to improve the quality of infrared thermography.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003ea,Significant revisions to improve articles and guide the completion of the overall workb,Perform dataset collection, network construction, and experimental collation processc,Guided the overall flow of the experiment smoothly, with important modifications for network improvement\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eSida. Research on circuit board fault detection method based on infrared thermal imaging technology [D]. Taiyuan University of Science and Technology, 2016.\u003c/li\u003e\n \u003cli\u003eCui Haohao,Zhang Zhipeng,Liu Yanfeng et al. Fault detection method for circuit boards based on infrared images [J]. Information Technology and Informatization, 2023, (09): 139-142.\u003c/li\u003e\n \u003cli\u003eLi Changwen, Cheng Zeyin, Zhang Xiaogang et al. Fault diagnosis of coal mining machine rocker gear based on deep residual network [J]. Industrial and Mining Automation, 2021, 47 (03): 71-78. DOI:10.13272/j.issn.1671-251x.2020110043\u003c/li\u003e\n \u003cli\u003eZhao Yan. Deep learning based infrared image recognition and thermal fault diagnosis of electrical equipment [D]. North China Electric Power University, 2022. doi:10.27139/d.cnki.ghbdu.2022.000089\u003c/li\u003e\n \u003cli\u003eHe K,Zhang X,Ren S, et al. Deep Residual Learning for Image Recognition.[J]. CoRR,2015, abs /1512.03385.\u003c/li\u003e\n \u003cli\u003eSRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021: 16519-16529.\u003c/li\u003e\n \u003cli\u003eSUNKARA R，LUO T.No more strided convolutions or pooling：a new CNN building block for low- resolution images and small objects[J].arXiv：2208.03641，2022.\u003c/li\u003e\n \u003cli\u003eR. Zhong,B. Jiang,N. Li et al. Face expression recognition method embedded with attention mechanism residual network [J]. Computer Engineering and Applications, 2023, 59 (11): 88-97.\u003c/li\u003e\n \u003cli\u003eHeavy rain Xuan, Lu Tianliang, Du Yanhui et al. Deep forgery video detection method based on i_ResNet34 model and data enhancement [J]. Computer Science, 2021, 48 (07): 77-85.\u003c/li\u003e\n \u003cli\u003eJIA Shuaikang, SUN Hairong, SU Zifan. Study on hot spot state classification of infrared images by improved residual network[J]. Industrial Control Computer,2021,34(2):79-82.\u003c/li\u003e\n \u003cli\u003eZHANG Qian,YANG Ying,LIU Gang,et al. Fusion of data enhancement and improved ResNet34 for mastitis detection in thermal infrared images of dairy cows[J]. Spectroscopy and Spectral Analysis, 2023, 43(1): 9.\u003c/li\u003e\n \u003cli\u003eJeon M, Yoo S, Kim S W. A contactless PCBA defect detection method: Convolutional neural networkswith thermographic images[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2022, 12(3): 489-501.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Table","content":"\u003cp\u003eTable 1 is available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Infrared thermal imaging, ResNet50, residual structure, attention mechanism","lastPublishedDoi":"10.21203/rs.3.rs-3932781/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3932781/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn practical industrial applications, infrared thermography fault diagnosis has received widespread attention. In order to detect the internal component faults of circuit boards, this paper proposes an novel circuit board fault diagnosis network using infrared thermal image.Firstly, a network LGC-Net network is proposed to improve ResNet50, and a two-layer attention module and a channel feature extraction module are introduced into the ResNet50 network, which strengthens the network's ability to extract global features and improves the model's extraction of features such as multi-scale key detail features of the input feature maps and the target contour, etc. Optimization of the residual structure and model pre-training plus fine-tuning are used for the Diagnostic model construction, so that the accuracy of fault diagnosis has been greatly improved. Finally, the acquired board infrared thermography dataset is preprocessed and experimented with a diagnostic fault network Experiments show that the LGC-Net network proposed in this paper has a fault diagnosis accuracy of 98.92%, which is 1.9% higher than the optimal accuracy of the classical network fault diagnosis, while the diagnosis time of a single infrared thermography is 246ms, which is 299ms lower than the shortest time consumed by other models for diagnosis.The experimental results show that the method proposed in this paper is able to efficiently identify infrared thermography faults of circuit board components, to improve the accuracy of circuit board fault diagnosis.\u003c/p\u003e","manuscriptTitle":"An novel circuit board fault diagnosis network using infrared thermal image","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-08 19:16:55","doi":"10.21203/rs.3.rs-3932781/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c62583c4-e6e9-4017-9e8f-6aacafa9050c","owner":[],"postedDate":"February 8th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-02-20T11:29:38+00:00","versionOfRecord":[],"versionCreatedAt":"2024-02-08 19:16:55","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3932781","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3932781","identity":"rs-3932781","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}