Utilizing Multimodal Data for Diagnosis of Kawasaki Disease: An AI Approach

Utilizing Multimodal Data for Diagnosis of Kawasaki Disease: An AI Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Utilizing Multimodal Data for Diagnosis of Kawasaki Disease: An AI Approach Zhixin Li, Gang Luo, Zhixian Ji, Wang Sibao, Silin Pan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4323083/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Objective We propose a new multimodal artificial intelligence model that facilitate the differentiation of Kawasaki disease through the integration of clinical symptom photographs and laboratory examination indices. Methods This study is a retrospective investigation based on laboratory examination data, palm images, and conjunctival image databases of 620 children (comprising those with both healthy physical examinations and Kawasaki disease) who visited our hospital between January 2022 and January 2024. The multimodal model was trained and evaluated using this database. GradCAM was incorporated to analyze the attention mechanisms of the multimodal model. A human-machine double-blind controlled trial was designed to evaluate the diagnostic accuracy of the obtained multimodal model and senior clinical physicians with advanced qualifications on external dataset. Results The performance evaluation of the multimodal model on the validation set yielded an area under the curve of 0.97 and an accuracy of 0.96.The GradCAM analysis reveals that the model's attention is concentrated on areas such as palm swelling and peeling, as well as conjunctivitis, which aligns with clinical reasoning.The human-machine double-blind trial validated that the multimodal model and senior pediatric physicians with advanced qualifications achieved comparable accuracy rates in identifying cases within an independent external cohort. Conclusion The multimodal model we developed can assist junior doctors in diagnosing Kawasaki disease, providing a new approach for the auxiliary diagnosis of Kawasaki disease in medically underserved areas. Health sciences/Rheumatology/Rheumatic diseases/Paediatric rheumatic diseases Physical sciences/Mathematics and computing/Computer science Artificial intelligence Kawasaki Disease Multimodal Model Attention Analysis Computer-aided diagnosis Figures Figure 1 Figure 2 Figure 3 Introduction Kawasaki disease (KD) is an acute self-limiting vasculitis of unknown etiology, which can lead to permanent coronary artery structural damage ( 1 , 2 ). It is the most common cause of acquired heart disease in developed countries ( 3 ). Coronary artery structural abnormalities occur early in the disease course, with over 80% of such abnormalities manifesting within the first 10 days of illness. Approximately 25% of untreated KD patients develop coronary artery lesions, significantly impacting the quality of life of affected children. Early detection and timely treatment can reduce the occurrence rate of coronary artery aneurysms in KD complications from 25% to around 4% ( 4 ). Therefore, timely detection and early diagnosis and treatment are crucial for reducing coronary artery lesions (CALs). Currently, the diagnosis of Kawasaki disease is typically made by combining clinical symptoms (such as fever, conjunctivitis, peripheral limb swelling, and desquamation) with laboratory test results and echocardiography ( 5 ). For patients with typical symptoms, experienced physicians can make a diagnosis within three days of the onset of fever. However, in medically underserved areas, the lack of experienced physicians and specialized pediatric cardiac ultrasound teams leads to high rates of misdiagnosis and underdiagnosis, resulting in delayed treatment and subsequent coronary artery abnormalities. Addressing how to assist in diagnosing Kawasaki disease in medically underserved areas has become an urgent clinical issue. In recent years, multimodal deep learning models have garnered increasing attention in the field of artificial intelligence ( 6 , 7 ). Different forms of existence or information sources can be referred to as modalities, and data composed of two or more modalities are termed multimodal data. Due to the generation of large volumes of diverse types of data in clinical practice, multimodal deep learning models have been widely applied and have seen vigorous development in the medical field ( 8 – 11 ). In the field of assisting the diagnosis of Kawasaki disease, existing research has mainly focused on developing single-modal models using either laboratory examination indices or clinical symptom images alone for identifying and aiding in the diagnosis of Kawasaki disease patients ( 12 – 17 ). However, these models exhibit poor generalization, as relying solely on one clinical data type cannot fully diagnose Kawasaki disease; comprehensive assessments involving multiple types of data are necessary to make informed judgments. In order to assist in the diagnosis of Kawasaki disease in clinical scenarios with uneven medical resources, we constructed an artificial intelligence-based multimodal model in this study. This model utilized both laboratory examination data and palm and conjunctival image data for model construction, training, and evaluation. Subsequently, we conducted interpretability analysis and designed a human-machine double-blind controlled trial, which yielded promising auxiliary diagnostic performance. This effectively aids clinicians in medically underserved areas, increasing the detection rate of Kawasaki disease and reducing the occurrence of coronary artery damage, thereby safeguarding the health of children. Materials and methods Study design This study is a retrospective investigation, with data collection from our hospital's database spanning from January 2022 to January 2024. The initial purpose of establishing the database was for medical education, which later transitioned into a machine learning database. The data collection is divided into two parts:1) Kawasaki Disease Group: Clinical symptom images and laboratory examination indices of Kawasaki disease patients were collected during outpatient visits or hospitalization periods when they were initially suspected of having Kawasaki disease. After confirming the diagnosis of Kawasaki disease, these data were included in the Kawasaki disease group.2) Healthy Control Group: Clinical symptom images and laboratory examination indices of healthy children were collected during routine health check-ups at pediatric health clinics. After confirming their health status, these data were included in the healthy control group. According to the diagnostic guidelines for Kawasaki disease, clinical symptoms are the primary diagnostic criteria. In this study, the palms and conjunctiva were chosen as research subjects due to their high stability and specificity in assisting Kawasaki disease diagnosis, thereby aiding physicians in diagnosing the disease. Another reason is the limited number of images of tongues and typical rash symptoms of Kawasaki disease patients stored in our hospital's database, which cannot adequately support subsequent research. Therefore, only palm and conjunctiva images were selected as research subjects. According to the Kawasaki disease diagnosis and treatment guidelines, laboratory examination indicators are the second diagnostic criteria following clinical symptoms in the diagnostic process of Kawasaki disease. Typical laboratory examination data, such as CRP and ESR, have significant auxiliary diagnostic value. Moreover, multiple studies have suggested that various laboratory examination indicators can assist in Kawasaki disease diagnosis. Therefore, this study comprehensively included all laboratory examination results obtained during the diagnosis and differential diagnosis process of Kawasaki disease in our hospital. The cohort consisted of 620 children (310 cases of Kawasaki disease children and 310 cases of healthy children for physical examination), with each child's data comprising 2 clinical symptom images (1 conjunctival image and 1 palm image) and 26 laboratory assay indicators. The dataset included a total of 1240 images (620 clinical symptom images from Kawasaki disease children and 620 clinical images from healthy children) and 16120 laboratory assay indicators (8060 from Kawasaki disease children and 8060 from healthy children). Inclusion criteria for Kawasaki disease group data: 1) Confirmed diagnosis of Kawasaki disease, 2) Kawasaki disease as the primary diagnosis, 3) Complete data on laboratory examination indicators, 4) Symptom onset within a narrow time frame of medical consultation (less than 5 days). Exclusion criteria: 1) Diagnoses such as trauma, congenital heart disease that may affect image quality, 2) Poor quality or blurry images, 3) Diagnoses such as pneumonia, infection that may affect laboratory assay results, 4) Missing laboratory examination indicators. Inclusion criteria for healthy children group data: 1) Absence of significant abnormalities in pediatric health examination, 2) Normal growth and development, 3) Complete data on laboratory examination indicators. Exclusion criteria: 1) History of genetic metabolic diseases or conditions affecting facial appearance, 2) History of hand injuries or conditions affecting image quality, 3) Missing laboratory examination indicators. Group characteristic matching involves matching the gender and age of the Kawasaki disease children group with those in an existing healthy children database. Furthermore, this study collected an additional 50 children from a peer hospital to serve as an external validation group. This group was utilized to validate the stability of the multimodal model and conduct a double-blind controlled trial involving human-machine interactions. The Ethics Review Committee of Qingdao Women's and Children's Hospital approved this study, confirming that all methods adhered to relevant guidelines and laws. Prior to preliminary data collection, guardians of children with Kawasaki disease and healthy children signed informed consent for data use. All data are rigorously protected, and palm images containing palm prints and palm veins ( 18 , 19 ), as well as conjunctival data ( 20 , 21 ) containing iris images, are non-deidentifiable personal privacy and must not be disclosed. Figure 1 depicts the flowchart of this study. Study patients, examination, and image acquisition Enhancing conjunctival and palm image data through augmentation can improve the generalization of multimodal models. This study primarily involves augmentation techniques, including basic image transformations, color and brightness adjustments, and the addition of blur and noise. Basic image transformations include horizontal flipping, vertical flipping, random rotation, and scaling operations. Color and brightness adjustments simulate variations in image appearance under different lighting conditions by randomly altering color channel values and adjusting brightness and contrast, thereby enabling the model to adapt to various real-world scenarios. Blur and noise adjustments utilize Gaussian blur to simulate focusing issues during image capture, while random noise simulates noise from image sensors. All images are downscaled to 512×512 JPG images through downsampling conversion. Blood laboratory tests encompass numerous parameters. To mitigate interference from irrelevant variables, statistical screening was conducted, resulting in the inclusion of 26 indicators for subsequent model training. These selected indicators comprise hematological analyses: neutrophil count, platelet count, lymphocyte count, neutrophil percentage, neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, white blood cell count, and hemoglobin. Additionally, biochemical tests include lactate dehydrogenase, aspartate aminotransferase, total bilirubin, alanine aminotransferase, globulin, albumin, glutamate dehydrogenase, potassium ion, sodium ion, and C-reactive protein. The dataset was randomly sampled and divided into training set (N = 496) and validation set (N = 124) in an 8:2 ratio. Model development The essence of the multimodal model in this study lies in the late fusion of residual neural networks and one-dimensional convolutional neural networks, while simultaneously addressing the classification of images and laboratory indicators. Model architecture The multimodal model primarily consists of three components: a ResNet image processing module with transfer learning capabilities, a one-dimensional CNN processing module for laboratory assay indicators, and a late fusion fully connected layer. The ResNet module addresses the optimization challenges in training deep neural networks by introducing the concept of residual blocks. The key aspect of this model is the late fusion of features from images and one-dimensional data at the fully connected layer, which occurs during the decision-making phase of the model. Features from each modality are initially processed independently in their respective neural networks and then merged at the model's fully connected layer. This strategy's advantage lies in maintaining the independence of each modality and potentially reducing the risk of overfitting. Additionally, to minimize the training time cost of the model, we adopted the concept of transfer learning. We chose a ResNet model pre-trained on the ImageNet dataset ( 22 ) as the baseline model for initial training and performed training of the multimodal model by locally unfreezing. Training was conducted using the Adam optimizer and grid parameters. Given that this model encompasses a 50-layer neural network and simultaneously processes both image and laboratory indicator data, it is named the ResNet50-clinical model. To assess the performance of the multimodal model, five single-modal residual convolutional networks (ResNet18, ResNet34, ResNet50, ResNet101, ResNet152) and separate traditional machine learning models (Support Vector Machines (SVM), Random Forest, Decision Tree, XGBoost, and LightGBM) were designed for model training and evaluation. Model optimization The hyperparameters were set to a Batch size of 32 and a learning rate of 0.000001. The random seed was set to 1024, and the parameters yielding the minimum loss function value on the validation dataset within 100 epochs were identified as the optimal model for performance. The model training, building, and validation were performed using PyTorch (2.2.0) ( 23 ) on a computer equipped with an AMD EPYC 7532 processor (32 cores 64 threads @2.4-3.3GHz) and 4 x RTX 4090 cards (24GB GDDR6X VRAM, 16384 CUDA cores). Model validation This study incorporates fusion of gradient-weighted class activation mapping (GradCAM) ( 24 – 26 ) into the multimodal model for attention analysis, enhancing the interpretability of the model and the confidence of physicians. Global average pooling is applied to the last convolutional layer of the ResNet module to generate classification activation maps. The training weights for each output of the global average pooling layer indicate the importance of each feature map from the last convolutional layer. These weights are then applied to the corresponding feature maps to generate significance maps, which are superimposed on palm and conjunctival images to achieve visualization of category differentiation in prioritizing regions of the multimodal model. To assess the generalization performance of the model, we conducted a human-machine double-blind controlled experiment, utilizing an external validation group. Following a thorough diagnostic process conducted by a professional diagnostic and treatment team, the external validation group's images and laboratory data were provided to the model and senior clinical physicians with advanced professional titles separately for evaluation (double-blind trial). Children in the external validation group were diagnosed with Kawasaki disease after undergoing a standard diagnostic process, and then matched with the same age and gender from the healthy children database. Subsequently, 50 Kawasaki disease children and 50 matched healthy children from the external validation group formed a new external validation group. Palm and conjunctival images, along with corresponding laboratory examination data, were independently evaluated by the multimodal model and three pediatricians with advanced professional titles (unknown diagnosis results), who did not participate in the diagnostic and treatment process. They could only access the laboratory examination indicators and palm-conjunctival symptom images of the children and made judgments based on these limited data to ensure the effectiveness of the human-machine controlled trial. Although their inability to access other auxiliary diagnostic data may lead to a noticeable decrease in the accuracy of their diagnoses compared to their routine practices, this does not affect the validation of the human-machine double-blind trial, demonstrating the reliability of the multimodal model. Statistical analysis In terms of performance metrics, the proposed model was compared with existing methods, including accuracy, sensitivity, specificity, area under the curve (AUC), and F1 score. Results Population demographic data This study involved 620 children, among which 310 underwent healthy physical examinations, and 310 were diagnosed with Kawasaki disease, with a male-to-female ratio of 1.5:1, aged between 1 and 8 years, and with an average age of 3.7 years. A total of 16,740 laboratory examination data and 1,240 images from these 620 children were analyzed. The distribution of demographic data is presented in Table 1 . The laboratory indicators with strong differences obtained by statistical analysis and included in subsequent model training are shown in Table 2 . Table 1 Population demographics Clinical characteristic Normal(N = 310) Kawasaki disease (N = 310) Total(N = 620) Sex Male 186 205 391 Female 124 105 229 Age 1 31 47 78 2 62 78 140 3 93 109 202 4 78 47 125 5 25 9 34 6 12 6 18 7 6 9 15 8 3 5 8 Table 2 Comparison of laboratory data between the Normal coronary arteries and KD group. Group Normal(N = 310) Kawasaki disease (N = 310) P Value ESR 13 141 4.92E-127 CRP 4.19 32.32 4.89E-85 Albumin 39.3 32.32 3.83E-67 LDL cholesterol 0.85 0.41 5.57E-27 Creatine kinase 54.43 39.27 9.4E-20 Alanine aminotransferase 6.69 22.37 4.87E-19 Glucose 3.2 2.2 4.08E-11 Sodium 113.64 108.93 1.76E-08 Prealbumin 0.34 0.05 3.21E-08 Triglycerides 0.46 0.22 0.0000069 Total cholesterol 1.28 0.64 0.0000492 Lactate dehydrogenase 239.69 224.89 0.000305809 HDL cholesterol 0.25 0.11 0.000834866 Transferrin 1 0.73 0.00084448 Calcium 1.85 1.77 0.000868096 Glutamate dehydrogenase 4.88 1.48 0.001332649 Total carbon dioxide 2.42 2.78 0.001912638 Aspartate aminotransferase 4.85 11.69 0.002187489 Complement C1q 28.1 22.65 0.003569859 Alkaline phosphatase 127.5 119.99 0.003767569 Glomerular filtration rate 3.32 2.56 0.003803922 Potassium 3.5 3.34 0.004414624 Creatine kinase isoenzyme 16.8 15.37 0.004490887 Troponin I 0.05 0.01 0.006224686 Total protein 59.81 49.66 0.008355241 Plasma osmolarity 26.54 32.37 0.008874509 Model evaluation The diagnostic accuracies of traditional machine learning models (SVM, RF, XGBoost, Decision Tree, LightGBM) were 82.88%, 80.37%, 91.45%, 91.82%, and 90.73%, respectively, significantly lower than those of the computer vision model (ResNet). The trained multimodal deep learning model (ResNet50-Clinical) exhibited the best performance in terms of accuracy, sensitivity, specificity, and F1 score, outperforming traditional machine learning models and single computer vision models significantly. Table 3 displays the diagnostic performance of each model using the validation dataset. The confusion matrix of the ResNet50-Clinical model can be found in Fig. 2 .a. A recall curve was plotted for the ResNet50-Clinical model, achieving an AUC of 0.97 (Fig. 2 .b). The ROC curves of the ResNet50-Clinical model and those of the physicians' manual judgments obtained from the human-machine double-blind experiment showed that the diagnostic performance of the ResNet50-Clinical model (AUC = 0.87) was close to that of pediatric clinical doctors with advanced titles (AUC = 0.88), suggesting that the model could effectively assist junior pediatricians, doctors with limited experience in diagnosing Kawasaki disease, and doctors in underdeveloped medical areas in forming early diagnostic opinions on Kawasaki disease (Figs. 2 .c and 2.d). ResNet50-Clinical model's GradCAM results, superimposed on palm and conjunctiva images, confirm the attention strategy of the ResNet50-Clinical model. We observed that the model's attention is primarily concentrated on the peeling area of the palm, fingertips, and sites of conjunctivitis, consistent with clinical experience, thus validating the feasibility of our model. Due to the presence of non-removable identifiable information in palm and conjunctiva images of children, which may compromise patient privacy, we conducted searches and curated palm and conjunctiva images of Kawasaki disease patients through search engines such as Google for result demonstration purposes, evaluating the model's attention as depicted in Fig. 3 . Table 3 Diagnostic performance of deep learning algorithms. The best performance is indicated in bold, while the second-best performance is indicated with an underline. Model Accuracy Sensitivity Specificity Precision F1 Score AUC SVM 0.828 0.853 0.900 0.851 0.808 0.877 RF 0.803 0.822 0.877 0.879 0.871 0.850 XGboost 0.914 0.836 0.852 0.881 0.804 0.844 Decisiontree 0.918 0.906 0.809 0.842 0.871 0.858 LightGBM 0.907 0.886 0.814 0.905 0.828 0.850 ResNet18 0.748 0.745 0.751 0.755 0.750 0.748 ResNet34 0.938 0.951 0.92 0.929 0.940 0.936 ResNet50 0.932 0.947 0.885 0.857 0.923 0.916 ResNet101 0.858 0.830 0.891 0.891 0.860 0.861 ResNet152 0.946 0.975 0.920 0.917 0.945 0.948 ResNet50-Clinical 0.967 0.967 0.936 0.941 0.964 0.972 Discussion This study has devised a multimodal model leveraging artificial intelligence to aid clinical physicians in diagnosing Kawasaki disease in medically underserved regions. To the best of our knowledge, we are the first to incorporate a multimodal approach that integrates clinical symptoms and blood laboratory test results for aiding in the diagnosis of Kawasaki disease. GradCAM visualization highlighted regions of significance in hand swelling and peeling, as well as conjunctivitis, validating the credibility of our model. While the results of the human-machine double-blind trial in the external validation group were slightly lower than those of the test set, the performance of our multimodal model is essentially equivalent to that of senior clinical practitioners. This study reveals that when confronted with the limitation of sparse medical image data, as the network depth increases, even the performance of the ResNet model approaches stability. Introducing a simpler CNN network for late fusion can effectively enhance the final model's classification performance while reducing both training time and hardware requirements. Moving forward, the integration of multiple modalities (such as laboratory test results, radiological findings, pathological images, and medical histories) on an enlarged scale may offer a cost-effective solution in the future development of medical imaging. Our multimodal model can assist clinical doctors in underdeveloped medical areas to diagnose Kawasaki disease, thereby reducing complications such as coronary artery damage caused by misdiagnosis and missed diagnosis. Limitations: Firstly, as this study is a single-center study, in order to ensure the generalizability of the model, we plan to conduct validation and testing of the model with a larger dataset from multiple centers. Secondly, in this study, under the premise of providing only palm and conjunctival images as well as laboratory examination indicators, the performance of the model is similar to that of pediatricians with advanced titles in terms of diagnostic accuracy. We plan to establish a larger-scale dataset in the next step to improve the accuracy of the multimodal model. Thirdly, for the diagnosis of Kawasaki disease, the nature of fever and accompanying symptoms included in the medical records are also crucial. Our multimodal model currently can only handle blood routine and image information. We plan to add the module of the large language model NLM in the next step to fully explore the clinical data of Kawasaki disease in medical records and further improve the performance of the model. Conclusion In this study, we have developed a multimodal model using deep learning to assist clinical doctors in diagnosing Kawasaki disease in medically underserved areas. This model has broad prospects for assisting doctors in the diagnosis and treatment of Kawasaki disease, reducing the risk of coronary artery damage, and better protecting children's health. Declarations Author Contribution LZ, LG, JZ, WS, and PS conceived and planned the model design approach.LZ, LG, JZ, and WS conducted the model training and initial evaluations.LZ, PS, and JZ designed and performed the human-machine double-blind experiments.PS, WS, LZ, LG, JZ, and others participated in building the database.LZ, LG, JZ, WS, and PS contributed to interpreting the results.LZ took the lead in writing the manuscript.All authors provided critical feedback and helped complete the research, analysis, and manuscript. Data Availability Data availabilityThe laboratory examination data and model codes can be applied for by contacting the first author ( [email protected] ) upon the reader's request.However, the palm and conjunctival image data cannot be publicly shared or displayed due to the presence of identifiable information of the patients, such as palm prints and irises, which cannot be removed. References Wang Z, Xie L, Ding G, Song S, Chen L, Li G, Xia M, Han D, Zheng Y, Liu J, et al. Single-cell RNA sequencing of peripheral blood mononuclear cells from acute Kawasaki disease patients. Nat Commun (2021) 12:5444. doi: 10.1038/s41467-021-25771-5 Denby KJ, Clark DE, Markham LW. Management of Kawasaki disease in adults. Heart Br Card Soc (2017) 103:1760–1769. doi: 10.1136/heartjnl-2017-311774 Zhang Y, Wang Y, Zhang L, Xia L, Zheng M, Zeng Z, Liu Y, Yarovinsky T, Ostriker AC, Fan X, et al. Reduced Platelet miR-223 Induction in Kawasaki Disease Leads To Severe Coronary Artery Pathology Through a miR-223/PDGFRβ Vascular Smooth Muscle Cell Axis. Circ Res (2020) 127:855–873. doi: 10.1161/CIRCRESAHA.120.316951 Singh S, Vignesh P, Burgner D. The epidemiology of Kawasaki disease: a global update. Arch Dis Child (2015) 100:1084–1088. doi: 10.1136/archdischild-2014-307536 McCrindle BW, Rowley AH, Newburger JW, Burns JC, Bolger AF, Gewitz M, Baker AL, Jackson MA, Takahashi M, Shah PB, et al. Diagnosis, Treatment, and Long-Term Management of Kawasaki Disease: A Scientific Statement for Health Professionals From the American Heart Association. Circulation (2017) 135:e927–e999. doi: 10.1161/CIR.0000000000000484 Cordier N, Delingette H, Le M, Ayache N. Extended Modality Propagation: Image Synthesis of Pathological Cases. IEEE Trans Med Imaging (2016) 35:2598–2608. doi: 10.1109/TMI.2016.2589760 Wang K, He R, Wang L, Wang W, Tan T. Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval. IEEE Trans Pattern Anal Mach Intell (2016) 38:2010–2023. doi: 10.1109/TPAMI.2015.2505311 Yawen Huang null, Ling Shao null, Frangi AF. Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning. IEEE Trans Med Imaging (2018) 37:815–827. doi: 10.1109/TMI.2017.2781192 Chen M, Carass A, Jog A, Lee J, Roy S, Prince JL. Cross contrast multi-channel image registration using image synthesis for MR brain images. Med Image Anal (2017) 36:2. doi: 10.1016/j.media.2016.10.005 Hu D, Zhang H, Wu Z, Wang F, Wang L, Smith JK, Lin W, Li G, Shen D. Disentangled-Multimodal Adversarial Autoencoder: Application to Infant Age Prediction With Incomplete Multimodal Neuroimages. IEEE Trans Med Imaging (2020) 39:4137–4149. doi: 10.1109/TMI.2020.3013825 Hao S, Jin B, Tan Z, Li Z, Ji J, Hu G, Wang Y, Deng X, Kanegaye JT, Tremoulet AH. A classification tool for differentiation of Kawasaki disease from other febrile illnesses. J Pediatr (2016) 176:114–120. doi: 10.1016/j.jpeds.2016.05.060 Sosa T, Brower L, Divanovic A. Diagnosis and management of Kawasaki disease. JAMA Pediatr (2019) 173:278–279. doi: 10.1001/jamapediatrics.2018.3307 Lam JY, Shimizu C, Tremoulet AH, Bainto E, Roberts SC, Sivilay N, Gardiner MA, Kanegaye JT, Hogan AH, Salazar JC. A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study. Lancet Digit Health (2022) 4:e717–e726. doi: 10.1016/S2589-7500(22)00149-2 Li C, Liu Y-C, Zhang D-R, Han Y-X, Chen B-J, Long Y, Wu C. A machine learning model for distinguishing Kawasaki disease from sepsis. Sci Rep (2023) 13:12553. doi: 10.1038/s41598-023-39745-8 Tsai C-M, Lin C-HR, Kuo H-C, Cheng F-J, Yu H-R, Hung T-C, Hung C-S, Huang C-M, Chu Y-C, Huang Y-H. Use of Machine Learning to Differentiate Children With Kawasaki Disease From Other Febrile Children in a Pediatric Emergency Department. JAMA Netw Open (2023) 6:e237489–e237489. doi: 10.1001/jamanetworkopen.2023.7489 Zhou Y, Kumar A. Human Identification Using Palm-Vein Images. IEEE Trans Inf Forensics Secur (2011) 6:1259–1274. doi: 10.1109/TIFS.2011.2158423 Xu E, Nemati S, Tremoulet AH. A deep convolutional neural network for Kawasaki disease diagnosis. Sci Rep (2022) 12:11438. doi: 10.1038/s41598-022-15495-x Bagade SB, Patil KD, Hatware KV, Pingale PL, Mhatre SVC. Palm Vein Technology: A Biometric Intelligence System for patients Authentication and Safety. Res J Pharm Technol (2023) 16:5554–5561. doi: 10.52711/0974-360X.2023.00898 Siew Chin C, Beng Jin AT, Chek Ling DN. High security Iris verification system based on random secret integration. Comput Vis Image Underst (2006) 102:169–177. doi: 10.1016/j.cviu.2006.01.002 Nedjah N, Wyant RS, Mourelle LM, Gupta BB. Efficient yet robust biometric iris matching on smart cards for data high security and privacy. Future Gener Comput Syst (2017) 76:18–32. doi: 10.1016/j.future.2017.05.008 He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV, USA: IEEE (2016). p. 770–778 doi: 10.1109/CVPR.2016.90 Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition . (2009). p. 248–255 doi: 10.1109/CVPR.2009.5206848 Imambi S, Prakash KB, Kanagachidambaresan GR. “PyTorch.,” In: Prakash KB, Kanagachidambaresan GR, editors. Programming with TensorFlow: Solution for Edge Computing Applications . EAI/Springer Innovations in Communication and Computing. Cham: Springer International Publishing (2021). p. 87–104 doi: 10.1007/978-3-030-57077-4_10 Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. Int J Comput Vis (2020) 128:336–359. doi: 10.1007/s11263-019-01228-7 Borg M, Jabangwe R, Aberg S, Ekblom A, Hedlund L, Lidfeldt A. Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for Vision AI? 2021 Ieee International Conference on Software Testing, Verification and Validation Workshops (icstw 2021) . Los Alamitos: Ieee Computer Soc (2021). p. 175–181 doi: 10.1109/ICSTW52544.2021.00039 Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM plus plus: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks. 2018 Ieee Winter Conference on Applications of Computer Vision (wacv 2018) . New York: Ieee (2018). p. 839–847 doi: 10.1109/WACV.2018.00097 Additional Declarations No competing interests reported. Supplementary Files floatimage1.png Graphical Abstract Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4323083","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":298509245,"identity":"1431e675-a3de-456e-8780-6e9219de8811","order_by":0,"name":"Zhixin Li","email":"","orcid":"","institution":"Qingdao University","correspondingAuthor":false,"prefix":"","firstName":"Zhixin","middleName":"","lastName":"Li","suffix":""},{"id":298509246,"identity":"69b008e4-4979-4704-891f-bb6ab488024c","order_by":1,"name":"Gang Luo","email":"","orcid":"","institution":"Qingdao University","correspondingAuthor":false,"prefix":"","firstName":"Gang","middleName":"","lastName":"Luo","suffix":""},{"id":298509248,"identity":"812dd72f-a644-4768-ac75-9072f5ce6ab6","order_by":2,"name":"Zhixian Ji","email":"","orcid":"","institution":"Qingdao Women and Children's Hospital","correspondingAuthor":false,"prefix":"","firstName":"Zhixian","middleName":"","lastName":"Ji","suffix":""},{"id":298509250,"identity":"a96f70b3-e4ea-4ad0-8376-5e9284e44601","order_by":3,"name":"Wang Sibao","email":"","orcid":"","institution":"Qingdao University","correspondingAuthor":false,"prefix":"","firstName":"Wang","middleName":"","lastName":"Sibao","suffix":""},{"id":298509253,"identity":"1331e527-86d9-43ce-ac02-6437af9d4d4f","order_by":4,"name":"Silin Pan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4klEQVRIiWNgGAWjYLCChAIJBjb2/o8PPhjY2BGpxUCCgY/ngLHhjIK0ZCKtMWBgkJNIMJPm+XCIsYGQYnP2s4c/PDCwyGOTSEiTtjE4wMzAfvjoBnxaLHvy0iSADitm43lw2DrH4A4fA09a2g28TjqQYwbyS2Ibe2Lj7RyDZ8wMEjxm+LWcf2P8AayFIZlB2sLgMGMDQS03cgwkwFo40pikGYjRYjnjjRlEC88ZZsMeg7RkNkJ+MefPMf74o6IucX57D+ODH39s7PjZDx/D7zAMETZ8yrFrGQWjYBSMglGADgAMYUbFP8W+4QAAAABJRU5ErkJggg==","orcid":"","institution":"Qingdao Women and Children's Hospital","correspondingAuthor":true,"prefix":"","firstName":"Silin","middleName":"","lastName":"Pan","suffix":""}],"badges":[],"createdAt":"2024-04-25 09:27:52","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4323083/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4323083/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":56035310,"identity":"a1ed2acc-242e-4a9c-99d1-f3f12a559110","added_by":"auto","created_at":"2024-05-07 18:37:28","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":171099,"visible":true,"origin":"","legend":"\u003cp\u003eStudy design flowchart.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4323083/v1/cd59b0e2dab090dafc88f0c4.png"},{"id":56035311,"identity":"36857b19-e508-477a-a245-c64954f50bdc","added_by":"auto","created_at":"2024-05-07 18:37:28","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":168249,"visible":true,"origin":"","legend":"\u003cp\u003eThe performance evaluation of the ResNet50-Clinical model. The evaluation includes: a) Confusion matrix of the ResNet50-Clinical on the validation dataset, b) Receiver operating characteristic curves of the ResNet50-Clinical model on the validation dataset, c) Receiver operating characteristic curves of the ResNet50-Clinical model on the external independent dataset, and d) Receiver operating characteristic curves of the clinical doctors on the external independent dataset.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-4323083/v1/d75588f7a3ec210eb65085e6.png"},{"id":56035312,"identity":"46522d20-f267-4ae1-a285-2f31ff9baa21","added_by":"auto","created_at":"2024-05-07 18:37:28","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":176684,"visible":true,"origin":"","legend":"\u003cp\u003eResNet50-Clinical Grad-CAM image. A. The palm of KD children with the overlaid GradCAM image. B. The conjunctiva of KD children with the overlaid GradCAM image.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4323083/v1/c494da1f2279980ec1d8d63d.png"},{"id":67722038,"identity":"baded458-b577-46dc-951a-790c968e5d38","added_by":"auto","created_at":"2024-10-29 05:25:52","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1034095,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4323083/v1/ca3b8523-1acf-4e2d-9cb3-8f63a4e119eb.pdf"},{"id":56035308,"identity":"7ef9e9f9-b9e8-455b-ad12-d1f1504f0eb7","added_by":"auto","created_at":"2024-05-07 18:37:28","extension":"png","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":231469,"visible":true,"origin":"","legend":"\u003cp\u003eGraphical Abstract\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4323083/v1/0a4c53723be9ab2a0a8491ac.png"}],"financialInterests":"No competing interests reported.","formattedTitle":"Utilizing Multimodal Data for Diagnosis of Kawasaki Disease: An AI Approach","fulltext":[{"header":"Introduction","content":"\u003cp\u003eKawasaki disease (KD) is an acute self-limiting vasculitis of unknown etiology, which can lead to permanent coronary artery structural damage (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). It is the most common cause of acquired heart disease in developed countries (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). Coronary artery structural abnormalities occur early in the disease course, with over 80% of such abnormalities manifesting within the first 10 days of illness. Approximately 25% of untreated KD patients develop coronary artery lesions, significantly impacting the quality of life of affected children. Early detection and timely treatment can reduce the occurrence rate of coronary artery aneurysms in KD complications from 25% to around 4% (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). Therefore, timely detection and early diagnosis and treatment are crucial for reducing coronary artery lesions (CALs). Currently, the diagnosis of Kawasaki disease is typically made by combining clinical symptoms (such as fever, conjunctivitis, peripheral limb swelling, and desquamation) with laboratory test results and echocardiography (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). For patients with typical symptoms, experienced physicians can make a diagnosis within three days of the onset of fever. However, in medically underserved areas, the lack of experienced physicians and specialized pediatric cardiac ultrasound teams leads to high rates of misdiagnosis and underdiagnosis, resulting in delayed treatment and subsequent coronary artery abnormalities. Addressing how to assist in diagnosing Kawasaki disease in medically underserved areas has become an urgent clinical issue.\u003c/p\u003e \u003cp\u003eIn recent years, multimodal deep learning models have garnered increasing attention in the field of artificial intelligence (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). Different forms of existence or information sources can be referred to as modalities, and data composed of two or more modalities are termed multimodal data. Due to the generation of large volumes of diverse types of data in clinical practice, multimodal deep learning models have been widely applied and have seen vigorous development in the medical field (\u003cspan additionalcitationids=\"CR9 CR10\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). In the field of assisting the diagnosis of Kawasaki disease, existing research has mainly focused on developing single-modal models using either laboratory examination indices or clinical symptom images alone for identifying and aiding in the diagnosis of Kawasaki disease patients (\u003cspan additionalcitationids=\"CR13 CR14 CR15 CR16\" citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). However, these models exhibit poor generalization, as relying solely on one clinical data type cannot fully diagnose Kawasaki disease; comprehensive assessments involving multiple types of data are necessary to make informed judgments.\u003c/p\u003e \u003cp\u003eIn order to assist in the diagnosis of Kawasaki disease in clinical scenarios with uneven medical resources, we constructed an artificial intelligence-based multimodal model in this study. This model utilized both laboratory examination data and palm and conjunctival image data for model construction, training, and evaluation. Subsequently, we conducted interpretability analysis and designed a human-machine double-blind controlled trial, which yielded promising auxiliary diagnostic performance. This effectively aids clinicians in medically underserved areas, increasing the detection rate of Kawasaki disease and reducing the occurrence of coronary artery damage, thereby safeguarding the health of children.\u003c/p\u003e"},{"header":"Materials and methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy design\u003c/h2\u003e \u003cp\u003eThis study is a retrospective investigation, with data collection from our hospital's database spanning from January 2022 to January 2024. The initial purpose of establishing the database was for medical education, which later transitioned into a machine learning database.\u003c/p\u003e \u003cp\u003eThe data collection is divided into two parts:1) Kawasaki Disease Group: Clinical symptom images and laboratory examination indices of Kawasaki disease patients were collected during outpatient visits or hospitalization periods when they were initially suspected of having Kawasaki disease. After confirming the diagnosis of Kawasaki disease, these data were included in the Kawasaki disease group.2) Healthy Control Group: Clinical symptom images and laboratory examination indices of healthy children were collected during routine health check-ups at pediatric health clinics. After confirming their health status, these data were included in the healthy control group.\u003c/p\u003e \u003cp\u003e According to the diagnostic guidelines for Kawasaki disease, clinical symptoms are the primary diagnostic criteria. In this study, the palms and conjunctiva were chosen as research subjects due to their high stability and specificity in assisting Kawasaki disease diagnosis, thereby aiding physicians in diagnosing the disease. Another reason is the limited number of images of tongues and typical rash symptoms of Kawasaki disease patients stored in our hospital's database, which cannot adequately support subsequent research. Therefore, only palm and conjunctiva images were selected as research subjects. According to the Kawasaki disease diagnosis and treatment guidelines, laboratory examination indicators are the second diagnostic criteria following clinical symptoms in the diagnostic process of Kawasaki disease. Typical laboratory examination data, such as CRP and ESR, have significant auxiliary diagnostic value. Moreover, multiple studies have suggested that various laboratory examination indicators can assist in Kawasaki disease diagnosis. Therefore, this study comprehensively included all laboratory examination results obtained during the diagnosis and differential diagnosis process of Kawasaki disease in our hospital.\u003c/p\u003e \u003cp\u003eThe cohort consisted of 620 children (310 cases of Kawasaki disease children and 310 cases of healthy children for physical examination), with each child's data comprising 2 clinical symptom images (1 conjunctival image and 1 palm image) and 26 laboratory assay indicators. The dataset included a total of 1240 images (620 clinical symptom images from Kawasaki disease children and 620 clinical images from healthy children) and 16120 laboratory assay indicators (8060 from Kawasaki disease children and 8060 from healthy children).\u003c/p\u003e \u003cp\u003eInclusion criteria for Kawasaki disease group data: 1) Confirmed diagnosis of Kawasaki disease, 2) Kawasaki disease as the primary diagnosis, 3) Complete data on laboratory examination indicators, 4) Symptom onset within a narrow time frame of medical consultation (less than 5 days). Exclusion criteria: 1) Diagnoses such as trauma, congenital heart disease that may affect image quality, 2) Poor quality or blurry images, 3) Diagnoses such as pneumonia, infection that may affect laboratory assay results, 4) Missing laboratory examination indicators.\u003c/p\u003e \u003cp\u003eInclusion criteria for healthy children group data: 1) Absence of significant abnormalities in pediatric health examination, 2) Normal growth and development, 3) Complete data on laboratory examination indicators. Exclusion criteria: 1) History of genetic metabolic diseases or conditions affecting facial appearance, 2) History of hand injuries or conditions affecting image quality, 3) Missing laboratory examination indicators.\u003c/p\u003e \u003cp\u003eGroup characteristic matching involves matching the gender and age of the Kawasaki disease\u003c/p\u003e \u003cp\u003echildren group with those in an existing healthy children database.\u003c/p\u003e \u003cp\u003eFurthermore, this study collected an additional 50 children from a peer hospital to serve as an external validation group. This group was utilized to validate the stability of the multimodal model and conduct a double-blind controlled trial involving human-machine interactions.\u003c/p\u003e \u003cp\u003e The Ethics Review Committee of Qingdao Women's and Children's Hospital approved this study, confirming that all methods adhered to relevant guidelines and laws. Prior to preliminary data collection, guardians of children with Kawasaki disease and healthy children signed informed consent for data use. All data are rigorously protected, and palm images containing palm prints and palm veins (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e), as well as conjunctival data (\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e) containing iris images, are non-deidentifiable personal privacy and must not be disclosed. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e depicts the flowchart of this study.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eStudy patients, examination, and image acquisition\u003c/h2\u003e \u003cp\u003eEnhancing conjunctival and palm image data through augmentation can improve the generalization of multimodal models. This study primarily involves augmentation techniques, including basic image transformations, color and brightness adjustments, and the addition of blur and noise. Basic image transformations include horizontal flipping, vertical flipping, random rotation, and scaling operations. Color and brightness adjustments simulate variations in image appearance under different lighting conditions by randomly altering color channel values and adjusting brightness and contrast, thereby enabling the model to adapt to various real-world scenarios. Blur and noise adjustments utilize Gaussian blur to simulate focusing issues during image capture, while random noise simulates noise from image sensors. All images are downscaled to 512\u0026times;512 JPG images through downsampling conversion.\u003c/p\u003e \u003cp\u003eBlood laboratory tests encompass numerous parameters. To mitigate interference from irrelevant variables, statistical screening was conducted, resulting in the inclusion of 26 indicators for subsequent model training. These selected indicators comprise hematological analyses: neutrophil count, platelet count, lymphocyte count, neutrophil percentage, neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, white blood cell count, and hemoglobin. Additionally, biochemical tests include lactate dehydrogenase, aspartate aminotransferase, total bilirubin, alanine aminotransferase, globulin, albumin, glutamate dehydrogenase, potassium ion, sodium ion, and C-reactive protein.\u003c/p\u003e \u003cp\u003eThe dataset was randomly sampled and divided into training set (N\u0026thinsp;=\u0026thinsp;496) and validation set (N\u0026thinsp;=\u0026thinsp;124) in an 8:2 ratio.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eModel development\u003c/h2\u003e \u003cp\u003eThe essence of the multimodal model in this study lies in the late fusion of residual neural networks and one-dimensional convolutional neural networks, while simultaneously addressing the classification of images and laboratory indicators.\u003c/p\u003e \u003cp\u003eModel architecture\u003c/p\u003e \u003cp\u003eThe multimodal model primarily consists of three components: a ResNet image processing module with transfer learning capabilities, a one-dimensional CNN processing module for laboratory assay indicators, and a late fusion fully connected layer. The ResNet module addresses the optimization challenges in training deep neural networks by introducing the concept of residual blocks. The key aspect of this model is the late fusion of features from images and one-dimensional data at the fully connected layer, which occurs during the decision-making phase of the model. Features from each modality are initially processed independently in their respective neural networks and then merged at the model's fully connected layer. This strategy's advantage lies in maintaining the independence of each modality and potentially reducing the risk of overfitting.\u003c/p\u003e \u003cp\u003eAdditionally, to minimize the training time cost of the model, we adopted the concept of transfer learning. We chose a ResNet model pre-trained on the ImageNet dataset (\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e) as the baseline model for initial training and performed training of the multimodal model by locally unfreezing. Training was conducted using the Adam optimizer and grid parameters. Given that this model encompasses a 50-layer neural network and simultaneously processes both image and laboratory indicator data, it is named the ResNet50-clinical model.\u003c/p\u003e \u003cp\u003eTo assess the performance of the multimodal model, five single-modal residual convolutional networks (ResNet18, ResNet34, ResNet50, ResNet101, ResNet152) and separate traditional machine learning models (Support Vector Machines (SVM), Random Forest, Decision Tree, XGBoost, and LightGBM) were designed for model training and evaluation.\u003c/p\u003e \u003cp\u003eModel optimization\u003c/p\u003e \u003cp\u003eThe hyperparameters were set to a Batch size of 32 and a learning rate of 0.000001. The random seed was set to 1024, and the parameters yielding the minimum loss function value on the validation dataset within 100 epochs were identified as the optimal model for performance.\u003c/p\u003e \u003cp\u003eThe model training, building, and validation were performed using PyTorch (2.2.0) (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) on a computer equipped with an AMD EPYC 7532 processor (32 cores 64 threads @2.4-3.3GHz) and 4 x RTX 4090 cards (24GB GDDR6X VRAM, 16384 CUDA cores).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eModel validation\u003c/h2\u003e \u003cp\u003eThis study incorporates fusion of gradient-weighted class activation mapping (GradCAM)\u003c/p\u003e \u003cp\u003e(\u003cspan additionalcitationids=\"CR25\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e) into the multimodal model for attention analysis, enhancing the interpretability of the model and the confidence of physicians. Global average pooling is applied to the last convolutional layer of the ResNet module to generate classification activation maps. The training weights for each output of the global average pooling layer indicate the importance of each feature map from the last convolutional layer. These weights are then applied to the corresponding feature maps to generate significance maps, which are superimposed on palm and conjunctival images to achieve visualization of category differentiation in prioritizing regions of the multimodal model.\u003c/p\u003e \u003cp\u003eTo assess the generalization performance of the model, we conducted a human-machine double-blind controlled experiment, utilizing an external validation group. Following a thorough diagnostic process conducted by a professional diagnostic and treatment team, the external validation group's images and laboratory data were provided to the model and senior clinical physicians with advanced professional titles separately for evaluation (double-blind trial). Children in the external validation group were diagnosed with Kawasaki disease after undergoing a standard diagnostic process, and then matched with the same age and gender from the healthy children database. Subsequently, 50 Kawasaki disease children and 50 matched healthy children from the external validation group formed a new external validation group. Palm and conjunctival images, along with corresponding laboratory examination data, were independently evaluated by the multimodal model and three pediatricians with advanced professional titles (unknown diagnosis results), who did not participate in the diagnostic and treatment process. They could only access the laboratory examination indicators and palm-conjunctival symptom images of the children and made judgments based on these limited data to ensure the effectiveness of the human-machine controlled trial. Although their inability to access other auxiliary diagnostic data may lead to a noticeable decrease in the accuracy of their diagnoses compared to their routine practices, this does not affect the validation of the human-machine double-blind trial, demonstrating the reliability of the multimodal model.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eStatistical analysis\u003c/h2\u003e \u003cp\u003eIn terms of performance metrics, the proposed model was compared with existing methods, including accuracy, sensitivity, specificity, area under the curve (AUC), and F1 score.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003ePopulation demographic data\u003c/h2\u003e \u003cp\u003eThis study involved 620 children, among which 310 underwent healthy physical examinations, and 310 were diagnosed with Kawasaki disease, with a male-to-female ratio of 1.5:1, aged between 1 and 8 years, and with an average age of 3.7 years. A total of 16,740 laboratory examination data and 1,240 images from these 620 children were analyzed. The distribution of demographic data is presented in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The laboratory indicators with strong differences obtained by statistical analysis and included in subsequent model training are shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePopulation demographics\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eClinical characteristic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNormal(N\u0026thinsp;=\u0026thinsp;310)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eKawasaki disease (N\u0026thinsp;=\u0026thinsp;310)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eTotal(N\u0026thinsp;=\u0026thinsp;620)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSex\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e186\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e205\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e391\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e124\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e105\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e229\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAge\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e31\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e140\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e93\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e109\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e202\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e125\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e34\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e15\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of laboratory data between the Normal coronary arteries and KD group.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGroup\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNormal(N\u0026thinsp;=\u0026thinsp;310)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eKawasaki disease \u003c/p\u003e \u003cp\u003e(N\u0026thinsp;=\u0026thinsp;310)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eP Value\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eESR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e141\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.92E-127\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCRP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e32.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.89E-85\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlbumin\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e39.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e32.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.83E-67\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLDL cholesterol\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5.57E-27\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCreatine kinase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e54.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e39.27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e9.4E-20\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlanine aminotransferase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e22.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.87E-19\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGlucose\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.08E-11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSodium\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e113.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e108.93\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.76E-08\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrealbumin\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.21E-08\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTriglycerides\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.22\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.0000069\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal cholesterol\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.0000492\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLactate dehydrogenase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e239.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e224.89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.000305809\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHDL cholesterol\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.000834866\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTransferrin\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.00084448\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCalcium\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.000868096\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGlutamate dehydrogenase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.88\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.001332649\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal carbon dioxide\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.001912638\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAspartate aminotransferase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.002187489\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eComplement C1q\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e28.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e22.65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.003569859\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlkaline phosphatase\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e127.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e119.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.003767569\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGlomerular filtration rate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.003803922\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePotassium\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.004414624\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCreatine kinase isoenzyme\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e16.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e15.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.004490887\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTroponin I\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.006224686\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal protein\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e59.81\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e49.66\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.008355241\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePlasma osmolarity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e26.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e32.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.008874509\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eModel evaluation\u003c/h3\u003e\n\u003cp\u003eThe diagnostic accuracies of traditional machine learning models (SVM, RF, XGBoost, Decision Tree, LightGBM) were 82.88%, 80.37%, 91.45%, 91.82%, and 90.73%, respectively, significantly lower than those of the computer vision model (ResNet). The trained multimodal deep learning model (ResNet50-Clinical) exhibited the best performance in terms of accuracy, sensitivity, specificity, and F1 score, outperforming traditional machine learning models and single computer vision models significantly. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e displays the diagnostic performance of each model using the validation dataset. The confusion matrix of the ResNet50-Clinical model can be found in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.a.\u003c/p\u003e \u003cp\u003eA recall curve was plotted for the ResNet50-Clinical model, achieving an AUC of 0.97 (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.b). The ROC curves of the ResNet50-Clinical model and those of the physicians' manual judgments obtained from the human-machine double-blind experiment showed that the diagnostic performance of the ResNet50-Clinical model (AUC\u0026thinsp;=\u0026thinsp;0.87) was close to that of pediatric clinical doctors with advanced titles (AUC\u0026thinsp;=\u0026thinsp;0.88), suggesting that the model could effectively assist junior pediatricians, doctors with limited experience in diagnosing Kawasaki disease, and doctors in underdeveloped medical areas in forming early diagnostic opinions on Kawasaki disease (Figs.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.c and 2.d).\u003c/p\u003e \u003cp\u003eResNet50-Clinical model's GradCAM results, superimposed on palm and conjunctiva images, confirm the attention strategy of the ResNet50-Clinical model. We observed that the model's attention is primarily concentrated on the peeling area of the palm, fingertips, and sites of conjunctivitis, consistent with clinical experience, thus validating the feasibility of our model. Due to the presence of non-removable identifiable information in palm and conjunctiva images of children, which may compromise patient privacy, we conducted searches and curated palm and conjunctiva images of Kawasaki disease patients through search engines such as Google for result demonstration purposes, evaluating the model's attention as depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDiagnostic performance of deep learning algorithms. The best performance is indicated in bold, while the second-best performance is indicated with an underline.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSensitivity\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSpecificity\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eF1 Score\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAUC\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.828\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.853\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.900\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.851\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.808\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.877\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRF\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.803\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.822\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.877\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.879\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.871\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.850\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eXGboost\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.914\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.836\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.852\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.881\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.804\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.844\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDecisiontree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.918\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.906\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.809\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.842\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.871\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.858\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLightGBM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.907\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.886\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.814\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.905\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.828\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.850\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.748\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.745\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.751\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.755\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.750\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.748\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.938\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.951\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.92\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.929\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.940\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.936\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.932\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.947\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.885\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.857\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.923\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.916\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet101\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.858\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.830\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.891\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.891\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.860\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.861\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet152\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.946\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e0.975\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.920\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.917\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.945\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.948\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet50-Clinical\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e0.967\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e0.967\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e0.936\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e0.941\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e0.964\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e0.972\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study has devised a multimodal model leveraging artificial intelligence to aid clinical physicians in diagnosing Kawasaki disease in medically underserved regions. To the best of our knowledge, we are the first to incorporate a multimodal approach that integrates clinical symptoms and blood laboratory test results for aiding in the diagnosis of Kawasaki disease. GradCAM visualization highlighted regions of significance in hand swelling and peeling, as well as conjunctivitis, validating the credibility of our model. While the results of the human-machine double-blind trial in the external validation group were slightly lower than those of the test set, the performance of our multimodal model is essentially equivalent to that of senior clinical practitioners.\u003c/p\u003e \u003cp\u003eThis study reveals that when confronted with the limitation of sparse medical image data, as the network depth increases, even the performance of the ResNet model approaches stability. Introducing a simpler CNN network for late fusion can effectively enhance the final model's classification performance while reducing both training time and hardware requirements. Moving forward, the integration of multiple modalities (such as laboratory test results, radiological findings, pathological images, and medical histories) on an enlarged scale may offer a cost-effective solution in the future development of medical imaging.\u003c/p\u003e \u003cp\u003eOur multimodal model can assist clinical doctors in underdeveloped medical areas to diagnose Kawasaki disease, thereby reducing complications such as coronary artery damage caused by misdiagnosis and missed diagnosis.\u003c/p\u003e \u003cp\u003eLimitations: Firstly, as this study is a single-center study, in order to ensure the generalizability of the model, we plan to conduct validation and testing of the model with a larger dataset from multiple centers. Secondly, in this study, under the premise of providing only palm and conjunctival images as well as laboratory examination indicators, the performance of the model is similar to that of pediatricians with advanced titles in terms of diagnostic accuracy. We plan to establish a larger-scale dataset in the next step to improve the accuracy of the multimodal model. Thirdly, for the diagnosis of Kawasaki disease, the nature of fever and accompanying symptoms included in the medical records are also crucial. Our multimodal model currently can only handle blood routine and image information. We plan to add the module of the large language model NLM in the next step to fully explore the clinical data of Kawasaki disease in medical records and further improve the performance of the model.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn this study, we have developed a multimodal model using deep learning to assist clinical doctors in diagnosing Kawasaki disease in medically underserved areas. This model has broad prospects for assisting doctors in the diagnosis and treatment of Kawasaki disease, reducing the risk of coronary artery damage, and better protecting children's health.\u003c/p\u003e "},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eLZ, LG, JZ, WS, and PS conceived and planned the model design approach.LZ, LG, JZ, and WS conducted the model training and initial evaluations.LZ, PS, and JZ designed and performed the human-machine double-blind experiments.PS, WS, LZ, LG, JZ, and others participated in building the database.LZ, LG, JZ, WS, and PS contributed to interpreting the results.LZ took the lead in writing the manuscript.All authors provided critical feedback and helped complete the research, analysis, and manuscript.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eData availabilityThe laboratory examination data and model codes can be applied for by contacting the first author ([email protected]) upon the reader's request.However, the palm and conjunctival image data cannot be publicly shared or displayed due to the presence of identifiable information of the patients, such as palm prints and irises, which cannot be removed.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eWang Z, Xie L, Ding G, Song S, Chen L, Li G, Xia M, Han D, Zheng Y, Liu J, et al. Single-cell RNA sequencing of peripheral blood mononuclear cells from acute Kawasaki disease patients. Nat Commun (2021) 12:5444. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41467-021-25771-5\u003c/span\u003e\u003cspan address=\"10.1038/s41467-021-25771-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDenby KJ, Clark DE, Markham LW. Management of Kawasaki disease in adults. Heart Br Card Soc (2017) 103:1760\u0026ndash;1769. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1136/heartjnl-2017-311774\u003c/span\u003e\u003cspan address=\"10.1136/heartjnl-2017-311774\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang Y, Wang Y, Zhang L, Xia L, Zheng M, Zeng Z, Liu Y, Yarovinsky T, Ostriker AC, Fan X, et al. Reduced Platelet miR-223 Induction in Kawasaki Disease Leads To Severe Coronary Artery Pathology Through a miR-223/PDGFRβ Vascular Smooth Muscle Cell Axis. Circ Res (2020) 127:855\u0026ndash;873. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1161/CIRCRESAHA.120.316951\u003c/span\u003e\u003cspan address=\"10.1161/CIRCRESAHA.120.316951\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSingh S, Vignesh P, Burgner D. The epidemiology of Kawasaki disease: a global update. Arch Dis Child (2015) 100:1084\u0026ndash;1088. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1136/archdischild-2014-307536\u003c/span\u003e\u003cspan address=\"10.1136/archdischild-2014-307536\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcCrindle BW, Rowley AH, Newburger JW, Burns JC, Bolger AF, Gewitz M, Baker AL, Jackson MA, Takahashi M, Shah PB, et al. Diagnosis, Treatment, and Long-Term Management of Kawasaki Disease: A Scientific Statement for Health Professionals From the American Heart Association. Circulation (2017) 135:e927\u0026ndash;e999. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1161/CIR.0000000000000484\u003c/span\u003e\u003cspan address=\"10.1161/CIR.0000000000000484\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCordier N, Delingette H, Le M, Ayache N. Extended Modality Propagation: Image Synthesis of Pathological Cases. IEEE Trans Med Imaging (2016) 35:2598\u0026ndash;2608. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TMI.2016.2589760\u003c/span\u003e\u003cspan address=\"10.1109/TMI.2016.2589760\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang K, He R, Wang L, Wang W, Tan T. Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval. IEEE Trans Pattern Anal Mach Intell (2016) 38:2010\u0026ndash;2023. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TPAMI.2015.2505311\u003c/span\u003e\u003cspan address=\"10.1109/TPAMI.2015.2505311\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYawen Huang null, Ling Shao null, Frangi AF. Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning. \u003cem\u003eIEEE Trans Med Imaging\u003c/em\u003e (2018) 37:815\u0026ndash;827. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TMI.2017.2781192\u003c/span\u003e\u003cspan address=\"10.1109/TMI.2017.2781192\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen M, Carass A, Jog A, Lee J, Roy S, Prince JL. Cross contrast multi-channel image registration using image synthesis for MR brain images. Med Image Anal (2017) 36:2. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.media.2016.10.005\u003c/span\u003e\u003cspan address=\"10.1016/j.media.2016.10.005\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu D, Zhang H, Wu Z, Wang F, Wang L, Smith JK, Lin W, Li G, Shen D. Disentangled-Multimodal Adversarial Autoencoder: Application to Infant Age Prediction With Incomplete Multimodal Neuroimages. IEEE Trans Med Imaging (2020) 39:4137\u0026ndash;4149. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TMI.2020.3013825\u003c/span\u003e\u003cspan address=\"10.1109/TMI.2020.3013825\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHao S, Jin B, Tan Z, Li Z, Ji J, Hu G, Wang Y, Deng X, Kanegaye JT, Tremoulet AH. A classification tool for differentiation of Kawasaki disease from other febrile illnesses. J Pediatr (2016) 176:114\u0026ndash;120. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.jpeds.2016.05.060\u003c/span\u003e\u003cspan address=\"10.1016/j.jpeds.2016.05.060\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSosa T, Brower L, Divanovic A. Diagnosis and management of Kawasaki disease. JAMA Pediatr (2019) 173:278\u0026ndash;279. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jamapediatrics.2018.3307\u003c/span\u003e\u003cspan address=\"10.1001/jamapediatrics.2018.3307\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLam JY, Shimizu C, Tremoulet AH, Bainto E, Roberts SC, Sivilay N, Gardiner MA, Kanegaye JT, Hogan AH, Salazar JC. A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study. Lancet Digit Health (2022) 4:e717\u0026ndash;e726. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/S2589-7500(22)00149-2\u003c/span\u003e\u003cspan address=\"10.1016/S2589-7500(22)00149-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi C, Liu Y-C, Zhang D-R, Han Y-X, Chen B-J, Long Y, Wu C. A machine learning model for distinguishing Kawasaki disease from sepsis. Sci Rep (2023) 13:12553. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41598-023-39745-8\u003c/span\u003e\u003cspan address=\"10.1038/s41598-023-39745-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTsai C-M, Lin C-HR, Kuo H-C, Cheng F-J, Yu H-R, Hung T-C, Hung C-S, Huang C-M, Chu Y-C, Huang Y-H. Use of Machine Learning to Differentiate Children With Kawasaki Disease From Other Febrile Children in a Pediatric Emergency Department. JAMA Netw Open (2023) 6:e237489\u0026ndash;e237489. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jamanetworkopen.2023.7489\u003c/span\u003e\u003cspan address=\"10.1001/jamanetworkopen.2023.7489\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou Y, Kumar A. Human Identification Using Palm-Vein Images. IEEE Trans Inf Forensics Secur (2011) 6:1259\u0026ndash;1274. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TIFS.2011.2158423\u003c/span\u003e\u003cspan address=\"10.1109/TIFS.2011.2158423\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu E, Nemati S, Tremoulet AH. A deep convolutional neural network for Kawasaki disease diagnosis. Sci Rep (2022) 12:11438. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41598-022-15495-x\u003c/span\u003e\u003cspan address=\"10.1038/s41598-022-15495-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBagade SB, Patil KD, Hatware KV, Pingale PL, Mhatre SVC. Palm Vein Technology: A Biometric Intelligence System for patients Authentication and Safety. Res J Pharm Technol (2023) 16:5554\u0026ndash;5561. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.52711/0974-360X.2023.00898\u003c/span\u003e\u003cspan address=\"10.52711/0974-360X.2023.00898\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSiew Chin C, Beng Jin AT, Chek Ling DN. High security Iris verification system based on random secret integration. Comput Vis Image Underst (2006) 102:169\u0026ndash;177. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.cviu.2006.01.002\u003c/span\u003e\u003cspan address=\"10.1016/j.cviu.2006.01.002\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNedjah N, Wyant RS, Mourelle LM, Gupta BB. Efficient yet robust biometric iris matching on smart cards for data high security and privacy. Future Gener Comput Syst (2017) 76:18\u0026ndash;32. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.future.2017.05.008\u003c/span\u003e\u003cspan address=\"10.1016/j.future.2017.05.008\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 \u003cem\u003eIEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e. Las Vegas, NV, USA: IEEE (2016). p. 770\u0026ndash;778 doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2016.90\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2016.90\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. \u003cem\u003e2009 IEEE Conference on Computer Vision and Pattern Recognition\u003c/em\u003e. (2009). p. 248\u0026ndash;255 doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2009.5206848\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2009.5206848\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eImambi S, Prakash KB, Kanagachidambaresan GR. \u0026ldquo;PyTorch.,\u0026rdquo; In: Prakash KB, Kanagachidambaresan GR, editors. \u003cem\u003eProgramming with TensorFlow: Solution for Edge Computing Applications\u003c/em\u003e. EAI/Springer Innovations in Communication and Computing. Cham: Springer International Publishing (2021). p. 87\u0026ndash;104 doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-3-030-57077-4_10\u003c/span\u003e\u003cspan address=\"10.1007/978-3-030-57077-4_10\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSelvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. Int J Comput Vis (2020) 128:336\u0026ndash;359. doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11263-019-01228-7\u003c/span\u003e\u003cspan address=\"10.1007/s11263-019-01228-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBorg M, Jabangwe R, Aberg S, Ekblom A, Hedlund L, Lidfeldt A. Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for Vision AI? 2021 \u003cem\u003eIeee International Conference on Software Testing, Verification and Validation Workshops (icstw 2021)\u003c/em\u003e. Los Alamitos: Ieee Computer Soc (2021). p. 175\u0026ndash;181 doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICSTW52544.2021.00039\u003c/span\u003e\u003cspan address=\"10.1109/ICSTW52544.2021.00039\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM plus plus: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks. 2018 \u003cem\u003eIeee Winter Conference on Applications of Computer Vision (wacv 2018)\u003c/em\u003e. New York: Ieee (2018). p. 839\u0026ndash;847 doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/WACV.2018.00097\u003c/span\u003e\u003cspan address=\"10.1109/WACV.2018.00097\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Artificial intelligence, Kawasaki Disease, Multimodal Model, Attention Analysis, Computer-aided diagnosis","lastPublishedDoi":"10.21203/rs.3.rs-4323083/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4323083/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eObjective\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe propose a new multimodal artificial intelligence model that facilitate the differentiation of Kawasaki disease through the integration of clinical symptom photographs and laboratory examination indices.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study is a retrospective investigation based on laboratory examination data, palm images, and conjunctival image databases of 620 children (comprising those with both healthy physical examinations and Kawasaki disease) who visited our hospital between January 2022 and January 2024. The multimodal model was trained and evaluated using this database. GradCAM was incorporated to analyze the attention mechanisms of the multimodal model. A human-machine double-blind controlled trial was designed to evaluate the diagnostic accuracy of the obtained multimodal model and senior clinical physicians with advanced qualifications on external dataset.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe performance evaluation of the multimodal model on the validation set yielded an area under the curve of 0.97 and an accuracy of 0.96.The GradCAM analysis reveals that the model's attention is concentrated on areas such as palm swelling and peeling, as well as conjunctivitis, which aligns with clinical reasoning.The human-machine double-blind trial validated that the multimodal model and senior pediatric physicians with advanced qualifications achieved comparable accuracy rates in identifying cases within an independent external cohort.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe multimodal model we developed can assist junior doctors in diagnosing Kawasaki disease, providing a new approach for the auxiliary diagnosis of Kawasaki disease in medically underserved areas.\u003c/p\u003e","manuscriptTitle":"Utilizing Multimodal Data for Diagnosis of Kawasaki Disease: An AI Approach","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-07 18:37:23","doi":"10.21203/rs.3.rs-4323083/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4c9c6160-aa25-4b3a-bd3d-62812950e9e0","owner":[],"postedDate":"May 7th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":31496470,"name":"Health sciences/Rheumatology/Rheumatic diseases/Paediatric rheumatic diseases"},{"id":31496471,"name":"Physical sciences/Mathematics and computing/Computer science"}],"tags":[],"updatedAt":"2024-10-29T05:09:32+00:00","versionOfRecord":[],"versionCreatedAt":"2024-05-07 18:37:23","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4323083","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4323083","identity":"rs-4323083","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}