A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists

A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists Kei Yamamoto, Goh Ohji, Isao Miyatsuka, Kei Furui-Ebisawa, Ataru Moriya, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4284383/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Gram staining results interpreted by microbiology specialists (MS) were compared to a developed computer-aided diagnosis system (CAD) using artificial intelligence for interpretation. Methods Using a non-inferiority study, CAD-predicted Gram stain results, generated from images of an iPhone camera in two hospitals, were compared to those of MS, between 1 April and 31 December 2022. The prediction accuracies, classified as Class 1 based on the bacterial morphology, were the primary endpoint. The datasets were created according to the separate hospitals; 10 MS interpreted 153 images from each hospital. CAD predicted 306 images overall. Results The accuracies (95% confidence intervals) of MS and CAD predictions were 83.0% (81.6–84.3) and 87.9% (83.7–91.3), respectively; with a difference of –4.93% (–8.43 to –0.62), indicating non-inferiority of CAD. Conclusions Non-inferiority of CAD to MS predictions was demonstrated; therefore, CAD urine Gram staining can be used to select empirical antibiotics in facilities without MS. Artificial intelligence Computer-aided diagnosis Gram staining Non-inferiority trial Urinary tract infection Figures Figure 1 Figure 2 Background Prompt administration of antimicrobial agents is ideal after diagnosis because bacterial infection outcomes can be improved by appropriate empirical antimicrobial agents at an early stage [1,2]. However, traditional culture tests for bacteria require time to determine the causative organism. Multiplex nucleic acid amplification tests and matrix-assisted laser desorption/ionization time-of-flight mass spectrometry can shorten the turnaround time, contributing to antimicrobial stewardship with appropriate support [3]. Although new point-of-care tests for urinary tract infections (UTI) that directly use urine specimens are being developed [4,5], such new testing methods for initial treatment are unavailable now. Moreover, that novel testing devices are expensive to install and maintain is a barrier against expanding new testing methods, even with ground-breaking ones. Although the choice of empirical antibiotics is based on treatment guidelines, patient risk, and symptoms, point-of-care Gram staining (PCGS) is also a useful test for empirical antimicrobial selection. Urine Gram staining contributes to UTI diagnosis [6,7] and empirical antimicrobial agent selection, reducing broad-spectrum antimicrobial agent usage and antimicrobial agent cost compared to guideline-based selection [8]. Urine specimens are relatively easy to collect, and specimen quality evaluation is not required; however, proficiency is required in reading Gram stain findings for accurate interpretation [9,10]. Facilities where clinical technologists can perform Gram staining of urine specimens after hours are rare, with a reported rate of 38% in Japan [11]. The accuracy of Gram stain decipherment between microbiology specialists (MS) and non-specialists showed significantly lower accuracy among non-specialists [12]. To put into practical use a tool to assist in the selection of an appropriate initial antimicrobial agent for urinary tract infection, with readings equivalent to MS in a situation such as Japan where 24-hour Gram stain decipherment by MS is not possible, we developed an inexpensive computer-aided diagnosis system (CAD) to speculate on bacterial classification (Table 1) in urine specimens by artificial intelligence (AI) and conducted a non-inferiority study to prove that CAD can decipher Gram stain findings to the same degree of accuracy as MS. Methods Study design A retrospective observational study using residual clinical specimens from two tertiary care hospitals (the National Center for Global Health and Medicine [NCGM] and Kobe University Hospital [KUH]) was conducted. The primary analysis was a non-inferiority test of CAD versus 20 MS in the two hospitals. MS included physicians and clinical laboratory technicians who were infectious disease specialists certified by the Japanese Association of Infectious Diseases, and technologists in microbiology certified by the College of Laboratory Medicine in Japan or by Medical Technologists in Clinical Microbiology. Appendix S1 shows the algorithm for developing the CAD model. Because residual clinical specimens were used retrospectively, the requirement of individual informed consent was waived by providing an opt-out opportunity. The research methods were collectively reviewed and approved by the Institutional Review Board of the NCGM (NCGM-S-004480-02). Inclusion criteria and collected information Gram-stained slides of urine specimens sampled between 1 April and 30 June 2022 at NCGM and 1 April and 31 December 2022 at KUH were used. Gram staining was performed at both sites using Bartholomew and Mittwer (B&M) methods. Clinical information including the sampling date, specimen type, Gram staining findings, and species identification results was collected. After the routine examination, samples with both Gram staining findings and confirmed species identification results were included. Samples were excluded if a single organism was observed by microscopy, but more than one organism was identified by bacterial culture (e.g., Gram-negative rod [GNR] was observed by microscopy, but Staphylococcus aureus and Escherichia coli were isolated), or identified organisms differed (e.g., Gram-positive rod [GPR] was observed by microscopy but Escherichia coli was isolated). Note that the images used as training data are based on specimens collected between February 1, 2020, and March 31, 2023, except for April 1 through December 31, 2022, and did not overlap with these experimental images. Table 2 shows the number of images captured according to the number of samples by class. For classifications with no more than three samples, spiked samples, which were generated using the ATCC standard strains (Supplementary Appendix S2), were added to make it at least three samples. In addition, 10 images were captured per sample from 15 to 50 samples randomly selected among samples with undetected bacteria during microscopy (labeled as ‘None’), and samples with two or more bacteria with different morphology (e.g., not applicable to cases where Gram-positive cocci [GPC] clusters and chains were mixed but applicable where GPC and Gram-negative cocci [GNC] were mixed), labeled as ‘Polymicrobial’. From these samples, three were randomly selected in each Class 2 category, and three image files per sample were included in the non-inferiority study (Appendix). The devices used for imaging and decipherment are presented in Appendix S3. Outcome The primary outcome, accuracy, was determined as the correct answer when the correct data matched the predicted data. As secondary outcomes, the macro-average sensitivity, recall, precision, and F1 score (harmonic mean of recall and precision) for Classes 1 and 2, respectively, were calculated. The agreement between correct and incorrect answers for MS and CAD was also evaluated as a secondary outcome. Sample size calculation The pilot study results showed accuracies of MS and CDA of 96.4% and 95.7%, respectively, in discriminating bacterial morphology (excluding ‘None’ and ‘Polymicrobial’ categories). Power was calculated for 306 images in the dataset, 306 predictions by CAD, and 3060 predictions by MS, analyzed based on a 5% non-inferiority margin and one-sided alpha error of 2.5%. The power was greater than 80% (approximately 83%). Statistical analysis Accuracy was expressed as point estimates with 95% confidence intervals. Non-inferiority was analyzed using Farrington and Manning analysis at a significance level of 5%. The agreement between correct and incorrect answers per image for each CAD and MS was evaluated using Cohen’s kappa coefficient and Gwet’s “first-order agreement coefficient” (AC1) statistics. Inter-rater reliability was evaluated using Fleiss’s kappa coefficient and Gwet’s AC1 statistics. R version 4.3.0 (R Foundation for Statistics. Computing, Vienna, Austria) was used for statistical analysis, and PASS 2023 (NCSS, LLC, Utah, USA) was used for sample size calculation. Results Of the 1,388 and 1,869 eligible samples, 253 and 152 clinical samples were included from NCGM and KUH, respectively (Supplementary Table S4). Six spiked samples (three each of GNC and other GNR glucose non-fermenting bacteria) and 11 spiked samples (three GNC, three other GNR glucose non-fermenting bacteria, two Enterococcus faecium , one other GPC, and two Klebsiella oxytoca ) were included in NCGM and KUH, respectively. The MS group included 9 physicians and 11 laboratory technicians. Class 1 The overall accuracy for the 3,060 predictions by MS was 83.0% (81.6 to 84.3), and individual results ranged from 75.1% to 89.5%. The Fleiss’s kappa coefficient and AC1 statistic were 0.79 and 0.85 in NCGM and 0.68 and 0.77 in KUH, with relatively high inter-rater agreement in both sites. The confusion matrix for the MS is shown in Fig. 1a. Predictions were relatively accurate for GNR, yeast, and ‘None’ categories but a little less accurate when classifying polymicrobial, GNC, GPC, and GPR. The macro-average sensitivity was 79.1%. The results for other secondary outcomes are shown in Table 3. The overall accuracy for the 306 predictions by CAD was 87.9% (83.7 to 91.3), with a macro-average sensitivity of 79.1%. The confusion matrix of CAD is shown in Fig. 1b. Predictions were relatively accurate for GNR, GNC, GPC, yeast, and ‘None’ categories but a little less accurate when classifying polymicrobial and GPR. The accuracy difference was −4.93% (95% confidence interval [CI]: −8.43 to −0.62) ( p =1.0), indicating non-inferiority of deciphering by CAD compared to MS, based on the Farrington and Manning analysis. In 2,323 predictions, both MS and CAD were correct but were incorrect in 256 predictions. Among the remaining images, 216 and 256 were correct only for MS and CAD, respectively. The agreement between correct and incorrect answers for MS and CAD was not very high, at a Cohen’s kappa coefficient of 0.42. However, the AC1 statistic showed a high agreement rate of 0.84. Class 2 The accuracy for MS was 34.3% (32.6 to 36.0), ranging from 24.2% to 46.4% for individuals. The Fleiss’s kappa coefficient and AC1 statistic of 0.40 and 0.44 for NCGM and 0.31 and 0.37 for KUH indicated a low inter-rater agreement. The confusion matrix for the MS is shown in Fig. 2a. The predictions were often incorrect when classifying GPC (U-02–U-06) and GNR (U-08–U-14). In particular, the precision and recall of GNR tended to be low, ranging from 9.2% to 28.4% and 3.3% to 31.1%, respectively (Supplementary Table S5). The accuracy of CAD was 46.1% (40.4 to 51.8). The predictions were often incorrect when classifying GNRs, similar to MS, whereas these were relatively correct for GPCs, especially in the GPC cluster. The precision and recall of GNRs tended to be higher than those of MS, ranging from 14.3% to 40.6% and 5.6% to 72.2%, respectively (Supplementary Table S5). In particular, the F1 scores for glucose non-fermenting bacteria were 38.4% for MS and 60.3% for CAD (Supplementary Table S6). The accuracy difference was –11.8% (95% CI: –17.6 to –6.04) (p=1.0), indicating non-inferiority of deciphering by CAD compared to MS, based on the Farrington and Manning analysis. However, the 95% CI demonstrated the superiority of CAD. Both predictions by MS and CAD were correct with 749 images but incorrect with 1,350 predictions. Among the remaining images, 300 and 661 were correct only for MS and CAD, respectively. Compared to Class 1, the agreement on correct answers was lower for MS and CAD, with a Cohen’s kappa coefficient of 0.36 and an AC1 statistic of 0.69. Discussion Although there have been attempts to use image AI analysis based on deep learning methods to assess Gram staining findings with blood cultures [13] and bacterial vaginosis [14], this is the first report to compare and validate Gram staining findings between CAD and MS using clinical urine specimens. In Class 1 differentiation, CAD performed better than MS with GPR and GNC but less with polymicrobial, based on F1 scores. The accuracy of the Gram stain readings by MS was 83%. The Gram staining error rate in discriminating bacterial morphologic features, verified in multiple institutions, was approximately 5% [15], and microbiologists in Japan are also surveyed annually to check the quality of their Gram stain interpretation by Japanese Association of Medical Technologists [Undisclosed data], in which the errors were below 5% most years. Nonetheless, MS performance in the present study was poor compared with that of this standard error rate. However, accuracy in discriminating the five bacterial classifications (GPC, GPR, GNR, GNC, and yeast) by 11 MS in urine specimens was 96.4%, the same as this error rate, in a pilot study [12]. Although the pilot study findings have been validated except those of polymicrobial and ‘None’ classifications, the prediction for polymicrobial by MS in this study was poor as well as that by CAD. In addition, the deciphering of Gram stain findings, normally performed using microscopy, was performed in this study using a single image file, for fairness, when assessing using CAD because of the no significant difference in accuracy between microscopy and single image assessment results in the pilot study [12]. However, this may have affected the prediction of polymicrobial classification using the single image rather than microscopy since the impact of the single-image judgment was verified in the pilot study. The accuracy of prediction for GPR was also poor with MS. Although a single specimen was used in the pilot study, the accuracy of predictions by both imaging and microscopy was low (data not shown). Corynebacterium spp. in urine specimens were rare compared to those in GNR and GPC. In this study, because samples were not selected to reflect their epidemiology, it is possible that this study’s higher GPR percentage than the proportion in the real world affected the overall accuracy. As reported, Corynebacterium spp. may form coccoids, depending on the culture condition [16,17], and such changes may occur more likely with urine. Although the reason for this is unclear, the use of a single-image judgment and the high rate of GPR, unlike actual epidemiology, may have contributed to the high error rate of classification of MS. The impact of accurate Gram staining prediction on antimicrobial use remains unclear, but the results of Gram staining are known to influence empirical antimicrobial selection for common infectious diseases, such as community-acquired pneumonia [18]. A randomized controlled trial on ventilator-associated pneumonia showed that broad-spectrum antimicrobial use was reduced compared to empirical antimicrobial selection by guideline compliance, with no change in clinical efficacy [19]. The possibility that a narrower spectrum of antibiotics can be selected using PCGS has also been shown when antimicrobials were selected using either PCGS or guidelines in patients with infectious diseases, with UTI accounting for approximately half of cases [8]. In pediatric patients with UTI, PCGS showed high diagnostic performance and concordance with bacteria detected in culture, suggesting the possibility of a more efficient selection of susceptible agents [20]. In adult patients with UTI, Taniguchi et al. noted that PCGS may contribute to narrow-spectrum antimicrobial selection because antimicrobial agents selected based on PCGS results and information, such as antibiograms, are susceptible to many detected uropathogens [21]. Although the predictive value of antibiograms for appropriate empirical antimicrobials may not be high [22], antibiograms are one of the few pieces of information available for empirical treatment selection, and their use may contribute to reducing broad-spectrum antimicrobial usage. As with the eligible specimens in Appendix Table S4, GNR is the main pathogen of urinary tract infection [23], and the F1 value for GNR of CAD is more than 95% and higher than that of MS. Therefore, the prerequisites for estimating the causative organism of urinary tract infection are satisfied. Differentiating bacterial species by Gram staining showed differentiation between S. aureus and other staphylococci in blood culture [24-26] and whether enterococci were penicillin-resistant [27]. Furthermore, the process of differentiating non-fermenting bacteria using the length and thickness ratio for GNR has been reported [28]. Although these studies showed relatively good performance, achieving sufficient accuracy when differentiating among various species remains challenging, as in this study, rather than within certain species. However, the Class 2 performance by MS was not high either. Although the differential performance of the GPC cluster was relatively good, the precision and recall of the main pathogens, such as GNRs, tended to be lower for MS than for CAD (Supplementary Table S5 and S6). The F1 score of non-fermenting bacteria was 60% in CAD. Although the results of decipherment by both CAD and MS in this study did not look good, it is true that PCGS decipherment by medical professionals, including MS, in actual practice has reduced the use of broad-spectrum antimicrobial agents compared with guideline-based antimicrobial use [8]. Therefore, future clinical evaluation is needed, though, it is possible that the decipherment of PCGS CAD, which was not inferior to that of MS, can be used effectively in PCGS for choosing empiric antimicrobial therapy. A limitation of this study is that its accuracy has not been verified at other centers, and generalizability remains an issue. Gram staining of urine specimens from facilities where training data were not collected was performed satisfactorily using a similar method and imaging device (data not shown). Although the included samples did not reflect the UTI epidemiological data, it is possible to generalize on bacterial species. This is because we estimated the top detected species in a previous single-center study [23] and national surveillance in Japan [29]. However, it is unclear whether CAD can be generalized to Gram staining methods other than the B&M methods and imaging data captured by smartphone devices other than the iPhone. Further studies may resolve this by increasing variations in the training data. Conclusions Predicting Gram staining findings in urine specimens by CAD is non-inferior to that by MS following B&M methods for staining samples, and images were captured using an iPhone camera. Although CAD could facilitate infectious disease treatment using PCGS for urine when MS is unavailable to interpret Gram staining findings, future studies are needed to determine its generalizability. Abbreviations AC1 - Agreement Coefficient 1 AI - Artificial Intelligence ATCC - American Type Culture Collection B&M - Bartholomew and Mittwer CAD - Computer-Aided Diagnosis CI - Confidence Interval F1 Score - F1 Score (harmonic mean of recall and precision) GNC - Gram-Negative Cocci GNR - Gram-Negative Rod GPC - Gram-Positive Cocci GPR - Gram-Positive Rod KUH - Kobe University Hospital MS - Microbiology Specialists NCGM - National Center for Global Health and Medicine PCGS - Point-of-Care Gram Staining UTI - Urinary Tract Infection Declarations Ethics approval and consent to participate The research methods were collectively reviewed and approved by the Institutional Review Board of the NCGM (NCGM-S-004480-02). Consent for publication Because residual clinical specimens were used retrospectively, the requirement of individual informed consent was waived by providing an opt-out opportunity. Availability of data and materials All data associated with this study are presented in the main text or appendix and are available from the authors upon reasonable request. Individual-level, de-identified participant data are not available for publication. Competing interests K.Y. received research grants from Fujirebio Inc., Mizuho Medy Co. Ltd., VisGene Co. Ltd., Canon Medical Systems Corp., Sanyo Chemical Industries Ltd., CarbGeM Inc., Sysmex Corp., and Toyobo Co. Ltd. outside the submitted work and has patent rights to this CAD, but no income has been generated from it. I.M. and S.M. are employees of CarbGeM Inc. Funding This work was supported by AMED (grant number JP23hk0102076h0003). Authors' contributions Conceptualization: KY, GO, KFE, YU, and NO Data curation: KY, AM, M. Kurokawa, GO, KFE, KO, M. Kusuki, SM, and IM Formal analysis: KY, SM, and YU Funding acquisition: KY, GO, and IM Investigation: KY, GO, KFE, SM, and IM Methodology: KY, AM, GO, KFE, and IM Project administration: KY, GO, IM, and NO Resources: AM, M. Kurokawa, KO, M. Kusuki, SM, and IM Software: SM and IM Supervision: NO Visualization: KY and SM Writing – original draft: KY Writing – review & editing: GO, IM, KFE, AM, SM, HN, M. Kurokawa, KO, M. Kusuki, YU, and NO Acknowledgements The authors express their deepest gratitude to Masami Yoshino for managing the imaging data used in this study. We thank Kayoko Hayakawa, Taketomo Maruki, Yutaro Akiyama, Yuki Moriyama, Ayano Motohashi, Kazuhisa Mezaki, Saeko Kinpara, Taiji Koyama, Kenta Iijima, Saori Kobayashi, Sayaka Matsumoto, Masako Nishida, and Nami Ishida for their help in deciphering the Gram staining. We also thank the staff, Yoshihiko Hirayama, Mayumi Kawakami, Motoko Ishida, Noriko Iwamoto, Gen Yamada, Satoshi Ide, Ayako Okuhama, Akiho Sugita, Nozomu Tsurumaki, Sei Ueda, Wataru Ochi, Reo Iguma, Kaito Mimura, Ryo Sakuma, Mayu Hiraiwa, Shunsuke Matsuoka, Akiko Hamana, Noriko Tomita, and Moto Kimura, at NCGM and KUH for their cooperation. We thank Editage (www.editage.jp) for the English language editing. References Im Y, Kang D, Ko RE, Lee YJ, Lim SY, Park S, et al. Time-to-antibiotics and clinical outcomes in patients with sepsis and septic shock: a prospective nationwide multicenter cohort study. Crit Care. 2022;26(1):19. doi:10.1186/s13054-021-03883-0. Ohnuma T, Chihara S, Costin B, Treggiari MM, Bartz RR, Raghunathan K et al. Association of appropriate empirical antimicrobial therapy with in-hospital mortality in patients with bloodstream infections in the US. JAMA Netw Open. 2023;6(1):e2249353. doi:10.1001/jamanetworkopen.2022.49353. Timbrook TT, Morton JB, McConeghy KW, Caffrey AR, Mylonakis E, LaPlante KL. The effect of molecular rapid diagnostic testing on clinical outcomes in bloodstream infections: A systematic review and meta-analysis. Clin Infect Dis. 2017;64(1):15-23. doi:10.1093/cid/ciw649. Baltekin Ö, Boucharin A, Tano E, Andersson DI, Elf J. Antibiotic susceptibility testing in less than 30 min using direct single-cell imaging. Proc Natl Acad Sci U S A. 2017;114(34):9170-5. doi:10.1073/pnas.1708558114. Arienzo A, Murgia L, Cellitti V, Ferrante V, Stalio O, Losito F, et al. A new point-of-care test for the rapid antimicrobial susceptibility assessment of uropathogens. PLOS ONE. 2023;18(7):e0284746. doi:10.1371/journal.pone.0284746. Wiwanitkit V, Udomsantisuk N, Boonchalermvichian C. Diagnostic value and cost utility analysis for urine Gram stain and urine microscopic examination as screening tests for urinary tract infection. Urol Res. 2005;33(3):220-2. doi:10.1007/s00240-004-0457-z. Williams GJ, Macaskill P, Chan SF, Turner RM, Hodson E, Craig JC. Absolute and relative accuracy of rapid urine tests for urinary tract infection in children: a meta-analysis. Lancet Infect Dis. 2010;10(4):240-50. doi:10.1016/S1473-3099(10)70031-1. Taniguchi T, Tsuha S, Shiiki S, Narita M. Gram-stain-based antimicrobial selection reduces cost and overuse compared with Japanese guidelines. BMC Infect Dis. 2015;15:458. doi:10.1186/s12879-015-1203-6. Guarner J, Street C, Matlock M, Cole L, Brierre F. Improving Gram stain proficiency in hospital and satellite laboratories that do not have microbiology. Clin Chem Lab Med. 2017;55(3):458-61. doi:10.1515/cclm-2016-0556. Munson E, Block T, Basile J, Hryciuk JE, Schell RF. Mechanisms to assess Gram stain interpretation proficiency of technologists at satellite laboratories. J Clin Microbiol. 2007;45(11):3754-8. doi:10.1128/JCM.01632-07. Shime N, Yanagihara K, Watanabe M, Morita M, Sasaki M, Shinagawa M, et al. A multicenter survey for microbiological testing in emergency department. Journal of Japan Society for Clinical Microbiology. 2019;29:28-31. [Japanese]. Ikegaki S, Ohji G, Yamamoto K, Ohnuma K, Ebisawa K, Kusuki M, et al. [O-149: A study of differences in prediction accuracy in urine Gram stain decipherment by decipherment method and rater]. The 97th Annual Meeting of the Japanese Association for Infectious Diseases. Yokohama; 2023/4/28. [Japanese, no abstract]. Smith KP, Kang AD, Kirby JE. Automated interpretation of blood culture Gram stains by use of a deep convolutional neural network. J Clin Microbiol. 2018;56(3):e01521-17. doi:10.1128/JCM.01521-17. Wang Z, Zhang L, Zhao M, Wang Y, Bai H, Wang Y, et al. Deep neural networks offer morphologic classification and diagnosis of bacterial vaginosis. J Clin Microbiol. 2021;59(2):e02236-20. doi:10.1128/JCM.02236-20. Samuel LP, Balada-Llasat JM, Harrington A, Cavagnolo R. Multicenter assessment of Gram stain error rates. J Clin Microbiol. 2016;54(6):1442-7. doi:10.1128/JCM.03066-15. Ichikawa L, Ishigaki S, Matsumura M, Asahara M. Case study of morphological changes of Corynebacterium species to gram-positive cocci. Jpn J Technol. 2018;67:299-306. doi:10.14932/jamt.17-143. Grubb TC. Coccus forms of Corynebacterium diphtheriae. J Infect Dis. 1935;56(1):64-77. doi:10.1093/infdis/56.1.64. Pendergrast J, Marrie T. Reasons for choice of antibiotic for the empirical treatment of CAP by Canadian infectious disease physicians. Can J Infect Dis. 1999;10(5):337-45. doi:10.1155/1999/928438. Yoshimura J, Yamakawa K, Ohta Y, Nakamura K, Hashimoto H, Kawada M, et al. Effect of Gram stain-guided initial antibiotic therapy on clinical response in patients with ventilator-associated pneumonia: the GRACE-VAP randomized clinical trial. JAMA Netw Open. 2022;5(4):e226136. doi:10.1001/jamanetworkopen.2022.6136. Yodoshi T, Matsushima M, Taniguchi T, Kinjo S. Utility of point-of-care Gram stain by physicians for urinary tract infection in children ≤36 months. Med (Baltim). 2019;98(14):e15101. doi:10.1097/MD.0000000000015101. Taniguchi T, Tsuha S, Shiiki S, Narita M. Point-of-care urine Gram stain led to narrower-spectrum antimicrobial selection for febrile urinary tract infection in adolescents and adults. BMC Infect Dis. 2022;22(1):198. doi:10.1186/s12879-022-07194-9. Hasegawa S, Livorsi DJ, Perencevich EN, Church JN, Goto M. Diagnostic accuracy of hospital antibiograms in predicting the risk of antimicrobial resistance in Enterobacteriaceae isolates: A nationwide multicenter evaluation at the Veterans Health Administration. Clin Infect Dis. 2023;77(11):1492-500. doi:10.1093/cid/ciad467. Shigemura K, Tanaka K, Okada H, Nakano Y, Kinoshita S, Gotoh A et al. Pathogen occurrence and antimicrobial susceptibility of urinary tract infection cases during a 20-year period (1983-2002) at a single institution in Japan. Jpn J Infect Dis. 2005;58(5):303-8. doi:10.7883/yoken.JJID.2005.303. Murdoch DR, Greenlees RL. Rapid identification of Staphylococcus aureus from BacT/ALERT blood culture bottles by direct Gram stain characteristics. J Clin Pathol. 2004;57(2):199-201. doi:10.1136/jcp.2003.10538. Hadano Y, Isoda M, Ishibashi K, Kakuma T. Validation of blood culture Gram staining for the detection of Staphylococcus aureus by the ’oozing sign’ surrounding clustered gram-positive cocci: a prospective observational study. BMC Infect Dis. 2018;18(1):490. doi:10.1186/s12879-018-3412-2. Kondo S, Yamada T, Misawa S, Nakamura A, Oguri T. [Morphological identification of Staphylococcus spp. from blood culture bottles based on direct Gram staining]. Kansenshogaku Zasshi. 2008;82(6):656-7. doi:10.11150/kansenshogakuzasshi1970.82.656. [Japanese]. Hayashi T, Yoshida M. The Peanuts Sign: a Novel Clue to the Rapid Discrimination of Penicillin-susceptible Enterococci Based on Gram Staining Findings. Kansenshogaku Zasshi. 2019;93(3):306-11. doi:10.11150/kansenshogakuzasshi.93.306. [Japanese]. Bartlett RC, Mazens-Sullivan MF, Lerer TJ. Differentiation of Enterobacteriaceae, Pseudomonas aeruginosa , and Bacteroides and Haemophilus species in Gram-stained direct smears. Diagn Microbiol Infect Dis. 1991;14(3):195-201. doi:10.1016/0732-8893(91)90032-b. Ministry of Health, Labour and Welfare, Japan. Japan nosocomial infections surveillance (JANIS) [cited Nov 21 2023]. Available from: https://janis.mhlw.go.jp/english/index.asp. Tables Table 1. Gram staining classification Code Class 1 Class 2 U-01 Yeast Candida spp. U-02 GPC GPC cluster U-03 GPC Enterococcus faecalis U-04 GPC Enterococcus faecium U-05 GPC Streptococcus agalactiae U-06 GPC Other GPC U-07 GPR Corynebacterium spp. U-08 GNR Enterobacter cloacae U-09 GNR Escherichia coli U-10 GNR Klebsiella oxytoca U-11 GNR Klebsiella pneumoniae U-12 GNR Other GNR Enterobacteriaceae U-13 GNR Pseudomonas aeruginosa U-14 GNR Other GNR glucose non-fermenting bacteria U-15 GNC GNC Poly Polymicrobial Polymicrobial None None None GPC, Gram-positive cocci; GPR, Gram-positive rod; GNC, Gram-negative cocci; GNR, Gram-negative rod. Table 2. Number of target specimens extracted, and images taken Actual number of samples from each institute Number of samples included Number of images per sample 0–4 3 a 12 5–9 5 7 10–14 10 4 15–19 15 3 ≥20 20 2 a When the available specimens were less than three, to incorporate three specimens, the specimens were supplemented with spiked specimens. Table 3. Precision, recall, and F1 score (harmonic mean) for Class 1 CAD Microbiology specialist Precision Recall F1 Precision Recall F1 None 62.1% 100.0% 76.6% 55.8% 96.7% 70.7% Polymicrobial 44.4% 44.4% 44.4% 62.5% 50.0% 55.6% GPC 93.2% 91.1% 92.1% 87.3% 85.6% 86.4% GPR 75.0% 66.7% 70.6% 75.7% 45.0% 56.4% GNR 95.3% 96.0% 95.7% 89.9% 95.8% 92.7% GNC 100.0% 61.1% 75.9% 60.6% 35.0% 44.4% Yeast 100.0% 94.4% 97.1% 91.7% 85.6% 88.5% CAD, computer-aided diagnosis system; GPC, Gram-positive cocci; GPR, Gram-positive rod; GNC, Gram-negative cocci; GNR, Gram-negative rod; F1, harmonic mean of precision and recall. Additional Declarations Competing interest reported. K.Y. received research grants from Fujirebio Inc., Mizuho Medy Co. Ltd., VisGene Co. Ltd., Canon Medical Systems Corp., Sanyo Chemical Industries Ltd., CarbGeM Inc., Sysmex Corp., and Toyobo Co. Ltd. outside the submitted work and has patent rights to this CAD, but no income has been generated from it. I.M. and S.M. are employees of CarbGeM Inc. Supplementary Files Supplementals1.1.docx GraphicalAbstract.tif Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4284383","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":293869903,"identity":"2c03b3f8-03ac-4add-9ba9-a705d911188a","order_by":0,"name":"Kei Yamamoto","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDklEQVRIiWNgGAWjYBACCTiLvQdVhpmQFgkGnjMka5HIIdJhkjOSn274mWNTxy/59uDDnzsY8vj7VycwfqlgYDfHoUVaIs3sZu+2NAnJ2XnJxrxnGIolbrzdwCxzhoHZsgG7FjmJBLMbvNsOSxjczjGTZmxjSNwgcXYDs2QbA7PBAVxa0r/d/AvScvOMmeRPYrRIS+SY3QbbcoPHTIIXpIW/dwPjRzxaJHvelN2W3ZYmObMnxxjoF4nEGTd4NxxmOCOB0y8Sx9O33Xy7zYafn/2MITDEbBL7+89ufPijwiYZV4gxCCQgcRgbgPEkkcBwmIdBItkAlxb+AyhaICKMPxgY7HBqGQWjYBSMgpEGALfYXAay9UQ0AAAAAElFTkSuQmCC","orcid":"","institution":"National Center for Global Health and Medicine","correspondingAuthor":true,"prefix":"","firstName":"Kei","middleName":"","lastName":"Yamamoto","suffix":""},{"id":293869904,"identity":"e1255ba0-4226-410a-b1cc-c686a5c73d55","order_by":1,"name":"Goh Ohji","email":"","orcid":"","institution":"Kobe University Graduate School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Goh","middleName":"","lastName":"Ohji","suffix":""},{"id":293869905,"identity":"38488983-d525-4285-a867-f6cb81367b42","order_by":2,"name":"Isao Miyatsuka","email":"","orcid":"","institution":"CarbGeM Inc","correspondingAuthor":false,"prefix":"","firstName":"Isao","middleName":"","lastName":"Miyatsuka","suffix":""},{"id":293869906,"identity":"6c01fe57-810b-4a2e-afa0-681a1ff593c1","order_by":3,"name":"Kei Furui-Ebisawa","email":"","orcid":"","institution":"Kobe University Graduate School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Kei","middleName":"","lastName":"Furui-Ebisawa","suffix":""},{"id":293869907,"identity":"5457436c-5ee4-40f2-9c68-81a1548d7485","order_by":4,"name":"Ataru Moriya","email":"","orcid":"","institution":"Ibaraki Higashi National Hospital","correspondingAuthor":false,"prefix":"","firstName":"Ataru","middleName":"","lastName":"Moriya","suffix":""},{"id":293869916,"identity":"01e997dc-1cf3-433c-9b48-2e4692d8ab74","order_by":5,"name":"Shogo Maeta","email":"","orcid":"","institution":"CarbGeM Inc","correspondingAuthor":false,"prefix":"","firstName":"Shogo","middleName":"","lastName":"Maeta","suffix":""},{"id":293869917,"identity":"383c125a-49d2-4b72-9a15-9a2d686b636d","order_by":6,"name":"Hidetoshi Nomoto","email":"","orcid":"","institution":"National Center for Global Health and Medicine","correspondingAuthor":false,"prefix":"","firstName":"Hidetoshi","middleName":"","lastName":"Nomoto","suffix":""},{"id":293869918,"identity":"f14ad6db-f063-4bf1-ae66-a85ce6a8ee08","order_by":7,"name":"Masami Kurokawa","email":"","orcid":"","institution":"National Center for Global Health and Medicine","correspondingAuthor":false,"prefix":"","firstName":"Masami","middleName":"","lastName":"Kurokawa","suffix":""},{"id":293869919,"identity":"17de6800-8b3c-4331-a8b7-052824febe96","order_by":8,"name":"Kenichiro Ohnuma","email":"","orcid":"","institution":"Kobe University Hospital","correspondingAuthor":false,"prefix":"","firstName":"Kenichiro","middleName":"","lastName":"Ohnuma","suffix":""},{"id":293869920,"identity":"2c284c2d-e170-4794-9267-9b8742832c44","order_by":9,"name":"Mari Kusuki","email":"","orcid":"","institution":"Kobe University Hospital","correspondingAuthor":false,"prefix":"","firstName":"Mari","middleName":"","lastName":"Kusuki","suffix":""},{"id":293869921,"identity":"98971cce-4579-4292-b902-ad043f3647a6","order_by":10,"name":"Yukari Uemura","email":"","orcid":"","institution":"National Center for Global Health and Medicine","correspondingAuthor":false,"prefix":"","firstName":"Yukari","middleName":"","lastName":"Uemura","suffix":""},{"id":293869922,"identity":"6ac922bc-4f4a-498b-9a11-d7bf334e6154","order_by":11,"name":"Norio Ohmagari","email":"","orcid":"","institution":"National Center for Global Health and Medicine","correspondingAuthor":false,"prefix":"","firstName":"Norio","middleName":"","lastName":"Ohmagari","suffix":""}],"badges":[],"createdAt":"2024-04-18 01:14:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4284383/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4284383/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":55339529,"identity":"4f887980-a6d8-43a2-915f-7bb072c77bae","added_by":"auto","created_at":"2024-04-26 01:51:43","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":201477,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix for Class 1 predictions, classified by bacterial morphology. (a) Microbiology specialist. (b) Computer-aided diagnosis system. A heat map was created by deriving the proportion from the number of actual values as the denominator and the number of correct predictions as the numerator\u003c/p\u003e","description":"","filename":"Fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-4284383/v1/6ea453aa964f8b4ae88a55b6.png"},{"id":55339531,"identity":"4a44f41f-a073-4961-88fb-fa64d991721a","added_by":"auto","created_at":"2024-04-26 01:51:43","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":346189,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix for Class 2 predictions, classified by detailed bacterial species or groups. (a) Microbiology specialist. (b) Computer-aided diagnosis system. A heat map was created by deriving the proportion from the number of actual values as the denominator and the number of correct predictions as the numerator\u003c/p\u003e","description":"","filename":"Fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-4284383/v1/fc0986684e31246b304e0658.png"},{"id":55340040,"identity":"c29eec73-e9ea-4265-a585-a3c8144a5492","added_by":"auto","created_at":"2024-04-26 01:59:43","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":634081,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4284383/v1/c0f45ecf-9a94-4740-878d-e970a1404173.pdf"},{"id":55339530,"identity":"f3928232-a5ac-4925-97d0-ec92419b4d2c","added_by":"auto","created_at":"2024-04-26 01:51:43","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":47881,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementals1.1.docx","url":"https://assets-eu.researchsquare.com/files/rs-4284383/v1/fa3d905f3bae62ac4ff74c81.docx"},{"id":55339533,"identity":"a594183b-aa17-4378-a622-a61217ce2cd7","added_by":"auto","created_at":"2024-04-26 01:51:43","extension":"tif","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":288198,"visible":true,"origin":"","legend":"","description":"","filename":"GraphicalAbstract.tif","url":"https://assets-eu.researchsquare.com/files/rs-4284383/v1/9bc6a231d9481bb0d5da4c69.tif"}],"financialInterests":"Competing interest reported. K.Y. received research grants from Fujirebio Inc., Mizuho Medy Co. Ltd., VisGene Co. Ltd., Canon Medical Systems Corp., Sanyo Chemical Industries Ltd., CarbGeM Inc., Sysmex Corp., and Toyobo Co. Ltd. outside the submitted work and has patent rights to this CAD, but no income has been generated from it. I.M. and S.M. are employees of CarbGeM Inc.","formattedTitle":"A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists","fulltext":[{"header":"Background","content":"\u003cp\u003ePrompt administration of antimicrobial agents is ideal after diagnosis because bacterial infection outcomes can be improved by appropriate empirical antimicrobial agents at an early stage [1,2]. However, traditional culture tests for bacteria require time to determine the causative organism. Multiplex nucleic acid amplification tests and matrix-assisted laser desorption/ionization time-of-flight mass spectrometry can shorten the turnaround time, contributing to antimicrobial stewardship with appropriate support [3]. Although new point-of-care tests for urinary tract infections (UTI) that directly use urine specimens are being developed [4,5], such new testing methods for initial treatment are unavailable now. Moreover, that novel testing devices are expensive to install and maintain is a barrier against expanding new testing methods, even with ground-breaking ones. Although the choice of empirical antibiotics is based on treatment guidelines, patient risk, and symptoms, point-of-care Gram staining (PCGS) is also a useful test for empirical antimicrobial selection.\u003c/p\u003e\n\u003cp\u003eUrine Gram staining contributes to UTI diagnosis [6,7] and empirical antimicrobial agent selection, reducing broad-spectrum antimicrobial agent usage and antimicrobial agent cost compared to guideline-based selection [8]. Urine specimens are relatively easy to collect, and specimen quality evaluation is not required; however, proficiency is required in reading Gram stain findings for accurate interpretation [9,10]. Facilities where clinical technologists can perform Gram staining of urine specimens after hours are rare, with a reported rate of 38% in Japan [11]. The accuracy of Gram stain decipherment between microbiology specialists (MS) and non-specialists showed significantly lower accuracy among non-specialists [12].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo put into practical use a tool to assist in the selection of an appropriate initial antimicrobial agent for urinary tract infection, with readings equivalent to MS in a situation such as Japan where 24-hour Gram stain decipherment by MS is not possible, we developed an inexpensive computer-aided diagnosis system (CAD) to speculate on bacterial classification (Table 1) in urine specimens by artificial intelligence (AI) and conducted a non-inferiority study to prove that CAD can decipher Gram stain findings to the same degree of accuracy as MS.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cem\u003eStudy design\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eA retrospective observational study using residual clinical specimens from two tertiary care hospitals (the National Center for Global Health and Medicine [NCGM] and Kobe University Hospital [KUH]) was conducted. The primary analysis was a non-inferiority test of CAD versus 20 MS in the two hospitals. MS included physicians and clinical laboratory technicians who were infectious disease specialists certified by the Japanese Association of Infectious Diseases, and technologists in microbiology certified by the College of Laboratory Medicine in Japan or by Medical Technologists in Clinical Microbiology. Appendix S1 shows the algorithm for developing the CAD model.\u003c/p\u003e\n\u003cp\u003eBecause residual clinical specimens were used retrospectively, the requirement of individual informed consent was waived by providing an opt-out opportunity. The research methods were collectively reviewed and approved by the Institutional Review Board of the NCGM (NCGM-S-004480-02).\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eInclusion criteria and collected information\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eGram-stained slides of urine specimens sampled between 1 April and 30 June 2022 at NCGM and 1 April and 31 December 2022 at KUH were used. Gram staining was performed at both sites using Bartholomew and Mittwer (B\u0026amp;M) methods. Clinical information including the sampling date, specimen type, Gram staining findings, and species identification results was collected.\u003c/p\u003e\n\u003cp\u003eAfter the routine examination, samples with both Gram staining findings and confirmed species identification results were included. Samples were excluded if a single organism was observed by microscopy, but more than one organism was identified by bacterial culture (e.g., Gram-negative rod [GNR] was observed by microscopy, but \u003cem\u003eStaphylococcus aureus\u003c/em\u003e and \u003cem\u003eEscherichia coli\u003c/em\u003e were isolated), or identified organisms differed (e.g., Gram-positive rod [GPR] was observed by microscopy but \u003cem\u003eEscherichia coli\u003c/em\u003e was isolated). Note that the images used as training data are based on specimens collected between February 1, 2020, and March 31, 2023, except for April 1 through December 31, 2022, and did not overlap with these experimental images.\u003c/p\u003e\n\u003cp\u003eTable 2 shows the number of images captured according to the number of samples by class. For classifications with no more than three samples, spiked samples, which were generated using the ATCC standard strains (Supplementary Appendix S2), were added to make it at least three samples. In addition, 10 images were captured per sample from 15 to 50 samples randomly selected among samples with undetected bacteria during microscopy (labeled as ‘None’), and samples with two or more bacteria with different morphology (e.g., not applicable to cases where Gram-positive cocci [GPC] clusters and chains were mixed but applicable where GPC and Gram-negative cocci [GNC] were mixed), labeled as ‘Polymicrobial’. From these samples, three were randomly selected in each Class 2 category, and three image files per sample were included in the non-inferiority study (Appendix). The devices used for imaging and decipherment are presented in Appendix S3.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eOutcome\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe primary outcome, accuracy, was determined as the correct answer when the correct data matched the predicted data. As secondary outcomes, the macro-average sensitivity, recall, precision, and F1 score (harmonic mean of recall and precision) for Classes 1 and 2, respectively, were calculated. The agreement between correct and incorrect answers for MS and CAD was also evaluated as a secondary outcome.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSample size calculation\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe pilot study results showed accuracies of MS and CDA of 96.4% and 95.7%, respectively, in discriminating bacterial morphology (excluding ‘None’ and ‘Polymicrobial’ categories). Power was calculated for 306 images in the dataset, 306 predictions by CAD, and 3060 predictions by MS, analyzed based on a 5% non-inferiority margin and one-sided alpha error of 2.5%. The power was greater than 80% (approximately 83%).\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStatistical analysis\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eAccuracy was expressed as point estimates with 95% confidence intervals. Non-inferiority was analyzed using Farrington and Manning analysis at a significance level of 5%. The agreement between correct and incorrect answers per image for each CAD and MS was evaluated using Cohen’s kappa coefficient and Gwet’s “first-order agreement coefficient” (AC1) statistics. Inter-rater reliability was evaluated using Fleiss’s kappa coefficient and Gwet’s AC1 statistics. R version 4.3.0 (R Foundation for Statistics. Computing, Vienna, Austria) was used for statistical analysis, and PASS 2023 (NCSS, LLC, Utah, USA) was used for sample size calculation.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eOf the 1,388 and 1,869 eligible samples, 253 and 152 clinical samples were included from NCGM and KUH, respectively (Supplementary Table S4). Six spiked samples (three each of GNC and other GNR glucose non-fermenting bacteria) and 11 spiked samples (three GNC, three other GNR glucose non-fermenting bacteria, two \u003cem\u003eEnterococcus faecium\u003c/em\u003e, one other GPC, and two \u003cem\u003eKlebsiella oxytoca\u003c/em\u003e) were included in NCGM and KUH, respectively. The MS group included 9 physicians and 11 laboratory technicians.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eClass 1\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe overall accuracy for the 3,060 predictions by MS was 83.0% (81.6 to 84.3), and individual results ranged from 75.1% to 89.5%. The Fleiss’s kappa coefficient and AC1 statistic were 0.79 and 0.85 in NCGM and 0.68 and 0.77 in KUH, with relatively high inter-rater agreement in both sites. The confusion matrix for the MS is shown in Fig. 1a. Predictions were relatively accurate for GNR, yeast, and ‘None’ categories but a little less accurate when classifying polymicrobial, GNC, GPC, and GPR. The macro-average sensitivity was 79.1%. The results for other secondary outcomes are shown in Table 3. The overall accuracy for the 306 predictions by CAD was 87.9% (83.7 to 91.3), with a macro-average sensitivity of 79.1%. The confusion matrix of CAD is shown in Fig. 1b. Predictions were relatively accurate for GNR, GNC, GPC, yeast, and ‘None’ categories but a little less accurate when classifying polymicrobial and GPR. The accuracy difference was −4.93% (95% confidence interval [CI]: −8.43 to −0.62) (\u003cem\u003ep\u003c/em\u003e=1.0), indicating non-inferiority of deciphering by CAD compared to MS, based on the Farrington and Manning analysis. In 2,323 predictions, both MS and CAD were correct but were incorrect in 256 predictions. Among the remaining images, 216 and 256 were correct only for MS and CAD, respectively. The agreement between correct and incorrect answers for MS and CAD was not very high, at a Cohen’s kappa coefficient of 0.42. However, the AC1 statistic showed a high agreement rate of 0.84.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eClass 2\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe accuracy for MS was 34.3% (32.6 to 36.0), ranging from 24.2% to 46.4% for individuals. The Fleiss’s kappa coefficient and AC1 statistic of 0.40 and 0.44 for NCGM and 0.31 and 0.37 for KUH indicated a low inter-rater agreement. The confusion matrix for the MS is shown in Fig. 2a. The predictions were often incorrect when classifying GPC (U-02–U-06) and GNR (U-08–U-14). In particular, the precision and recall of GNR tended to be low, ranging from 9.2% to 28.4% and 3.3% to 31.1%, respectively (Supplementary Table S5). The accuracy of CAD was 46.1% (40.4 to 51.8). The predictions were often incorrect when classifying GNRs, similar to MS, whereas these were relatively correct for GPCs, especially in the GPC cluster. The precision and recall of GNRs tended to be higher than those of MS, ranging from 14.3% to 40.6% and 5.6% to 72.2%, respectively (Supplementary Table S5). In particular, the F1 scores for glucose non-fermenting bacteria were 38.4% for MS and 60.3% for CAD (Supplementary Table S6).\u003c/p\u003e\n\u003cp\u003eThe accuracy difference was –11.8% (95% CI: –17.6 to –6.04) (p=1.0), indicating non-inferiority of deciphering by CAD compared to MS, based on the Farrington and Manning analysis. However, the 95% CI demonstrated the superiority of CAD.\u003c/p\u003e\n\u003cp\u003eBoth predictions by MS and CAD were correct with 749 images but incorrect with 1,350 predictions. Among the remaining images, 300 and 661 were correct only for MS and CAD, respectively. Compared to Class 1, the agreement on correct answers was lower for MS and CAD, with a Cohen’s kappa coefficient of 0.36 and an AC1 statistic of 0.69.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eAlthough there have been attempts to use image AI analysis based on deep learning methods to assess Gram staining findings with blood cultures [13] and bacterial vaginosis [14], this is the first report to compare and validate Gram staining findings between CAD and MS using clinical urine specimens. In Class 1 differentiation, CAD performed better than MS with GPR and GNC but less with polymicrobial, based on F1 scores. The accuracy of the Gram stain readings by MS was 83%. The Gram staining error rate in discriminating bacterial morphologic features, verified in multiple institutions, was approximately 5% [15], and microbiologists in Japan are also surveyed annually to check the quality of their Gram stain interpretation by\u0026nbsp;Japanese Association of Medical Technologists [Undisclosed data], in which the errors were below 5% most years. Nonetheless, MS performance in the present study was poor compared with that of this standard error rate. However, accuracy in discriminating the five bacterial classifications (GPC, GPR, GNR, GNC, and yeast) by 11 MS in urine specimens was 96.4%, the same as this error rate, in a pilot study [12]. Although the pilot study findings have been validated except those of polymicrobial and ‘None’ classifications, the prediction for polymicrobial by MS in this study was poor as well as that by CAD. In addition, the deciphering of Gram stain findings, normally performed using microscopy, was performed in this study using a single image file, for fairness, when assessing using CAD because of the no significant difference in accuracy between microscopy and single image assessment results in the pilot study [12]. However, this may have affected the prediction of polymicrobial classification using the single image rather than microscopy since the impact of the single-image judgment was verified in the pilot study. The accuracy of prediction for GPR was also poor with MS. Although a single specimen was used in the pilot study, the accuracy of predictions by both imaging and microscopy was low (data not shown). \u003cem\u003eCorynebacterium\u003c/em\u003e spp. in urine specimens were rare compared to those in GNR and GPC. In this study, because samples were not selected to reflect their epidemiology, it is possible that this study’s higher GPR percentage than the proportion in the real world affected the overall accuracy. As reported, \u003cem\u003eCorynebacterium\u003c/em\u003e spp. may form coccoids, depending on the culture condition [16,17], and such changes may occur more likely with urine. Although the reason for this is unclear,\u0026nbsp;the use of a single-image judgment and the high rate of GPR, unlike actual epidemiology, may have contributed to the high error rate of classification of MS.\u003c/p\u003e\n\u003cp\u003eThe impact of accurate Gram staining prediction on antimicrobial use remains unclear, but the results of Gram staining are known to influence empirical antimicrobial selection for common infectious diseases, such as community-acquired pneumonia [18]. A randomized controlled trial on ventilator-associated pneumonia showed that broad-spectrum antimicrobial use was reduced compared to empirical antimicrobial selection by guideline compliance, with no change in clinical efficacy [19]. The possibility that a narrower spectrum of antibiotics can be selected using PCGS has also been shown when antimicrobials were selected using either PCGS or guidelines in patients with infectious diseases, with UTI accounting for approximately half of cases [8]. In pediatric patients with UTI, PCGS showed high diagnostic performance and concordance with bacteria detected in culture, suggesting the possibility of a more efficient selection of susceptible agents [20]. In adult patients with UTI, Taniguchi et al. noted that PCGS may contribute to narrow-spectrum antimicrobial selection because antimicrobial agents selected based on PCGS results and information, such as antibiograms, are susceptible to many detected uropathogens [21]. Although the predictive value of antibiograms for appropriate empirical antimicrobials may not be high [22], antibiograms are one of the few pieces of information available for empirical treatment selection, and their use may contribute to reducing broad-spectrum antimicrobial usage.\u003c/p\u003e\n\u003cp\u003eAs with the eligible specimens in Appendix Table S4, GNR is the main pathogen of urinary tract infection [23], and the F1 value for GNR of CAD is more than 95% and higher than that of MS. Therefore, the prerequisites for estimating the causative organism of urinary tract infection are satisfied. Differentiating bacterial species by Gram staining showed differentiation between \u003cem\u003eS. aureus\u003c/em\u003e and other staphylococci in blood culture [24-26] and whether enterococci were penicillin-resistant [27]. Furthermore, the process of differentiating non-fermenting bacteria using the length and thickness ratio for GNR has been reported [28]. Although these studies showed relatively good performance, achieving sufficient accuracy when differentiating among various species remains challenging, as in this study, rather than within certain species. However, the Class 2 performance by MS was not high either. Although the differential performance of the GPC cluster was relatively good, the precision and recall of the main pathogens, such as GNRs, tended to be lower for MS than for CAD\u0026nbsp;(Supplementary Table S5 and S6). The F1 score of non-fermenting bacteria was \u0026lt;40% in MS and \u0026gt;60% in CAD. Although the results of decipherment by both CAD and MS in this study did\u0026nbsp;not look good, it is true that PCGS decipherment by medical professionals, including MS, in actual practice has reduced the use of broad-spectrum antimicrobial agents compared with guideline-based antimicrobial use [8]. Therefore, future clinical evaluation is needed, though, it is possible that the decipherment of PCGS CAD, which was not inferior to that of MS, can be used effectively in PCGS for choosing empiric antimicrobial therapy.\u003c/p\u003e\n\u003cp\u003eA limitation of this study is that its accuracy has not been verified at other centers, and generalizability remains an issue. Gram staining of urine specimens from facilities where training data were not collected was performed satisfactorily using a similar method and imaging device (data not shown). Although the included samples did not reflect the UTI epidemiological data, it is possible to generalize on bacterial species. This is because we estimated the top detected species in a previous single-center study [23] and national surveillance in Japan [29]. However, it is unclear whether CAD can be generalized to Gram staining methods other than the B\u0026amp;M methods and imaging data captured by smartphone devices other than the iPhone. Further studies may resolve this by increasing variations in the training data.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003ePredicting Gram staining findings in urine specimens by CAD is non-inferior to that by MS following B\u0026amp;M methods for staining samples, and images were captured using an iPhone camera. Although CAD could facilitate infectious disease treatment using PCGS for urine when MS is unavailable to interpret Gram staining findings, future studies are needed to determine its generalizability.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eAC1 - Agreement Coefficient 1\u003c/p\u003e\n\u003cp\u003eAI - Artificial Intelligence\u003c/p\u003e\n\u003cp\u003eATCC - American Type Culture Collection\u003c/p\u003e\n\u003cp\u003eB\u0026amp;M - Bartholomew\u0026nbsp;and\u0026nbsp;Mittwer\u003c/p\u003e\n\u003cp\u003eCAD - Computer-Aided Diagnosis\u003c/p\u003e\n\u003cp\u003eCI - Confidence Interval\u003c/p\u003e\n\u003cp\u003eF1 Score - F1 Score (harmonic mean of recall and precision)\u003c/p\u003e\n\u003cp\u003eGNC - Gram-Negative Cocci\u003c/p\u003e\n\u003cp\u003eGNR - Gram-Negative Rod\u003c/p\u003e\n\u003cp\u003eGPC - Gram-Positive Cocci\u003c/p\u003e\n\u003cp\u003eGPR - Gram-Positive Rod\u003c/p\u003e\n\u003cp\u003eKUH - Kobe University Hospital\u003c/p\u003e\n\u003cp\u003eMS - Microbiology Specialists\u003c/p\u003e\n\u003cp\u003eNCGM - National Center for Global Health and Medicine\u003c/p\u003e\n\u003cp\u003ePCGS - Point-of-Care Gram Staining\u003c/p\u003e\n\u003cp\u003eUTI - Urinary Tract Infection\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe research methods were collectively reviewed and approved by the Institutional Review Board of the NCGM (NCGM-S-004480-02).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBecause residual clinical specimens were used retrospectively, the requirement of individual informed consent was waived by providing an opt-out opportunity.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data associated with this study are presented in the main text or appendix and are available from the authors upon reasonable request. Individual-level, de-identified participant data are not available for publication.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eK.Y. received research grants from Fujirebio Inc., Mizuho Medy Co. Ltd., VisGene Co. Ltd., Canon Medical Systems Corp., Sanyo Chemical Industries Ltd., CarbGeM Inc., Sysmex Corp., and Toyobo Co. Ltd. outside the submitted work and has patent rights to this CAD, but no income has been generated from it. I.M. and S.M. are employees of CarbGeM Inc.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported by AMED (grant number JP23hk0102076h0003).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors' contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualization: KY, GO, KFE, YU, and NO\u003c/p\u003e\n\u003cp\u003eData curation: KY, AM, M. Kurokawa, GO, KFE, KO, M. Kusuki, SM, and IM\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFormal analysis: KY, SM, and YU\u003c/p\u003e\n\u003cp\u003eFunding acquisition: KY, GO, and IM\u003c/p\u003e\n\u003cp\u003eInvestigation: KY, GO, KFE, SM, and IM\u003c/p\u003e\n\u003cp\u003eMethodology: KY, AM, GO, KFE, and IM\u003c/p\u003e\n\u003cp\u003eProject administration: KY, GO, IM, and NO\u003c/p\u003e\n\u003cp\u003eResources: AM, M. Kurokawa, KO, M. Kusuki, SM, and IM\u003c/p\u003e\n\u003cp\u003eSoftware: SM and IM\u003c/p\u003e\n\u003cp\u003eSupervision: NO\u003c/p\u003e\n\u003cp\u003eVisualization: KY and SM\u003c/p\u003e\n\u003cp\u003eWriting – original draft: KY\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWriting – review \u0026amp; editing: GO, IM, KFE, AM, SM, HN, M. Kurokawa, KO, M. Kusuki, YU, and NO\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors express their deepest gratitude to Masami Yoshino for managing the imaging data used in this study. We thank Kayoko Hayakawa, Taketomo Maruki, Yutaro Akiyama, Yuki Moriyama, Ayano Motohashi, Kazuhisa Mezaki, Saeko Kinpara, Taiji Koyama, Kenta Iijima, Saori Kobayashi, Sayaka Matsumoto, Masako Nishida, and Nami Ishida for their help in deciphering the Gram staining. We also thank the staff, Yoshihiko Hirayama, Mayumi Kawakami, Motoko Ishida, Noriko Iwamoto, Gen Yamada, Satoshi Ide, Ayako Okuhama, Akiho Sugita, Nozomu Tsurumaki, Sei Ueda, Wataru Ochi, Reo Iguma, Kaito Mimura, Ryo Sakuma, Mayu Hiraiwa, Shunsuke Matsuoka, Akiko Hamana, Noriko Tomita, and Moto Kimura, at NCGM and KUH for their cooperation. We thank Editage (www.editage.jp) for the English language editing.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eIm Y, Kang D, Ko RE, Lee YJ, Lim SY, Park S, et al. Time-to-antibiotics and clinical outcomes in patients with sepsis and septic shock: a prospective nationwide multicenter cohort study. Crit Care. 2022;26(1):19. doi:10.1186/s13054-021-03883-0.\u003c/li\u003e\n \u003cli\u003eOhnuma T, Chihara S, Costin B, Treggiari MM, Bartz RR, Raghunathan K et al. Association of appropriate empirical antimicrobial therapy with in-hospital mortality in patients with bloodstream infections in the US. JAMA Netw Open. 2023;6(1):e2249353. doi:10.1001/jamanetworkopen.2022.49353.\u003c/li\u003e\n \u003cli\u003eTimbrook TT, Morton JB, McConeghy KW, Caffrey AR, Mylonakis E, LaPlante KL. The effect of molecular rapid diagnostic testing on clinical outcomes in bloodstream infections: A systematic review and meta-analysis. Clin Infect Dis. 2017;64(1):15-23. doi:10.1093/cid/ciw649.\u003c/li\u003e\n \u003cli\u003eBaltekin \u0026Ouml;, Boucharin A, Tano E, Andersson DI, Elf J. Antibiotic susceptibility testing in less than 30 min using direct single-cell imaging. Proc Natl Acad Sci U S A. 2017;114(34):9170-5. doi:10.1073/pnas.1708558114.\u003c/li\u003e\n \u003cli\u003eArienzo A, Murgia L, Cellitti V, Ferrante V, Stalio O, Losito F, et al. A new point-of-care test for the rapid antimicrobial susceptibility assessment of uropathogens. PLOS ONE. 2023;18(7):e0284746. doi:10.1371/journal.pone.0284746.\u003c/li\u003e\n \u003cli\u003eWiwanitkit V, Udomsantisuk N, Boonchalermvichian C. Diagnostic value and cost utility analysis for urine Gram stain and urine microscopic examination as screening tests for urinary tract infection. Urol Res. 2005;33(3):220-2. doi:10.1007/s00240-004-0457-z.\u003c/li\u003e\n \u003cli\u003eWilliams GJ, Macaskill P, Chan SF, Turner RM, Hodson E, Craig JC. Absolute and relative accuracy of rapid urine tests for urinary tract infection in children: a meta-analysis. Lancet Infect Dis. 2010;10(4):240-50. doi:10.1016/S1473-3099(10)70031-1.\u003c/li\u003e\n \u003cli\u003eTaniguchi T, Tsuha S, Shiiki S, Narita M. Gram-stain-based antimicrobial selection reduces cost and overuse compared with Japanese guidelines. BMC Infect Dis. 2015;15:458. doi:10.1186/s12879-015-1203-6.\u003c/li\u003e\n \u003cli\u003eGuarner J, Street C, Matlock M, Cole L, Brierre F. Improving Gram stain proficiency in hospital and satellite laboratories that do not have microbiology. Clin Chem Lab Med. 2017;55(3):458-61. doi:10.1515/cclm-2016-0556.\u003c/li\u003e\n \u003cli\u003eMunson E, Block T, Basile J, Hryciuk JE, Schell RF. Mechanisms to assess Gram stain interpretation proficiency of technologists at satellite laboratories. J Clin Microbiol. 2007;45(11):3754-8. doi:10.1128/JCM.01632-07.\u003c/li\u003e\n \u003cli\u003eShime N, Yanagihara K, Watanabe M, Morita M, Sasaki M, Shinagawa M, et al. A multicenter survey for microbiological testing in emergency department. Journal of Japan Society for Clinical Microbiology. 2019;29:28-31. [Japanese].\u003c/li\u003e\n \u003cli\u003eIkegaki S, Ohji G, Yamamoto K, Ohnuma K, Ebisawa K, Kusuki M, et al. [O-149: A study of differences in prediction accuracy in urine Gram stain decipherment by decipherment method and rater]. The 97th Annual Meeting of the Japanese Association for Infectious Diseases. Yokohama; 2023/4/28. [Japanese, no abstract].\u003c/li\u003e\n \u003cli\u003eSmith KP, Kang AD, Kirby JE. Automated interpretation of blood culture Gram stains by use of a deep convolutional neural network. J Clin Microbiol. 2018;56(3):e01521-17. doi:10.1128/JCM.01521-17.\u003c/li\u003e\n \u003cli\u003eWang Z, Zhang L, Zhao M, Wang Y, Bai H, Wang Y, et al. Deep neural networks offer morphologic classification and diagnosis of bacterial vaginosis. J Clin Microbiol. 2021;59(2):e02236-20. doi:10.1128/JCM.02236-20.\u003c/li\u003e\n \u003cli\u003eSamuel LP, Balada-Llasat JM, Harrington A, Cavagnolo R. Multicenter assessment of Gram stain error rates. J Clin Microbiol. 2016;54(6):1442-7. doi:10.1128/JCM.03066-15.\u003c/li\u003e\n \u003cli\u003eIchikawa L, Ishigaki S, Matsumura M, Asahara M. Case study of morphological changes of Corynebacterium species to gram-positive cocci. Jpn J Technol. 2018;67:299-306. doi:10.14932/jamt.17-143.\u003c/li\u003e\n \u003cli\u003eGrubb TC. Coccus forms of Corynebacterium diphtheriae. J Infect Dis. 1935;56(1):64-77. doi:10.1093/infdis/56.1.64.\u003c/li\u003e\n \u003cli\u003ePendergrast J, Marrie T. Reasons for choice of antibiotic for the empirical treatment of CAP by Canadian infectious disease physicians. Can J Infect Dis. 1999;10(5):337-45. doi:10.1155/1999/928438.\u003c/li\u003e\n \u003cli\u003eYoshimura J, Yamakawa K, Ohta Y, Nakamura K, Hashimoto H, Kawada M, et al. Effect of Gram stain-guided initial antibiotic therapy on clinical response in patients with ventilator-associated pneumonia: the GRACE-VAP randomized clinical trial. JAMA Netw Open. 2022;5(4):e226136. doi:10.1001/jamanetworkopen.2022.6136.\u003c/li\u003e\n \u003cli\u003eYodoshi T, Matsushima M, Taniguchi T, Kinjo S. Utility of point-of-care Gram stain by physicians for urinary tract infection in children \u0026le;36 months. Med (Baltim). 2019;98(14):e15101. doi:10.1097/MD.0000000000015101.\u003c/li\u003e\n \u003cli\u003eTaniguchi T, Tsuha S, Shiiki S, Narita M. Point-of-care urine Gram stain led to narrower-spectrum antimicrobial selection for febrile urinary tract infection in adolescents and adults. BMC Infect Dis. 2022;22(1):198. doi:10.1186/s12879-022-07194-9.\u003c/li\u003e\n \u003cli\u003eHasegawa S, Livorsi DJ, Perencevich EN, Church JN, Goto M. Diagnostic accuracy of hospital antibiograms in predicting the risk of antimicrobial resistance in Enterobacteriaceae isolates: A nationwide multicenter evaluation at the Veterans Health Administration. Clin Infect Dis. 2023;77(11):1492-500. doi:10.1093/cid/ciad467.\u003c/li\u003e\n \u003cli\u003eShigemura K, Tanaka K, Okada H, Nakano Y, Kinoshita S, Gotoh A et al. Pathogen occurrence and antimicrobial susceptibility of urinary tract infection cases during a 20-year period (1983-2002) at a single institution in Japan. Jpn J Infect Dis. 2005;58(5):303-8. doi:10.7883/yoken.JJID.2005.303.\u003c/li\u003e\n \u003cli\u003eMurdoch DR, Greenlees RL. Rapid identification of Staphylococcus aureus from BacT/ALERT blood culture bottles by direct Gram stain characteristics. J Clin Pathol. 2004;57(2):199-201. doi:10.1136/jcp.2003.10538.\u003c/li\u003e\n \u003cli\u003eHadano Y, Isoda M, Ishibashi K, Kakuma T. Validation of blood culture Gram staining for the detection of Staphylococcus aureus by the \u0026rsquo;oozing sign\u0026rsquo; surrounding clustered gram-positive cocci: a prospective observational study. BMC Infect Dis. 2018;18(1):490. doi:10.1186/s12879-018-3412-2.\u003c/li\u003e\n \u003cli\u003eKondo S, Yamada T, Misawa S, Nakamura A, Oguri T. [Morphological identification of Staphylococcus spp. from blood culture bottles based on direct Gram staining]. Kansenshogaku Zasshi. 2008;82(6):656-7. doi:10.11150/kansenshogakuzasshi1970.82.656. [Japanese].\u003c/li\u003e\n \u003cli\u003eHayashi T, Yoshida M. The Peanuts Sign: a Novel Clue to the Rapid Discrimination of Penicillin-susceptible Enterococci Based on Gram Staining Findings. Kansenshogaku Zasshi. 2019;93(3):306-11. doi:10.11150/kansenshogakuzasshi.93.306. [Japanese].\u003c/li\u003e\n \u003cli\u003eBartlett RC, Mazens-Sullivan MF, Lerer TJ. Differentiation of Enterobacteriaceae, \u003cem\u003ePseudomonas aeruginosa\u003c/em\u003e, and \u003cem\u003eBacteroides\u003c/em\u003e and \u003cem\u003eHaemophilus\u003c/em\u003e species in Gram-stained direct smears. Diagn Microbiol Infect Dis. 1991;14(3):195-201. doi:10.1016/0732-8893(91)90032-b.\u003c/li\u003e\n \u003cli\u003eMinistry of Health, Labour and Welfare, Japan. Japan nosocomial infections surveillance (JANIS) [cited Nov 21 2023]. Available from: https://janis.mhlw.go.jp/english/index.asp.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTable 1. Gram staining classification\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"453\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCode\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eClass 1\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eClass 2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eYeast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eCandida\u003c/em\u003e spp.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eGPC cluster\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eEnterococcus faecalis\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eEnterococcus faecium\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eStreptococcus agalactiae\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eOther GPC\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGPR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eCorynebacterium\u003c/em\u003e spp.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eEnterobacter cloacae\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-09\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eEscherichia coli\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eKlebsiella oxytoca\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eKlebsiella pneumoniae\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eOther GNR \u003cem\u003eEnterobacteriaceae\u003c/em\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003ePseudomonas aeruginosa\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eOther GNR glucose non-fermenting bacteria\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eU-15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eGNC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eGNC\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003ePoly\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003ePolymicrobial\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003ePolymicrobial\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"22.026431718061673%\" valign=\"top\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.484581497797357%\" valign=\"top\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"57.48898678414097%\" valign=\"top\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"3\" valign=\"top\"\u003e\n \u003cp\u003eGPC, Gram-positive cocci; GPR, Gram-positive rod; GNC, Gram-negative cocci; GNR, Gram-negative rod.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eTable 2. Number of target specimens extracted, and images taken\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003eActual number of samples from each institute\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of samples included\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of images per sample\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e0\u0026ndash;4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e3\u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e5\u0026ndash;9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e10\u0026ndash;14\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e15\u0026ndash;19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026ge;20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.47457627118644%\" valign=\"top\"\u003e\n \u003cp\u003e20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.05084745762712%\" valign=\"top\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"3\" valign=\"top\"\u003e\n \u003cp\u003e\u003csup\u003ea\u003c/sup\u003eWhen the available specimens were less than three, to incorporate three specimens, the specimens were supplemented with spiked specimens.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3. Precision, recall, and F1 score (harmonic mean) for Class 1\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.197573656845755%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"41.074523396880416%\" colspan=\"3\" valign=\"top\"\u003e\n \u003cp\u003eCAD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"40.72790294627383%\" colspan=\"3\" valign=\"top\"\u003e\n \u003cp\u003eMicrobiology specialist\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\" valign=\"top\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\" valign=\"top\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\" valign=\"top\"\u003e\n \u003cp\u003eF1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\" valign=\"top\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\" valign=\"top\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\" valign=\"top\"\u003e\n \u003cp\u003eF1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e62.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e100.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e76.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e55.8%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e96.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e70.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003ePolymicrobial\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e44.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e44.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e44.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e62.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e50.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e55.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eGPC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e93.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e91.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e92.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e87.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e85.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e86.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eGPR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e75.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e66.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e70.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e75.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e45.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e56.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eGNR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e95.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e96.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e95.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e89.9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e95.8%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e92.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eGNC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e100.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e61.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e75.9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e60.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e35.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e44.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.229166666666668%\" valign=\"top\"\u003e\n \u003cp\u003eYeast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.104166666666666%\"\u003e\n \u003cp\u003e100.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.847222222222221%\"\u003e\n \u003cp\u003e94.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.020833333333334%\"\u003e\n \u003cp\u003e97.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.0625%\"\u003e\n \u003cp\u003e91.7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e85.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.368055555555555%\"\u003e\n \u003cp\u003e88.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"7\" valign=\"top\"\u003e\n \u003cp\u003eCAD, computer-aided diagnosis system; GPC, Gram-positive cocci; GPR, Gram-positive rod; GNC, Gram-negative cocci; GNR, Gram-negative rod; F1, harmonic mean of precision and recall.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Artificial intelligence, Computer-aided diagnosis, Gram staining, Non-inferiority trial, Urinary tract infection ","lastPublishedDoi":"10.21203/rs.3.rs-4284383/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4284383/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eGram staining results interpreted by microbiology specialists (MS) were compared to a developed computer-aided diagnosis system (CAD) using artificial intelligence for interpretation.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eUsing a non-inferiority study, CAD-predicted Gram stain results, generated from images of an iPhone camera in two hospitals, were compared to those of MS, between 1 April and 31 December 2022. The prediction accuracies, classified as Class 1 based on the bacterial morphology, were the primary endpoint. The datasets were created according to the separate hospitals; 10 MS interpreted 153 images from each hospital. CAD predicted 306 images overall.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe accuracies (95% confidence intervals) of MS and CAD predictions were 83.0% (81.6–84.3) and 87.9% (83.7–91.3), respectively; with a difference of –4.93% (–8.43 to –0.62), indicating non-inferiority of CAD.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNon-inferiority of CAD to MS predictions was demonstrated; therefore, CAD urine Gram staining can be used to select empirical antibiotics in facilities without MS.\u003c/p\u003e","manuscriptTitle":"A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-26 01:51:38","doi":"10.21203/rs.3.rs-4284383/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e1ee85b4-2121-4f29-b418-e602f6bd9184","owner":[],"postedDate":"April 26th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-04-26T01:51:38+00:00","versionOfRecord":[],"versionCreatedAt":"2024-04-26 01:51:38","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4284383","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4284383","identity":"rs-4284383","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}