Standalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review

Standalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Systematic Review Standalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review Amir Srour, Mahmud Omar, Yiftach Barash, Diana Litmanovich, Mayse Srour, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9705085/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Pulmonary nodules may be missed on chest radiographs despite their clinical relevance for early lung cancer detection. Commercial artificial intelligence (AI) systems are increasingly available for chest radiograph interpretation, but evidence for their standalone performance remains fragmented across products, datasets, operating thresholds, and comparator designs. Purpose To systematically evaluate the standalone diagnostic performance of commercially available AI software for pulmonary nodule detection on chest radiographs compared with radiologists reading alone, and to identify methodological factors shaping interpretation of the evidence. Methods A systematic review was conducted according to PRISMA 2020 guidelines and registered with PROSPERO (CRD420261383905). PubMed, PubMed Central, Scopus, and Web of Science were searched. Eligible studies evaluated commercially available AI systems as standalone detectors for pulmonary nodules on chest radiographs and directly compared their performance with radiologists reading alone using an accepted clinical reference standard. Risk of bias was assessed using an adapted QUADAS-2 framework. Findings were synthesized narratively because of heterogeneity in study design, populations, reference standards, operating thresholds, and outcome measures. Results Six studies met the inclusion criteria, comprising 12 product-level evaluations across 10 commercial AI systems. Standalone AI performance varied: some evaluations showed AI exceeding radiologist-level performance, others showed no significant difference, and one showed AI below individual reader performance with approximately twice the false-positive rate. In the only standardized within-study comparison of multiple commercial products on the same dataset, four of seven systems significantly exceeded mean radiologist AUC. Standalone AI AUC ranged from 0.79 to 0.93 against a mean reader AUC of 0.81, demonstrating that performance varied even among cleared products tested under identical conditions. Interpretation was limited by differences in operating thresholds, comparator design, reference standards, study populations, and outcome reporting. Risk-of-bias concerns were common, particularly in patient selection. Conclusions In the available peer-reviewed evidence, standalone commercial AI systems for chest radiograph pulmonary nodule detection show variable performance. Evidence remains insufficient to support class-wide standalone deployment because all studies were retrospective or reader-study based, thresholds were inconsistently reported, and real-world false-positive burden remains uncertain. Prospective real world evaluation with clearly reported operating thresholds, standardized comparator designs and complete diagnostic performance are needed. Pulmonary nodule Chest radiograph Artificial intelligence Diagnostic accuracy Full Text Additional Declarations The authors declare no competing interests. Supplementary Files SupplementaryAppendixStandaloneFINAL.docx11.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9705085","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Systematic Review","associatedPublications":[],"authors":[{"id":639797554,"identity":"b2ab1c6f-0541-48c6-aedc-ff0ec06073d4","order_by":0,"name":"Amir Srour","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA10lEQVRIiWNgGAWjYDADfgkwJSFDvBbJGQyMDUAtPMRrMbgB1sJAWAv/tMMHH1cw1OUb324+/uhGjQUPA/vhoxvwaZG4nZZseIbhsOW2O8cSm3OOAR3Gk5Z2A681t3PMJBsYDhiY3cgxbM5hA2qR4DHDq0X+dv73nw0MdQbGM0Ba/hGhxeB2DhvQ28wGBhJALbltRGgxvJ1mLNlgcNhA4kZa4uzcPgkeNkJ+kbud/PBjQ0WdAf+M5AOfc77VyfGzHz6G3/sQ5yGx2QgrHwWjYBSMglFACAAAeaZDlHQd/1wAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0008-2302-8774","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":true,"prefix":"","firstName":"Amir","middleName":"","lastName":"Srour","suffix":""},{"id":639798369,"identity":"64d5faba-6c19-438e-9b9f-cbdee0dde867","order_by":1,"name":"Mahmud Omar","email":"","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Mahmud","middleName":"","lastName":"Omar","suffix":""},{"id":639798370,"identity":"ef8ad841-64bf-4abf-8f07-4183de9b7252","order_by":2,"name":"Yiftach Barash","email":"","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Yiftach","middleName":"","lastName":"Barash","suffix":""},{"id":639798371,"identity":"86752697-66c4-4cd3-9fff-18391fe58672","order_by":3,"name":"Diana Litmanovich","email":"","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Diana","middleName":"","lastName":"Litmanovich","suffix":""},{"id":639798372,"identity":"5441c6a3-3f0c-4bce-ba6c-2d667492f8ee","order_by":4,"name":"Mayse Srour","email":"","orcid":"","institution":"University of Debrecen","correspondingAuthor":false,"prefix":"","firstName":"Mayse","middleName":"","lastName":"Srour","suffix":""},{"id":639798373,"identity":"63ab3f09-19a3-440b-81aa-e2f055d35aec","order_by":5,"name":"Alon Gorenshtein","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABC0lEQVRIiWNgGAWjYBCDBH4JxgYJEAPCZwPiAwS0SM4gWYvBDQYG4rTw9y8+9uHjjro849vNjbdu1DDk8c8+Y/i5ooxBju9GAlYtEjeeJc+ceeZwsdmdg83WOccYiiXO5RhLnjnHYCyJQwvDjTPGzLxtBxK33Uhsk85tYEhsOMOWINnYxpC4AYcWeZCWv211iZtnQLXMP8OW/BOopR6XFoPzPcbMjG3MiRskoFo2nGE+BrIFGCDYtRjeYEtm7G07nDgD4heJxI1ALZYN5yQMZ555gFWL3PnDhxl+Ah3WP7v94e2cGpvEeWcYm282lNnI8x3H4X0JVHEJDAYm4D+AW24UjIJRMApGARgAACqEahXT3MrWAAAAAElFTkSuQmCC","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":true,"prefix":"","firstName":"Alon","middleName":"","lastName":"Gorenshtein","suffix":""},{"id":639798374,"identity":"5d1da00f-faeb-4e08-9bcc-0dbf4e9e9513","order_by":6,"name":"Eyal Klang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2klEQVRIiWNgGAWjYHACxoMNBQxyBmC2gQVxeg42GDAYGzAwg7RIEK8lcQNYCwMRWvglkh8cnGFgl76dvf/ohh8FEgz87d0JeLVIzkgzOLjBIDl3Z89htps9QIdJnDm7Aa8WgxsJBgcfGDDnbriRzHaDB6jFQCIXvxb7G+kfgFrq0w2AWm7+IUaLgUQOyGGHE0BabhNli8SZNwVA7x833HDmsNltGQMJHoJ+4W9P3/iwp6Ja3uB447Obb/7YyPG39+LXwiCQgMrnwa8cbM0BwmpGwSgYBaNghAMA+qVMGpgXPMUAAAAASUVORK5CYII=","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":true,"prefix":"","firstName":"Eyal","middleName":"","lastName":"Klang","suffix":""}],"badges":[],"createdAt":"2026-05-13 14:18:36","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9705085/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9705085/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":109249601,"identity":"b48e8e87-ddf8-4d46-8b0e-7a03da15ce03","added_by":"auto","created_at":"2026-05-14 08:57:18","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":694972,"visible":true,"origin":"","legend":"","description":"","filename":"StandalonecommercialAIsoftwareforpulmonarynoduleFINAL2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9705085/v1_covered_4357530c-cf41-46e4-93b2-b305d9ae3044.pdf"},{"id":109238807,"identity":"e1e55fa4-7b24-4f2c-be1a-11bbc562bd76","added_by":"auto","created_at":"2026-05-14 05:59:47","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":308924,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryAppendixStandaloneFINAL.docx11.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9705085/v1/f8c55a9f0bc594d1710e547f.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eStandalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Beth Israel Deaconess Medical Center","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Pulmonary nodule, Chest radiograph, Artificial intelligence, Diagnostic accuracy","lastPublishedDoi":"10.21203/rs.3.rs-9705085/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9705085/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003cbr\u003e\n\u003c/strong\u003ePulmonary nodules may be missed on chest radiographs despite their clinical relevance for early lung cancer detection. Commercial artificial intelligence (AI) systems are increasingly available for chest radiograph interpretation, but evidence for their standalone performance remains fragmented across products, datasets, operating thresholds, and comparator designs.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePurpose\u003cbr\u003e\n\u003c/strong\u003eTo systematically evaluate the standalone diagnostic performance of commercially available AI software for pulmonary nodule detection on chest radiographs compared with radiologists reading alone, and to identify methodological factors shaping interpretation of the evidence.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003cbr\u003e\n\u003c/strong\u003eA systematic review was conducted according to PRISMA 2020 guidelines and registered with PROSPERO (CRD420261383905). PubMed, PubMed Central, Scopus, and Web of Science were searched. Eligible studies evaluated commercially available AI systems as standalone detectors for pulmonary nodules on chest radiographs and directly compared their performance with radiologists reading alone using an accepted clinical reference standard. Risk of bias was assessed using an adapted QUADAS-2 framework. Findings were synthesized narratively because of heterogeneity in study design, populations, reference standards, operating thresholds, and outcome measures.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003cbr\u003e\n\u003c/strong\u003eSix studies met the inclusion criteria, comprising 12 product-level evaluations across 10 commercial AI systems. Standalone AI performance varied: some evaluations showed AI exceeding radiologist-level performance, others showed no significant difference, and one showed AI below individual reader performance with approximately twice the false-positive rate.\u003c/p\u003e\n\u003cp\u003eIn the only standardized within-study comparison of multiple commercial products on the same dataset, four of seven systems significantly exceeded mean radiologist AUC. Standalone AI AUC ranged from 0.79 to 0.93 against a mean reader AUC of 0.81, demonstrating that performance varied even among cleared products tested under identical conditions. Interpretation was limited by differences in operating thresholds, comparator design, reference standards, study populations, and outcome reporting. Risk-of-bias concerns were common, particularly in patient selection.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003cbr\u003e\n\u003c/strong\u003eIn the available peer-reviewed evidence, standalone commercial AI systems for chest radiograph pulmonary nodule detection show variable performance. Evidence remains insufficient to support class-wide standalone deployment because all studies were retrospective or reader-study based, thresholds were inconsistently reported, and real-world false-positive burden remains uncertain. Prospective real world evaluation with clearly reported operating thresholds, standardized comparator designs and complete diagnostic performance are needed.\u003c/p\u003e","manuscriptTitle":"Standalone commercial artificial intelligence software for pulmonary nodule detection on chest radiographs: a systematic review","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-14 05:59:44","doi":"10.21203/rs.3.rs-9705085/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1c915ed8-660d-4d09-8609-601a7551bb23","owner":[],"postedDate":"May 14th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-14T05:59:44+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-14 05:59:44","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9705085","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9705085","identity":"rs-9705085","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}