Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization

Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization Jinli Suo, Kaiming Dong, Yuxiao Cheng, Kunlun He, Qionghai Dai This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4111078/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Jul, 2025 Read the published version in Nature Biomedical Engineering → Version 1 posted You are reading this latest preprint version Abstract Medical artificial intelligence (AI) offers great potential for automatic pathology interpretation, but the performance is far behind providing a practical tool in clinical settings, which demands both pixel-level accuracy and high interpretability for diagnosis. The main challenges lie in that the construction of such AI models relies on substantial training data with fine-grained labeling that is impractical in real applications. To circumvent this barrier, we propose a prompt-driven constrained generative model to produce anatomically aligned healthy/diseased image pairs and then learn a superb pathology localization model in a supervised manner. The new paradigm effectively addresses the lack of high-fidelity chest X-ray images with pathology labeling at fine scales. Benefiting from the emerging text-driven generative foundation model and the newly incorporated constraint, our model presents promising localization accuracy of the subtle pathologies, high interpretability for clinical decisions, and good transfer ability to many unseen pathological categories (e.g., new prompts and mixed pathologies). These advantageous features establish our model as a promising solution to assist chest X-ray analysis. Besides, the proposed approach is also inspiring for other tasks lacking massive training data and time-consuming manual labeling. Health sciences/Health care/Medical imaging/Radiography Biological sciences/Computational biology and bioinformatics/Image processing Biological sciences/Computational biology and bioinformatics/Machine learning Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Published Journal Publication published 14 Jul, 2025 Read the published version in Nature Biomedical Engineering → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4111078","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":283818456,"identity":"f838dec3-56de-4f5b-9493-701c16d516ee","order_by":0,"name":"Jinli Suo","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYBACxnYQWcHAwAZGIHCAkJZmEHnGgAQtDMwgfW0GICaRWpib2Z9JfJz3x55Puv3aY54aBjm+GwmMnwvwOozHTHLmNoPENpkz5cY8xxiMJW8kMEvPwK+FTZp3m0ECm0ROmjQPG0PihhsJbMw8eLWwP5PmnWNgD9Hyj6GeCC0MZtK8DQaMbRLpx6R52xgSDAhr4TG2nHHMOLFNIodNcm6fhOHMMw+bpfFpMWxvf3jjQ42cvfyM9GcSb77ZyPMdTz74Ga+WBjiTBxQ5EiCbG3AohgB5BJP9AV6Vo2AUjIJRMHIBAHyMQUDLygrJAAAAAElFTkSuQmCC","orcid":"","institution":"Tsinghua University","correspondingAuthor":true,"prefix":"","firstName":"Jinli","middleName":"","lastName":"Suo","suffix":""},{"id":283818457,"identity":"53c77f8a-d968-4e56-8656-d02f024651ac","order_by":1,"name":"Kaiming Dong","email":"","orcid":"https://orcid.org/0000-0002-9748-1482","institution":"Tsinghua University","correspondingAuthor":false,"prefix":"","firstName":"Kaiming","middleName":"","lastName":"Dong","suffix":""},{"id":283818458,"identity":"eeb15273-ce7c-4c8d-9bc2-2908e7aad248","order_by":2,"name":"Yuxiao Cheng","email":"","orcid":"","institution":"Tsinghua University","correspondingAuthor":false,"prefix":"","firstName":"Yuxiao","middleName":"","lastName":"Cheng","suffix":""},{"id":283818459,"identity":"7ac8cc55-e90d-4033-a649-c56b29575546","order_by":3,"name":"Kunlun He","email":"","orcid":"https://orcid.org/0000-0002-3335-5700","institution":"Chinese PLA General Hospital","correspondingAuthor":false,"prefix":"","firstName":"Kunlun","middleName":"","lastName":"He","suffix":""},{"id":283818460,"identity":"f68d530a-967d-4484-a1ba-15ff210e28ed","order_by":4,"name":"Qionghai Dai","email":"","orcid":"","institution":"Tsinghua University","correspondingAuthor":false,"prefix":"","firstName":"Qionghai","middleName":"","lastName":"Dai","suffix":""}],"badges":[],"createdAt":"2024-03-16 03:35:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4111078/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4111078/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41551-025-01456-y","type":"published","date":"2025-07-14T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":86742454,"identity":"c49c799d-f34a-491a-86f3-24c45125778f","added_by":"auto","created_at":"2025-07-15 07:09:50","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4113029,"visible":true,"origin":"","legend":"","description":"","filename":"PromptdrivenHealthyDiseasedImagePairsEnablingPixellevelChestXrayPathologyLocalization.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4111078/v1_covered_7280393f-c057-46cc-8ce5-14d209f3bcd3.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4111078/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4111078/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Medical artificial intelligence (AI) offers great potential for automatic pathology interpretation, but the performance is far behind providing a practical tool in clinical settings, which demands both pixel-level accuracy and high interpretability for diagnosis. The main challenges lie in that the construction of such AI models relies on substantial training data with fine-grained labeling that is impractical in real applications. To circumvent this barrier, we propose a prompt-driven constrained generative model to produce anatomically aligned healthy/diseased image pairs and then learn a superb pathology localization model in a supervised manner. \r\nThe new paradigm effectively addresses the lack of high-fidelity chest X-ray images with pathology labeling at fine scales. Benefiting from the emerging text-driven generative foundation model and the newly incorporated constraint, our model presents promising localization accuracy of the subtle pathologies, high interpretability for clinical decisions, and good transfer ability to many unseen pathological categories (e.g., new prompts and mixed pathologies). These advantageous features establish our model as a promising solution to assist chest X-ray analysis. Besides, the proposed approach is also inspiring for other tasks lacking massive training data and time-consuming manual labeling.","manuscriptTitle":"Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-27 10:49:22","doi":"10.21203/rs.3.rs-4111078/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-biomedical-engineering","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"natbiomedeng","sideBox":"Learn more about [Nature Biomedical Engineering](http://www.nature.com/natbiomedeng/)","snPcode":"41551","submissionUrl":"https://mts-natbiomedeng.nature.com/cgi-bin/main.plex","title":"Nature Biomedical Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Research","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"559a592d-acfc-405c-8ceb-a334dce004f3","owner":[],"postedDate":"March 27th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":29876135,"name":"Health sciences/Health care/Medical imaging/Radiography"},{"id":29876136,"name":"Biological sciences/Computational biology and bioinformatics/Image processing"},{"id":29876137,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"}],"tags":[],"updatedAt":"2025-07-15T07:09:40+00:00","versionOfRecord":{"articleIdentity":"rs-4111078","link":"https://doi.org/10.1038/s41551-025-01456-y","journal":{"identity":"nature-biomedical-engineering","isVorOnly":false,"title":"Nature Biomedical Engineering"},"publishedOn":"2025-07-14 04:00:00","publishedOnDateReadable":"July 14th, 2025"},"versionCreatedAt":"2024-03-27 10:49:22","video":"","vorDoi":"10.1038/s41551-025-01456-y","vorDoiUrl":"https://doi.org/10.1038/s41551-025-01456-y","workflowStages":[]},"version":"v1","identity":"rs-4111078","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4111078","identity":"rs-4111078","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}