Generating 3D Optical Coherence Tomography from 2D Fundus Images via Diffusion Models

Generating 3D Optical Coherence Tomography from 2D Fundus Images via Diffusion Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Generating 3D Optical Coherence Tomography from 2D Fundus Images via Diffusion Models Bowen Liu, Yue Wu, Ruoyu Chen, Pusheng Xu, Peng Xiao, Zhen Tian, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6160112/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Training machine learning models with synthetic data effectively addresses data scarcity, particularly in domains where acquiring large-scale 3D datasets is costly. We present Fundus2OCT, the first framework to synthesize high-fidelity 3D optical coherence tomography (OCT) volumes from 2D fundus photographs using diffusion models. Developed on paired fundus-OCT data from the UK Biobank, Fundus2OCT leverages a two-stage latent diffusion process to generate anatomically coherent OCT volumes (32 B-scans per volume) conditioned on fundus inputs. Quantitative evaluations demonstrate superior performance over existing methods, with Fréchet Inception Distance (FID) and Fréchet Video Distance (FVD) scores of 12.3 and 58.7, respectively. In a clinical Turing test, two ophthalmologists achieved accuracies of 49.0–57.0% (near chance-level) in distinguishing synthetic from real OCTs. To validate clinical utility, we augmented four public fundus-based disease detection tasks (AMD, glaucoma, DR, DME) with synthetic OCT data, improving multimodal classification AUC by 4.2–8.6%. By bridging 2D fundus findings with 3D structural insights, Fundus2OCT advances multimodal retinal analysis, offering a scalable solution to enhance diagnostic accuracy and accessibility in ophthalmic care. Health sciences/Health care Health sciences/Diseases/Eye diseases Full Text Additional Declarations There is NO Competing Interest. Supplementary Files supplementary.pdf supplementary materiel Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6160112","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":424661154,"identity":"c9cac558-1192-468a-b30a-ca160283056d","order_by":0,"name":"Bowen Liu","email":"","orcid":"","institution":"School of Optometry, The Hong Kong Polytechnic University","correspondingAuthor":false,"prefix":"","firstName":"Bowen","middleName":"","lastName":"Liu","suffix":""},{"id":424661155,"identity":"15b28012-77a1-4356-b1a3-a730717ca9da","order_by":1,"name":"Yue Wu","email":"","orcid":"","institution":"The Hong Kong Polytechnic University","correspondingAuthor":false,"prefix":"","firstName":"Yue","middleName":"","lastName":"Wu","suffix":""},{"id":424661156,"identity":"3e20a08e-32d1-4907-93bc-766faafddeaa","order_by":2,"name":"Ruoyu Chen","email":"","orcid":"","institution":"The Hong Kong Polytechnic University","correspondingAuthor":false,"prefix":"","firstName":"Ruoyu","middleName":"","lastName":"Chen","suffix":""},{"id":424661157,"identity":"a4276389-4e2e-4fe9-98d9-76473e911838","order_by":3,"name":"Pusheng Xu","email":"","orcid":"","institution":"The Hong Kong Polytechnic University","correspondingAuthor":false,"prefix":"","firstName":"Pusheng","middleName":"","lastName":"Xu","suffix":""},{"id":424661158,"identity":"23e63b94-a14b-473f-a890-2dbedd423793","order_by":4,"name":"Peng Xiao","email":"","orcid":"","institution":"State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University","correspondingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"Xiao","suffix":""},{"id":424661159,"identity":"d33597b0-76a1-4bdc-ac25-f4790165cf11","order_by":5,"name":"Zhen Tian","email":"","orcid":"","institution":"The Hong Kong Polytechnic University","correspondingAuthor":false,"prefix":"","firstName":"Zhen","middleName":"","lastName":"Tian","suffix":""},{"id":424661162,"identity":"7af831d8-62d3-417d-a45f-2fc7e34dde07","order_by":6,"name":"Binwei Huang","email":"","orcid":"","institution":"Shantou University Medical College","correspondingAuthor":false,"prefix":"","firstName":"Binwei","middleName":"","lastName":"Huang","suffix":""},{"id":424661160,"identity":"1aefbcc3-4ab1-446d-bb84-014b8ff08a67","order_by":7,"name":"Mingguang He","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDklEQVRIiWNgGAWjYDACZjiLh4HhAwODDIydIEGMFsYZYMWEtCAADwMzDzFadNt5D7/82maXJ+/Ae/Cx7Q4bHoMbCYwP3rYx5Ek2YNdidpgvzVq2LbnY8ABfsnHumTSQFmbDuW0MxdI4bDE7zGNmLNnGnLixgcdMOrftMEgLmzRvG0PiPPxa6kFazH9bQrSw/yagxfjhx7bDifMZeMyYGaG2MIO0zMZjCzPDueOJG5h5jCV7gX6RPPOwWXLOOYnEmbi8f/6M8ccfZdWJ89t7DD/83GEjx3c8+eCHN2U2iTMO4LCGgYFNGhQXBoeBBCPQYIUDIJIBb0Qyf/wBJOUboFrkcbhnFIyCUTAKRi4AADMMWg/8YNJmAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-6912-2810","institution":"The Hong Kong Polytechnic University","correspondingAuthor":true,"prefix":"","firstName":"Mingguang","middleName":"","lastName":"He","suffix":""},{"id":424661153,"identity":"1782690f-d6aa-402a-9489-ee314e97f501","order_by":8,"name":"Danli Shi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIie2RsQrCMBCGTwp1CXVNEHyGuLg4+CoJDi6lCII4Fae4+ABuPoNvECnoUnXtWBGcFBwdHMxVB5fGjoL5hhvC//HfEQCH4xfRUNMCQPj1KUBePAnwzLNNgZdCNIarKkWMiopKsN1pnUMcBey6zqVKoFEPuTdMyxWWRsIsloz8ZtTnqLD5hXuLrFzhOuRG0VI1ww5FhWemhdwsyuGCSiwVS19K76uSFS2eVJS8WygqlsVYhi08kYqYW8R+QGh6HibEcn5wCNvH+ySWy9lund/G3VZj1l+dyKZcea+HgwDUfJxg/chPMPyomHU4HI6/4gmITFi/oXfLkAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0001-6094-137X","institution":"The Hong Kong Polytechnic University","correspondingAuthor":true,"prefix":"","firstName":"Danli","middleName":"","lastName":"Shi","suffix":""}],"badges":[],"createdAt":"2025-03-05 07:50:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6160112/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6160112/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":81289991,"identity":"145d9ccb-7015-4ac6-b010-f66c9638ba62","added_by":"auto","created_at":"2025-04-24 11:44:01","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":793269,"visible":true,"origin":"","legend":"Article File","description":"","filename":"manuscriptblind.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6160112/v1_covered_46ad3403-a951-4787-8754-54bcb04fb99c.pdf"},{"id":77837627,"identity":"501a3f33-6b16-4b6c-b11f-1cd89f35c520","added_by":"auto","created_at":"2025-03-06 03:57:40","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":29983260,"visible":true,"origin":"","legend":"supplementary materiel","description":"","filename":"supplementary.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6160112/v1/fedc5746f2510bb4420419e2.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Generating 3D Optical Coherence Tomography from 2D Fundus Images via Diffusion Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6160112/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6160112/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Training machine learning models with synthetic data effectively addresses data scarcity, particularly in domains where acquiring large-scale 3D datasets is costly. We present Fundus2OCT, the first framework to synthesize high-fidelity 3D optical coherence tomography (OCT) volumes from 2D fundus photographs using diffusion models. Developed on paired fundus-OCT data from the UK Biobank, Fundus2OCT leverages a two-stage latent diffusion process to generate anatomically coherent OCT volumes (32 B-scans per volume) conditioned on fundus inputs. Quantitative evaluations demonstrate superior performance over existing methods, with Fréchet Inception Distance (FID) and Fréchet Video Distance (FVD) scores of 12.3 and 58.7, respectively. In a clinical Turing test, two ophthalmologists achieved accuracies of 49.0–57.0% (near chance-level) in distinguishing synthetic from real OCTs. To validate clinical utility, we augmented four public fundus-based disease detection tasks (AMD, glaucoma, DR, DME) with synthetic OCT data, improving multimodal classification AUC by 4.2–8.6%. By bridging 2D fundus findings with 3D structural insights, Fundus2OCT advances multimodal retinal analysis, offering a scalable solution to enhance diagnostic accuracy and accessibility in ophthalmic care.","manuscriptTitle":"Generating 3D Optical Coherence Tomography from 2D Fundus Images via Diffusion Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-06 03:57:35","doi":"10.21203/rs.3.rs-6160112/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9481de51-f44b-42cd-9db3-5faf65c7ed36","owner":[],"postedDate":"March 6th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":45256500,"name":"Health sciences/Health care"},{"id":45256501,"name":"Health sciences/Diseases/Eye diseases"}],"tags":[],"updatedAt":"2025-04-30T05:10:21+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-06 03:57:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6160112","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6160112","identity":"rs-6160112","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}