Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy

Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy Qionghai Dai, Chang Qiao, Yunmin Zeng, Quan Meng, Xingye Chen, and 11 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3252567/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Computational super-resolution (SR) methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding SR performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acquire due to the high dynamics of living cells. Here, we develop zero-shot deconvolution networks (ZS-DeconvNet) that instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary SR imaging conditions in an unsupervised manner without the need for either ground truths or additional data acquisition. We demonstrate the versatile applicability of ZS-DeconvNet on multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional (3D) wide-field microscopy, confocal microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy (SIM), which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans. Physical sciences/Optics and photonics/Optical techniques/Microscopy/Super-resolution microscopy Physical sciences/Optics and photonics/Optical techniques/Imaging and sensing Physical sciences/Optics and photonics/Optical techniques/Microscopy Full Text Additional Declarations There is a conflict of interest Dong Li, C.Q. and Y.Z. have a pending patent application on the presented frameworks. Supplementary Files SuppVideo1CmpwithTiS.mp4 Supplementary Video 1 SuppVideo2LifeactMyosin2case1.mp4 Supplementary Video 2 SuppVideo3LifeactMyosin2case2.mp4 Supplementary Video 3 SuppVideo4Rab11Lamp1.mp4 Supplementary Video 4 SuppVideo5MitosisMitoHeLa.mp4 Supplementary Video 5 SuppVideo6MitosisSC35.mp4 Supplementary Video 6 SuppVideo7ConfocalMouseEmbryo.mp4 Supplementary Video 7 SuppVideo8WFCelegansEmbryo.mp4 Supplementary Video 8 SuppVideo9DemoPlugin.mp4 Supplementary Video 9 SupplementaryMaterials.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3252567","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":233861530,"identity":"4170f03f-5d7f-4709-9729-62db7d3d8b69","order_by":0,"name":"Qionghai Dai","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6klEQVRIiWNgGAWjYDACCQYDhoQKBgY2MAKBA0RpOWNAqhbGNgMQk0gtfLebN354OO+PPZ90+7XHPDUMcnw3Ehg/F+DRInnnWLFE4jaDxDaZM+XGPMcYjCVvJDBLz8CjxeBGjgFISwKbRE6aNA8bQ+KGGwlszDz4tRj/SJxjYA/R8o+hnhgtZhKJDQaMbRLpx6R52xgSDAhpkbyRVmaRcMw4sU0ih01ybp+E4cwzD5ul8Wnhu5G8+eaPGjl7+RnpzyTefLOR5zuefPAzPi1IscADihwJIGZswKcBWQv7A/wqR8EoGAWjYMQCAFNISdcRyCx6AAAAAElFTkSuQmCC","orcid":"","institution":"Tsinghua University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Qionghai","middleName":"","lastName":"Dai","suffix":""},{"id":233861531,"identity":"8742132c-c205-4d0d-9315-e6a22caca1fe","order_by":1,"name":"Chang Qiao","email":"","orcid":"https://orcid.org/0000-0002-6037-0842","institution":"Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Chang","middleName":"","lastName":"Qiao","suffix":""},{"id":233861532,"identity":"fcae23c4-958e-4f46-9338-61b8b351ea9f","order_by":2,"name":"Yunmin Zeng","email":"","orcid":"","institution":"Department of Automation, Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yunmin","middleName":"","lastName":"Zeng","suffix":""},{"id":233861533,"identity":"5b83d8a8-e5d1-4694-a449-dc5bfe79db79","order_by":3,"name":"Quan Meng","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Quan","middleName":"","lastName":"Meng","suffix":""},{"id":233861534,"identity":"68247ddf-0f7a-458b-beca-c184b402f632","order_by":4,"name":"Xingye Chen","email":"","orcid":"","institution":"Department of Automation, Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Xingye","middleName":"","lastName":"Chen","suffix":""},{"id":233861535,"identity":"8a599277-87f5-4b95-ad3d-497148b3c085","order_by":5,"name":"Haoyu Chen","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Haoyu","middleName":"","lastName":"Chen","suffix":""},{"id":233861536,"identity":"c773ebe0-5cee-4c86-a8d4-b93077319c6e","order_by":6,"name":"Tao Jiang","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Tao","middleName":"","lastName":"Jiang","suffix":""},{"id":233861537,"identity":"53894df0-ae9b-48e4-aea4-3d0e113909c9","order_by":7,"name":"Rongfei Wei","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Rongfei","middleName":"","lastName":"Wei","suffix":""},{"id":233861538,"identity":"f6928c7a-1055-4426-8b72-e44abf18664e","order_by":8,"name":"Jiabao Guo","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jiabao","middleName":"","lastName":"Guo","suffix":""},{"id":233861539,"identity":"3ae95fa6-ff13-4234-adf4-9698136cffb1","order_by":9,"name":"Wenfeng Fu","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Wenfeng","middleName":"","lastName":"Fu","suffix":""},{"id":233861540,"identity":"88ea42ff-337a-48ec-bd36-30922298f4b6","order_by":10,"name":"Huaide Lu","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Huaide","middleName":"","lastName":"Lu","suffix":""},{"id":233861541,"identity":"8795d05a-9f87-42dd-b615-0d1e7abb56a0","order_by":11,"name":"Di Li","email":"","orcid":"","institution":"Institute of Biophysics, Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Di","middleName":"","lastName":"Li","suffix":""},{"id":233861542,"identity":"01650ad3-4c45-43b3-a16a-cbd2f5960a6d","order_by":12,"name":"Yuwang Wang","email":"","orcid":"","institution":"Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yuwang","middleName":"","lastName":"Wang","suffix":""},{"id":233861543,"identity":"d36a1f72-546f-451c-93e8-e1464b52b3fd","order_by":13,"name":"Hui Qiao","email":"","orcid":"","institution":"BNRist and Department of Automation, Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Qiao","suffix":""},{"id":233861544,"identity":"1b554222-8aa8-450f-8835-25da556a96f6","order_by":14,"name":"Jiamin Wu","email":"","orcid":"https://orcid.org/0000-0003-3479-1026","institution":"Tsinghua University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jiamin","middleName":"","lastName":"Wu","suffix":""},{"id":233861545,"identity":"6bc16ad4-a5b6-451c-94ab-1e31002fa88a","order_by":15,"name":"Dong Li","email":"","orcid":"","institution":"Chinese Academy of Sciences","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Dong","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2023-08-10 14:02:50","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3252567/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3252567/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":44228600,"identity":"f56fad04-f76b-4dc1-b4a6-5ae58dff6b6c","added_by":"auto","created_at":"2023-10-07 07:54:11","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4099367,"visible":true,"origin":"","legend":"","description":"","filename":"MainText.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1_covered_80ef69eb-ae64-41bf-a5ca-b4de4cb93880.pdf"},{"id":43460491,"identity":"11ebc1a7-146b-4959-a6f4-58bee96470a9","added_by":"auto","created_at":"2023-09-21 05:57:35","extension":"mp4","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":12499498,"visible":true,"origin":"","legend":"Supplementary Video 1","description":"","filename":"SuppVideo1CmpwithTiS.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/e92f0017bf49d6410ef13f96.mp4"},{"id":43460497,"identity":"7ed8e7d2-06f4-409d-b4b1-9a3606e6a4e9","added_by":"auto","created_at":"2023-09-21 05:57:36","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":18310055,"visible":true,"origin":"","legend":"Supplementary Video 2","description":"","filename":"SuppVideo2LifeactMyosin2case1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/b55010406cef1bb6906f2a65.mp4"},{"id":43460725,"identity":"953faab9-50bf-451c-a265-e494ec389c46","added_by":"auto","created_at":"2023-09-21 06:05:35","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":14910434,"visible":true,"origin":"","legend":"Supplementary Video 3","description":"","filename":"SuppVideo3LifeactMyosin2case2.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/a9745b0682c5499152114618.mp4"},{"id":43460495,"identity":"a44f19bc-a673-4424-92e7-b40b33057c04","added_by":"auto","created_at":"2023-09-21 05:57:36","extension":"mp4","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":28413555,"visible":true,"origin":"","legend":"Supplementary Video 4","description":"","filename":"SuppVideo4Rab11Lamp1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/097a2441c8ba871e6e0e7ce6.mp4"},{"id":43460499,"identity":"00bac7b5-dd73-45f9-8e0d-66bd7e3e0aef","added_by":"auto","created_at":"2023-09-21 05:57:36","extension":"mp4","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":49840103,"visible":true,"origin":"","legend":"Supplementary Video 5","description":"","filename":"SuppVideo5MitosisMitoHeLa.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/c89fcd446fc6d433e53ce077.mp4"},{"id":43460784,"identity":"34d7e796-fd62-4dbd-a640-8ae7aedec9fa","added_by":"auto","created_at":"2023-09-21 06:13:35","extension":"mp4","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":5236113,"visible":true,"origin":"","legend":"Supplementary Video 6","description":"","filename":"SuppVideo6MitosisSC35.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/a3df6b478e2fe011e45bcfac.mp4"},{"id":43460496,"identity":"247d9d48-c76d-473c-a602-30158a8540a9","added_by":"auto","created_at":"2023-09-21 05:57:36","extension":"mp4","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":23986491,"visible":true,"origin":"","legend":"Supplementary Video 7","description":"","filename":"SuppVideo7ConfocalMouseEmbryo.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/1790dd0f2036627d20a5c64a.mp4"},{"id":43460727,"identity":"92e16342-0fc9-4681-8b42-7dde8ccad2f9","added_by":"auto","created_at":"2023-09-21 06:05:35","extension":"mp4","order_by":8,"title":"","display":"","copyAsset":false,"role":"supplement","size":16260791,"visible":true,"origin":"","legend":"Supplementary Video 8","description":"","filename":"SuppVideo8WFCelegansEmbryo.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/7cf38359ebb0d8c62bb969b2.mp4"},{"id":43460498,"identity":"43566fc5-816b-4cea-a808-401b62b2a43f","added_by":"auto","created_at":"2023-09-21 05:57:36","extension":"mp4","order_by":9,"title":"","display":"","copyAsset":false,"role":"supplement","size":30596562,"visible":true,"origin":"","legend":"Supplementary Video 9","description":"","filename":"SuppVideo9DemoPlugin.mp4","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/f7c1950ac6b983b42d11db8a.mp4"},{"id":43460500,"identity":"32b7d2b3-43cd-4ef1-bb2b-fa7da30e90cb","added_by":"auto","created_at":"2023-09-21 05:57:38","extension":"pdf","order_by":10,"title":"","display":"","copyAsset":false,"role":"supplement","size":57408512,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryMaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3252567/v1/7a520178feb12479140ba3a7.pdf"}],"financialInterests":"There is a conflict of interest\nDong Li, C.Q. and Y.Z. have a pending patent application on the presented frameworks.","formattedTitle":"Zero-shot learning enables instant denoising and super-resolution\r\nin optical fluorescence microscopy","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3252567/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3252567/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Computational super-resolution (SR) methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding SR performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acquire due to the high dynamics of living cells. Here, we develop zero-shot deconvolution networks (ZS-DeconvNet) that instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary SR imaging conditions in an unsupervised manner without the need for either ground truths or additional data acquisition. We demonstrate the versatile applicability of ZS-DeconvNet on multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional (3D) wide-field microscopy, confocal microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy (SIM), which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans.","manuscriptTitle":"Zero-shot learning enables instant denoising and super-resolution\nin optical fluorescence microscopy","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-09-21 05:57:30","doi":"10.21203/rs.3.rs-3252567/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1e434f6c-5b3d-4511-b97b-55f29f5099e1","owner":[],"postedDate":"September 21st, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":24766349,"name":"Physical sciences/Optics and photonics/Optical techniques/Microscopy/Super-resolution microscopy"},{"id":24766350,"name":"Physical sciences/Optics and photonics/Optical techniques/Imaging and sensing"},{"id":24766351,"name":"Physical sciences/Optics and photonics/Optical techniques/Microscopy"}],"tags":[],"updatedAt":"2023-10-07T07:45:55+00:00","versionOfRecord":[],"versionCreatedAt":"2023-09-21 05:57:30","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3252567","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3252567","identity":"rs-3252567","version":["v1"]},"buildId":"ehx78VzkSd0WSzXnipQa-","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}