Deep Learning Driven Adaptive Optics for Single Molecule Localization Microscopy

Deep Learning Driven Adaptive Optics for Single Molecule Localization Microscopy | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Brief Communication Deep Learning Driven Adaptive Optics for Single Molecule Localization Microscopy Fang Huang, Peiyi Zhang, Donghan Ma, Xi Cheng, Andy Tsai, Yu Tang, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1690151/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 28 Sep, 2023 Read the published version in Nature Methods → Version 1 posted You are reading this latest preprint version Abstract The inhomogeneous refractive indices of biological tissues blur and distort single molecule emission patterns generating image artifacts and decreasing the achievable resolution of single molecule localization microscopy (SMLM). To compensate tissue induced aberrations, conventional sensorless adaptive optics methods rely on iterative mirror-changes and image-quality metrics. However, these metrics result in inconsistent, and sometimes opposite, metric responses and thus fundamentally limit the efficacy of these approaches for aberration correction in tissues. Bypassing the previous iterative trial-then-evaluate processes, we developed deep learning driven adaptive optics for SMLM to allow direct inference of wavefront distortion and near real-time compensation. Our trained deep neural network monitors the individual emission patterns from single molecule experiments, infers their shared wavefront distortion, feeds the estimates through a dynamic filter, and drives a deformable mirror to compensate sample induced aberrations. We demonstrated that our method simultaneously estimates and compensates 28 types of wavefront deformation shapes, restores single molecule emission patterns approaching the conditions untouched by the specimen, and improves the resolution and fidelity of 3D SMLM through >130 µm thick tissue specimens, with as few as 3-20 mirror changes. Full Text Additional Declarations There is NO Competing Interest. Supplementary Files 20220523DLAOSupplementssubmit.pdf Supplementary Figures and Notes SupplementaryVideos.zip Supplementary Videos Cite Share Download PDF Status: Published Journal Publication published 28 Sep, 2023 Read the published version in Nature Methods → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1690151","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Brief Communication","associatedPublications":[],"authors":[{"id":110441512,"identity":"abd07fa3-4c32-449b-83c4-979d339a37cf","order_by":0,"name":"Fang Huang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYBAC9gYGBmY4i4ENzDTAq4XnAEwLiEWiFokEYrWwnz38urDNLk8+8u0x6Yqyw3IM7M3bJPBq4clLs57ZllxseDsvTfLMucPGDDzHyvBqsWfIMTPmbWNO3Dg7x0yyse1wYoNEjhl+W/jfgLTUJ26ceQaspb5B/g0BLRI5xo95gYbPl+ABa0lgADIIaHljxsxz7njiBp68ZMuGc+mGbTxpxRb4HZZj/JmnrDpxfvvZgzcbyqzl+dkPb7yBTwsQsIGdYXCAB8oloBwEmD+ASPkGHiLUjoJRMApGwYgEAN7wRN9P90ndAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-1301-1799","institution":"Purdue University West Lafayette","correspondingAuthor":true,"prefix":"","firstName":"Fang","middleName":"","lastName":"Huang","suffix":""},{"id":110441513,"identity":"6d9efa2b-305c-4129-903c-783f447203ed","order_by":1,"name":"Peiyi Zhang","email":"","orcid":"https://orcid.org/0000-0002-1100-3720","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Peiyi","middleName":"","lastName":"Zhang","suffix":""},{"id":110441514,"identity":"61a3bbad-a217-4b38-96de-639f99191e97","order_by":2,"name":"Donghan Ma","email":"","orcid":"https://orcid.org/0000-0001-6264-2824","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Donghan","middleName":"","lastName":"Ma","suffix":""},{"id":110441515,"identity":"652df4eb-ce61-46e7-900a-0bec201b82c8","order_by":3,"name":"Xi Cheng","email":"","orcid":"","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Xi","middleName":"","lastName":"Cheng","suffix":""},{"id":110441516,"identity":"28c3ee60-49b4-4ab5-b88c-a75c779fb73d","order_by":4,"name":"Andy Tsai","email":"","orcid":"","institution":"Indiana University","correspondingAuthor":false,"prefix":"","firstName":"Andy","middleName":"","lastName":"Tsai","suffix":""},{"id":110441517,"identity":"cead4021-e744-4bbd-a9af-93c75b367ae0","order_by":5,"name":"Yu Tang","email":"","orcid":"","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Yu","middleName":"","lastName":"Tang","suffix":""},{"id":110441518,"identity":"75740c0f-bc85-40e2-b505-3a916e009626","order_by":6,"name":"Hao-Cheng Gao","email":"","orcid":"","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Hao-Cheng","middleName":"","lastName":"Gao","suffix":""},{"id":110441519,"identity":"1bcb3fb1-1300-4699-8a75-17f38c04ece1","order_by":7,"name":"Li Fang","email":"","orcid":"","institution":"Purdue University","correspondingAuthor":false,"prefix":"","firstName":"Li","middleName":"","lastName":"Fang","suffix":""},{"id":110441520,"identity":"3bfe6a8d-6a6d-4d67-b136-48b39481b7b6","order_by":8,"name":"Cheng Bi","email":"","orcid":"","institution":"Purdue University West Lafayette","correspondingAuthor":false,"prefix":"","firstName":"Cheng","middleName":"","lastName":"Bi","suffix":""},{"id":110441521,"identity":"7b3369fc-4096-4e68-a2c0-0bff32fde503","order_by":9,"name":"Gary Landreth","email":"","orcid":"https://orcid.org/0000-0002-8808-424X","institution":"Indiana University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Gary","middleName":"","lastName":"Landreth","suffix":""},{"id":110441522,"identity":"ca07241d-baf3-4128-ad74-beb9a06394c6","order_by":10,"name":"Alexander Chubykin","email":"","orcid":"https://orcid.org/0000-0001-8224-9296","institution":"Purdue University West Lafayette","correspondingAuthor":false,"prefix":"","firstName":"Alexander","middleName":"","lastName":"Chubykin","suffix":""}],"badges":[],"createdAt":"2022-05-24 20:26:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1690151/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1690151/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41592-023-02029-0","type":"published","date":"2023-09-28T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":22179766,"identity":"d0c7db55-d13a-4eb9-bfda-6c7c32667dae","added_by":"auto","created_at":"2022-06-02 14:35:02","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":9128670,"visible":true,"origin":"","legend":"","description":"","filename":"20220523Manuscriptcombinedsubmit.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1690151/v1_covered.pdf"},{"id":22179744,"identity":"30eda7d0-a903-4f60-ac50-e9fa52cd3de1","added_by":"auto","created_at":"2022-06-02 14:34:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":10634671,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary Figures and Notes\u003c/p\u003e","description":"","filename":"20220523DLAOSupplementssubmit.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1690151/v1/a5d8fed858c76c3cf0850573.pdf"},{"id":22179765,"identity":"93fe4a4d-5f6d-41dc-baf7-54721cc8860e","added_by":"auto","created_at":"2022-06-02 14:34:51","extension":"zip","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":928570420,"visible":true,"origin":"","legend":"Supplementary Videos","description":"","filename":"SupplementaryVideos.zip","url":"https://assets-eu.researchsquare.com/files/rs-1690151/v1/6d44cc4454b83382cf883e0c.zip"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Deep Learning Driven Adaptive Optics for Single Molecule Localization Microscopy","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1690151/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-1690151/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1690151/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The inhomogeneous refractive indices of biological tissues blur and distort single molecule emission patterns generating image artifacts and decreasing the achievable resolution of single molecule localization microscopy (SMLM). To compensate tissue induced aberrations, conventional sensorless adaptive optics methods rely on iterative mirror-changes and image-quality metrics. However, these metrics result in inconsistent, and sometimes opposite, metric responses and thus fundamentally limit the efficacy of these approaches for aberration correction in tissues. Bypassing the previous iterative trial-then-evaluate processes, we developed deep learning driven adaptive optics for SMLM to allow direct inference of wavefront distortion and near real-time compensation. Our trained deep neural network monitors the individual emission patterns from single molecule experiments, infers their shared wavefront distortion, feeds the estimates through a dynamic filter, and drives a deformable mirror to compensate sample induced aberrations. We demonstrated that our method simultaneously estimates and compensates 28 types of wavefront deformation shapes, restores single molecule emission patterns approaching the conditions untouched by the specimen, and improves the resolution and fidelity of 3D SMLM through \u003e130 µm thick tissue specimens, with as few as 3-20 mirror changes.","manuscriptTitle":"Deep Learning Driven Adaptive Optics for Single Molecule Localization Microscopy","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-06-02 14:34:27","doi":"10.21203/rs.3.rs-1690151/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-methods","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"nmeth","sideBox":"Learn more about [Nature Methods](http://www.nature.com/nmeth)","snPcode":"","submissionUrl":"","title":"Nature Methods","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Research","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"9f51d4c7-719b-4d72-8f7f-c0285f61c626","owner":[],"postedDate":"June 2nd, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2023-09-29T16:57:38+00:00","versionOfRecord":{"articleIdentity":"rs-1690151","link":"https://doi.org/10.1038/s41592-023-02029-0","journal":{"identity":"nature-methods","isVorOnly":false,"title":"Nature Methods"},"publishedOn":"2023-09-28 04:00:00","publishedOnDateReadable":"September 28th, 2023"},"versionCreatedAt":"2022-06-02 14:34:27","video":"","vorDoi":"10.1038/s41592-023-02029-0","vorDoiUrl":"https://doi.org/10.1038/s41592-023-02029-0","workflowStages":[]},"version":"v1","identity":"rs-1690151","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1690151","identity":"rs-1690151","version":["v1"]},"buildId":"_2-kVJe1T_tPrBINL-cwx","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}