Autonomous Microscopy Experiments through Large Language Model Agents

Autonomous Microscopy Experiments through Large Language Model Agents | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Autonomous Microscopy Experiments through Large Language Model Agents N M Anoop Krishnan, Indrajeet Mandal, Jitendra Soni, Mohd Zaki, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5600537/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Oct, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract The emergence of large language models (LLMs) has accelerated the development of self-driving laboratories (SDLs) for materials research. Despite their transformative potential, current SDL implementations rely on rigid, predefined protocols that limit their adaptability to dynamic experimental scenarios across different labs. A significant challenge persists in measuring how effectively AI agents can replicate the adaptive decision-making and experimental intuition of expert scientists. Here, we introduce AILA (Artificially Intelligent Lab Assistant), a framework that automates atomic force microscopy (AFM) through LLM-driven agents. Using AFM as an experimental testbed, we develop AFMBench—a comprehensive evaluation suite that challenges AI agents based on language models like GPT-4o and GPT-3.5 to perform tasks spanning the scientific workflow: from experimental design to results analysis. Our systematic assessment shows that state-of-the-art language models struggle even with basic tasks such as documentation retrieval, leading to a significant decline in performance in multi-agent coordination scenarios.Further, we observe that LLMs exhibit a tendency to not adhere to instructions or even divagate to additional tasks beyond the original request, raising serious concerns regarding safety alignment aspects of AI agents for SDLs. Finally, we demonstrate the application of AILA on increasingly complex experiments open-ended experiments: automated AFM calibration, high-resolution feature detection, and mechanical property measurement. Our findings emphasize the necessity for stringent benchmarking protocols before deploying AI agents as laboratory assistants across scientific disciplines. Physical sciences/Materials science/Techniques and instrumentation/Imaging techniques Physical sciences/Materials science/Theory and computation/Computational methods self-driving laboratory atomic force microscopy benchmarking AI agents Full Text Additional Declarations There is NO Competing Interest. Supplementary Files SupplementaryMaterials.pdf SUPPLEMENTARY INFORMATION Cite Share Download PDF Status: Published Journal Publication published 14 Oct, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5600537","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":388214541,"identity":"0013a842-7517-4896-9d5a-621d9d8cf584","order_by":0,"name":"N M Anoop Krishnan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIiWNgGAWjYBACCQbGBmYQg5+ZsYGBB8RiJlaLZDNjYwORWqBqDA4ANfMQ4zDJ2YebPxfusMszPs7c/uANg508AzvvAbxapPkS26RnnkkuNjvM2Ng4hyHZsIGZLwGvFjkexjZm3jbmxG1ALc08DMwJDMw8BoS0NH/mbatP3NwM1lJPWIs0D2ODNG/b4cQNzGAthwlrkexhbANqOZ44A+iwmXMMjhu2EdIicYb9MdBh1Yn9/ccffHhTUS3Pz38GvxY0AFTMRor6UTAKRsEoGAXYAQBaDToKQaMdVwAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0003-1500-4947","institution":"Indian Institute of Technology Delhi","correspondingAuthor":true,"prefix":"","firstName":"N","middleName":"M Anoop","lastName":"Krishnan","suffix":""},{"id":388214542,"identity":"a30105c4-b5bf-4f2b-b2fb-8c4469de2896","order_by":1,"name":"Indrajeet Mandal","email":"","orcid":"","institution":"Indian Institute of Technology Delhi","correspondingAuthor":false,"prefix":"","firstName":"Indrajeet","middleName":"","lastName":"Mandal","suffix":""},{"id":388214543,"identity":"545433b7-e2ac-4e68-95a9-a1724ec3c499","order_by":2,"name":"Jitendra Soni","email":"","orcid":"","institution":"Indian Institute of Technology Delhi","correspondingAuthor":false,"prefix":"","firstName":"Jitendra","middleName":"","lastName":"Soni","suffix":""},{"id":388214544,"identity":"4117a1c1-78a2-4c65-a62d-3d14bff2790e","order_by":3,"name":"Mohd Zaki","email":"","orcid":"https://orcid.org/0000-0002-4551-3470","institution":"Indian Institute of Technology Delhi","correspondingAuthor":false,"prefix":"","firstName":"Mohd","middleName":"","lastName":"Zaki","suffix":""},{"id":388214545,"identity":"1e7b4f4a-15d7-4d56-b64b-c39f125ceb26","order_by":4,"name":"Morten Smedskjaer","email":"","orcid":"https://orcid.org/0000-0003-0476-2021","institution":"Aalborg University","correspondingAuthor":false,"prefix":"","firstName":"Morten","middleName":"","lastName":"Smedskjaer","suffix":""},{"id":388214546,"identity":"d4f3bbfe-a492-4ebc-8633-f077bac41b22","order_by":5,"name":"Katrin Wondraczek","email":"","orcid":"","institution":"Leibniz Institute of Photonic Technology","correspondingAuthor":false,"prefix":"","firstName":"Katrin","middleName":"","lastName":"Wondraczek","suffix":""},{"id":388214547,"identity":"40e4c4da-caed-4eb9-b331-2ce34db3881d","order_by":6,"name":"Lothar Wondraczek","email":"","orcid":"","institution":"University of Jena","correspondingAuthor":false,"prefix":"","firstName":"Lothar","middleName":"","lastName":"Wondraczek","suffix":""},{"id":388214548,"identity":"4c697619-8849-4760-adc8-91feb1a3e865","order_by":7,"name":"Nitya Nand Gosvami","email":"","orcid":"","institution":"Indian Institute of Technology Delhi","correspondingAuthor":false,"prefix":"","firstName":"Nitya","middleName":"Nand","lastName":"Gosvami","suffix":""}],"badges":[],"createdAt":"2024-12-07 20:50:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5600537/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5600537/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-025-64105-7","type":"published","date":"2025-10-14T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":93559094,"identity":"ec2dd90e-528e-4615-baf6-f17a564e3ff2","added_by":"auto","created_at":"2025-10-15 07:09:20","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1086409,"visible":true,"origin":"","legend":"Article File","description":"","filename":"ManuscriptAFMfinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5600537/v1_covered_94b820ed-7f73-4487-9f17-4234a8dd6cf9.pdf"},{"id":71830042,"identity":"5b73dad1-f6e7-40f9-922c-f35bff958354","added_by":"auto","created_at":"2024-12-19 02:26:21","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":522568,"visible":true,"origin":"","legend":"SUPPLEMENTARY INFORMATION","description":"","filename":"SupplementaryMaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5600537/v1/0bbf13eca35257b6d8dea4af.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Autonomous Microscopy Experiments through Large Language Model Agents","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"self-driving laboratory, atomic force microscopy, benchmarking, AI agents","lastPublishedDoi":"10.21203/rs.3.rs-5600537/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5600537/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The emergence of large language models (LLMs) has accelerated the development of self-driving laboratories (SDLs) for materials research. Despite their transformative potential, current SDL implementations rely on rigid, predefined protocols that limit their adaptability to dynamic experimental scenarios across different labs. A significant challenge persists in measuring how effectively AI agents can replicate the adaptive decision-making and experimental intuition of expert scientists. Here, we introduce AILA (Artificially Intelligent Lab Assistant), a framework that automates atomic force microscopy (AFM) through LLM-driven agents. Using AFM as an experimental testbed, we develop AFMBench—a comprehensive evaluation suite that challenges AI agents based on language models like GPT-4o and GPT-3.5 to perform tasks spanning the scientific workflow: from experimental design to results analysis. Our systematic assessment shows that state-of-the-art language models struggle even with basic tasks such as documentation retrieval, leading to a significant decline in performance in multi-agent coordination scenarios.Further, we observe that LLMs exhibit a tendency to not adhere to instructions or even divagate to additional tasks beyond the original request, raising serious concerns regarding safety alignment aspects of AI agents for SDLs. Finally, we demonstrate the application of AILA on increasingly complex experiments open-ended experiments: automated AFM calibration, high-resolution feature detection, and mechanical property measurement. Our findings emphasize the necessity for stringent benchmarking protocols before deploying AI agents as laboratory assistants across scientific disciplines.","manuscriptTitle":"Autonomous Microscopy Experiments through Large Language Model Agents","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-19 02:26:16","doi":"10.21203/rs.3.rs-5600537/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"9892fa8c-de50-4b91-b152-ed5db763fcfb","owner":[],"postedDate":"December 19th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":41371801,"name":"Physical sciences/Materials science/Techniques and instrumentation/Imaging techniques"},{"id":41371802,"name":"Physical sciences/Materials science/Theory and computation/Computational methods"}],"tags":[],"updatedAt":"2025-10-15T07:09:14+00:00","versionOfRecord":{"articleIdentity":"rs-5600537","link":"https://doi.org/10.1038/s41467-025-64105-7","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2025-10-14 04:00:00","publishedOnDateReadable":"October 14th, 2025"},"versionCreatedAt":"2024-12-19 02:26:16","video":"","vorDoi":"10.1038/s41467-025-64105-7","vorDoiUrl":"https://doi.org/10.1038/s41467-025-64105-7","workflowStages":[]},"version":"v1","identity":"rs-5600537","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5600537","identity":"rs-5600537","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}