Bayesian Optimization for Biochemical Discovery with LLMs

Bayesian Optimization for Biochemical Discovery with LLMs | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Bayesian Optimization for Biochemical Discovery with LLMs Rafael Gómez-Bombarelli, Mattias Akke, Soojung Yang, Jurgis Ruza, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8216063/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Incorporating prior domain knowledge into Bayesian optimization (BO) remains difficult for statistical methods, which also typically suffer from limited interpretability. Large language models (LLMs) offer complementary strengths in reasoning and knowledge integration, but it remains unclear when and how they improve BO. We address this gap through a systematic analysis of the success and failure modes of LLM-enabled approaches to BO and propose strategies to overcome their limitations. Two types of scientific tasks are benchmarked: molecular optimization using string-based representations and optimization of four-residue binding motifs within proteins. In the molecular task, we find that poor data comprehension and the limited ability to process large in-context data hinder LLM performance. Accordingly, we propose an agentic workflow that orchestrates chemical tools and statistical BO and show its effectiveness. In the protein task, reasoning LLMs prove effective at domain knowledge-based hypothesis generation, improving optimization performance while providing interpretable design strategies. Across both domains, we observe a tendency for LLMs to overfixate on irrelevant context, where withholding information paradoxically improves performance. These results clarify the conditions under which LLMs enhance BO and suggest hybrid approaches that combine statistical rigor with LLM-enabled reasoning and interpretability. Biological sciences/Computational biology and bioinformatics/Machine learning Physical sciences/Chemistry/Cheminformatics Full Text Additional Declarations There is NO Competing Interest. Supplementary Files LLMALsupp.pdf Supplementary Information Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8216063","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":578131205,"identity":"34909a33-582e-4047-80d5-834f17d63571","order_by":0,"name":"Rafael Gómez-Bombarelli","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtElEQVRIiWNgGAWjYFACxgcMjA02DGwgNg9xWpgNgFrSGNjYSNRymIGBaC3m7YcZP37dcT6aT76B8cHbNiK0yJxJZpaWPXM7t42NgdlwLjFaJBjyD0hLtoG1sEnzEqWF/zHzb8m2cyAt7L+J0yKRzCb5se0A2BZmIrU8ZrNmPJMM1JLYLDnnHFEOS2a++XOHXe785sMHP7wpI0ILCDBDooOxgUj1ILU/iFc7CkbBKBgFIxEAAN+XMWVKNQ+BAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-9495-8599","institution":"Massachusetts Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Rafael","middleName":"","lastName":"Gómez-Bombarelli","suffix":""},{"id":578131206,"identity":"c869b59b-71e6-4096-b935-63fa29c11911","order_by":1,"name":"Mattias Akke","email":"","orcid":"https://orcid.org/0000-0001-9207-0695","institution":"MIT","correspondingAuthor":false,"prefix":"","firstName":"Mattias","middleName":"","lastName":"Akke","suffix":""},{"id":578131207,"identity":"c29f3d79-3091-433a-aaa0-0586ab10661c","order_by":2,"name":"Soojung Yang","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Soojung","middleName":"","lastName":"Yang","suffix":""},{"id":578131208,"identity":"caf70ee0-39d3-499f-9b98-154e1f0d2900","order_by":3,"name":"Jurgis Ruza","email":"","orcid":"","institution":"MIT","correspondingAuthor":false,"prefix":"","firstName":"Jurgis","middleName":"","lastName":"Ruza","suffix":""},{"id":578131209,"identity":"34a349d4-1525-4fc1-beb4-929e4db812a9","order_by":4,"name":"Jinyeop Song","email":"","orcid":"","institution":"MIT","correspondingAuthor":false,"prefix":"","firstName":"Jinyeop","middleName":"","lastName":"Song","suffix":""},{"id":578131210,"identity":"b6020e7f-c76e-4c24-8562-f53a8a7e2971","order_by":5,"name":"Elton Pan","email":"","orcid":"https://orcid.org/0000-0003-0629-8429","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Elton","middleName":"","lastName":"Pan","suffix":""}],"badges":[],"createdAt":"2025-11-26 20:40:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8216063/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8216063/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105567076,"identity":"0eb44343-7a65-40d2-a717-beebf4a0c397","added_by":"auto","created_at":"2026-03-27 12:58:14","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2659162,"visible":true,"origin":"","legend":"Article File","description":"","filename":"NATMACHINTELLrevision.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8216063/v1_covered_6a6ba4cf-767d-478f-a832-3c7576b06cf7.pdf"},{"id":100861675,"identity":"608f631a-93f8-4791-94b6-066ff1529919","added_by":"auto","created_at":"2026-01-22 07:46:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":2595121,"visible":true,"origin":"","legend":"Supplementary Information","description":"","filename":"LLMALsupp.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8216063/v1/c78e2f8f31686be2574e82aa.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Bayesian Optimization for Biochemical Discovery with LLMs","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8216063/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8216063/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Incorporating prior domain knowledge into Bayesian optimization (BO) remains difficult for statistical methods, which also typically suffer from limited interpretability. Large language models (LLMs) offer complementary strengths in reasoning and knowledge integration, but it remains unclear when and how they improve BO. We address this gap through a systematic analysis of the success and failure modes of LLM-enabled approaches to BO and propose strategies to overcome their limitations. Two types of scientific tasks are benchmarked: molecular optimization using string-based representations and optimization of four-residue binding motifs within proteins. In the molecular task, we find that poor data comprehension and the limited ability to process large in-context data hinder LLM performance. Accordingly, we propose an agentic workflow that orchestrates chemical tools and statistical BO and show its effectiveness. In the protein task, reasoning LLMs prove effective at domain knowledge-based hypothesis generation, improving optimization performance while providing interpretable design strategies. Across both domains, we observe a tendency for LLMs to overfixate on irrelevant context, where withholding information paradoxically improves performance. These results clarify the conditions under which LLMs enhance BO and suggest hybrid approaches that combine statistical rigor with LLM-enabled reasoning and interpretability.","manuscriptTitle":"Bayesian Optimization for Biochemical Discovery with LLMs","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-22 07:35:28","doi":"10.21203/rs.3.rs-8216063/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"eaffb6e2-d440-47aa-a8a5-fe6cb8a0f61e","owner":[],"postedDate":"January 22nd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":61512590,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":61512591,"name":"Physical sciences/Chemistry/Cheminformatics"}],"tags":[],"updatedAt":"2026-03-26T15:52:50+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-22 07:35:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8216063","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8216063","identity":"rs-8216063","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}