Deep Learning Foundation Models from Classical Molecular Descriptors

Deep Learning Foundation Models from Classical Molecular Descriptors | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Deep Learning Foundation Models from Classical Molecular Descriptors William Green, Jackson Burns, Akshat Shirish Zalte, Charlles Abreu, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8834086/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Fast and accurate data-driven prediction of molecular properties is pivotal to scientific advancements across myriad chemical domains. Deep learning methods have recently garnered much attention, despite their inability to outperform classical machine learning methods when tested on practical, real-world benchmarks with limited training data. This study seeks to bridge this gap with CheMeleon, a O(10M) parameter foundation model that enables directed message-passing neural networks to finally exceed the performance of classical methods. Evaluated on 58 benchmark datasets from Polaris and MoleculeACE, CheMeleon achieves a win rate of 75% on Polaris tasks, outperforming baselines like Random Forest (68%), fastprop (36%), and Chemprop (32%), and a 97% win rate on MoleculeACE assays, surpassing Random Forest (50%) and other foundation models. Unlike conventional pre-training approaches that rely on noisy experimental data or biased quantum mechanical simulations, CheMeleon utilizes low-noise molecular descriptors to learn rich and highly transferable molecular representations, suggesting a new avenue for foundation model pre-training. Biological sciences/Computational biology and bioinformatics/Machine learning Physical sciences/Chemistry/Cheminformatics Biological sciences/Drug discovery Physical sciences/Mathematics and computing/Computational science foundation models descriptors chemistry Full Text Additional Declarations There is NO Competing Interest. Supplementary Files si.pdf Supporting Information: Deep Learning Foundation Models from Classical Molecular Descriptors figure2sourcedata.csv figure3sourcedata.csv figure4sourcedata.csv Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8834086","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":605563825,"identity":"dbbbfab7-f7ca-4866-bb63-d6fa9333f56d","order_by":0,"name":"William Green","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAsklEQVRIiWNgGAWjYDACCQglx8AMothI0GLMwMxMopbEBgZitRjcbr/24UPFnfS17fwHGD6UHSZCy50zxTNnnHmWu+0wMwPjjHNEaDG7kZPMzNt2GKwFxCBSy99/h9PNQFr+Eqcl/TAzY8PhBLAWRmK02N/IYWbsOXbYEOgwg4M959IJa5Gckf6Y4UfNYXmz8wcfPvhRZk1YCwMDjwGceYAY9UDA/oBIhaNgFIyCUTBiAQAngD1S9xUvDwAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0003-2603-9694","institution":"Massachusetts Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"William","middleName":"","lastName":"Green","suffix":""},{"id":605563826,"identity":"c055a1b3-eca4-4215-ad18-48554f12bff1","order_by":1,"name":"Jackson Burns","email":"","orcid":"https://orcid.org/0000-0002-0657-9426","institution":"MIT","correspondingAuthor":false,"prefix":"","firstName":"Jackson","middleName":"","lastName":"Burns","suffix":""},{"id":605563827,"identity":"d15e493b-4ea2-42fc-9767-ae22580d2c99","order_by":2,"name":"Akshat Shirish Zalte","email":"","orcid":"https://orcid.org/0000-0001-9549-7547","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Akshat","middleName":"Shirish","lastName":"Zalte","suffix":""},{"id":605563828,"identity":"b94c55a7-2911-4f7c-919b-d0a413aec124","order_by":3,"name":"Charlles Abreu","email":"","orcid":"","institution":"Massachusetts Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Charlles","middleName":"","lastName":"Abreu","suffix":""},{"id":605563829,"identity":"123e1f75-64bd-4f9a-8dfe-1c145ca5494e","order_by":4,"name":"Jochen Sieg","email":"","orcid":"","institution":"BASF SE","correspondingAuthor":false,"prefix":"","firstName":"Jochen","middleName":"","lastName":"Sieg","suffix":""},{"id":605563830,"identity":"48f80ec7-a5e1-4ac7-94fd-2b47a2fb8dd0","order_by":5,"name":"Christian Feldmann","email":"","orcid":"","institution":"BASF SE","correspondingAuthor":false,"prefix":"","firstName":"Christian","middleName":"","lastName":"Feldmann","suffix":""},{"id":605563831,"identity":"cab64d6c-29fb-458a-a711-651ff5b55863","order_by":6,"name":"Miriam Mathea","email":"","orcid":"","institution":"BASF","correspondingAuthor":false,"prefix":"","firstName":"Miriam","middleName":"","lastName":"Mathea","suffix":""}],"badges":[],"createdAt":"2026-02-09 20:10:41","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8834086/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8834086/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107705792,"identity":"461ba87e-aa5b-48f2-b328-e58ee9f70fe2","added_by":"auto","created_at":"2026-04-24 09:15:18","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":431113,"visible":true,"origin":"","legend":"Article File","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8834086/v1_covered_10d40c13-ea74-4abe-9285-5fa0074f16f1.pdf"},{"id":104703006,"identity":"454b7483-dd8e-4b49-82c0-5a788a756df7","added_by":"auto","created_at":"2026-03-16 08:52:06","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":844341,"visible":true,"origin":"","legend":"Supporting Information: Deep Learning Foundation Models from Classical Molecular Descriptors","description":"","filename":"si.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8834086/v1/c5362286af93146edbbdf1ed.pdf"},{"id":104783216,"identity":"4a597b59-a008-4c84-9db8-3adc8a7a5519","added_by":"auto","created_at":"2026-03-17 07:58:24","extension":"csv","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":52657,"visible":true,"origin":"","legend":"","description":"","filename":"figure2sourcedata.csv","url":"https://assets-eu.researchsquare.com/files/rs-8834086/v1/1b22dc2831ef8580a24970e6.csv"},{"id":104703007,"identity":"72c73fa5-af7b-4939-8372-68ac82542287","added_by":"auto","created_at":"2026-03-16 08:52:06","extension":"csv","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":101370,"visible":true,"origin":"","legend":"","description":"","filename":"figure3sourcedata.csv","url":"https://assets-eu.researchsquare.com/files/rs-8834086/v1/6c771d85be1af3b64bc4024c.csv"},{"id":104703008,"identity":"addf6631-cf84-4cee-81da-bedb6ff92f23","added_by":"auto","created_at":"2026-03-16 08:52:06","extension":"csv","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":475674,"visible":true,"origin":"","legend":"","description":"","filename":"figure4sourcedata.csv","url":"https://assets-eu.researchsquare.com/files/rs-8834086/v1/e5ba27dd76cb92fa086abe24.csv"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Deep Learning Foundation Models from Classical Molecular Descriptors","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"foundation models, descriptors, chemistry","lastPublishedDoi":"10.21203/rs.3.rs-8834086/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8834086/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Fast and accurate data-driven prediction of molecular properties is pivotal to scientific advancements across myriad chemical domains. Deep learning methods have recently garnered much attention, despite their inability to outperform classical machine learning methods when tested on practical, real-world benchmarks with limited training data. This study seeks to bridge this gap with CheMeleon, a O(10M) parameter foundation model that enables directed message-passing neural networks to finally exceed the performance of classical methods. Evaluated on 58 benchmark datasets from Polaris and MoleculeACE, CheMeleon achieves a win rate of 75% on Polaris tasks, outperforming baselines like Random Forest (68%), fastprop (36%), and Chemprop (32%), and a 97% win rate on MoleculeACE assays, surpassing Random Forest (50%) and other foundation models. Unlike conventional pre-training approaches that rely on noisy experimental data or biased quantum mechanical simulations, CheMeleon utilizes low-noise molecular descriptors to learn rich and highly transferable molecular representations, suggesting a new avenue for foundation model pre-training.","manuscriptTitle":"Deep Learning Foundation Models from Classical Molecular Descriptors","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-16 08:51:44","doi":"10.21203/rs.3.rs-8834086/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"895aebae-e583-41b7-8dac-734bf24562b4","owner":[],"postedDate":"March 16th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":64447093,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":64447094,"name":"Physical sciences/Chemistry/Cheminformatics"},{"id":64447095,"name":"Biological sciences/Drug discovery"},{"id":64447096,"name":"Physical sciences/Mathematics and computing/Computational science"}],"tags":[],"updatedAt":"2026-04-23T08:46:31+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-16 08:51:44","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8834086","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8834086","identity":"rs-8834086","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}