Systematic benchmarking of foundation models and classical baselines for microbiome-based disease prediction

Systematic benchmarking of foundation models and classical baselines for microbiome-based disease prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Systematic benchmarking of foundation models and classical baselines for microbiome-based disease prediction Jin Mu, Zheng-Zheng Tang, Guanhua Chen This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8912605/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Background: Microbiome-based disease prediction is often hindered by sparse, compositional features and substantial inter-study heterogeneity. Foundation models and LLM-derived representations could, in principle, improve robustness and cross-cohort generalization, but their utility for microbiome prediction has not been systematically benchmarked. Results: We benchmarked classical machine-learning baselines (regularized logistic regression and random forests), standard numerical feature representations, GPT-derived semantic embeddings, and two foundation-model paradigms: a general-purpose tabular foundation model (TabPFN) and a microbiome-specific foundation model (MGM). Using 83 publicly curated case–control cohorts spanning 20 diseases profiled by 16S rRNA sequencing and shotgun metagenomics, we assessed performance under three settings: intra-cohort cross-validation, cross-cohort transfer (train on one cohort, test on others), and leave-one-study-out (LOSO) validation. GPT-derived semantic embeddings consistently underperformed standard numerical representations. TabPFN achieved strong out-of-the-box performance and competitive cross-cohort robustness, but did not consistently outperform well-tuned classical baselines across cohorts. MGM’s performance was disease-dependent and generally lagged behind the strongest tabular baselines, suggesting that current microbiome-specific pretraining at genus resolution does not yet confer a consistent advantage under study heterogeneity. Batch-effect correction methods provided limited and non-uniform improvements in LOSO evaluations. Conclusions: In this large-scale benchmark, current foundation-model approaches offer, at best, modest gains over strong classical baselines for microbiome-based disease prediction. Our results highlight that standard numerical representations remain difficult to beat, general-purpose tabular foundation models can provide strong out-of-the-box performance under domain shift, and microbiome-specific foundation models may require advances in pretraining scale, taxonomic resolution, and architecture to translate pretraining into reliable cross-study generalization. Full Text Additional Declarations No competing interests reported. Supplementary Files Supptable.xlsx Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 17 May, 2026 Reviews received at journal 30 Apr, 2026 Reviewers agreed at journal 23 Apr, 2026 Reviewers agreed at journal 20 Apr, 2026 Reviews received at journal 18 Mar, 2026 Reviewers agreed at journal 12 Mar, 2026 Reviewers agreed at journal 09 Mar, 2026 Reviewers invited by journal 06 Mar, 2026 Editor assigned by journal 20 Feb, 2026 Submission checks completed at journal 19 Feb, 2026 First submitted to journal 18 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8912605","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":596330375,"identity":"d7da7e48-0267-4637-a3af-437b9b04eeed","order_by":0,"name":"Jin Mu","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"prefix":"","firstName":"Jin","middleName":"","lastName":"Mu","suffix":""},{"id":596330378,"identity":"b855e4fb-5b51-4649-8374-d6b4e5afc901","order_by":1,"name":"Zheng-Zheng Tang","email":"","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":false,"prefix":"","firstName":"Zheng-Zheng","middleName":"","lastName":"Tang","suffix":""},{"id":596330379,"identity":"1af5d35c-3430-4870-9ba5-613d5e01fd38","order_by":2,"name":"Guanhua Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAw0lEQVRIiWNgGAWjYFACHiCu+CfHwAzmMROr5cwBYxK1MLYdSGxgIFaLvPvZg48L2O6kbzjO/vADQ4U1TC9uYHgmL9l4Bs+z3A2HeYwlGM6kE6FlBo+ZNI8Ec+62wzxsQBceJkqL+W8eA+Z0s8PszxgY/xGhRV6Cx4yZJ+FwgtlhBjMGxgYitBjw5BhL8xxIM9wP8kvCsXRjwra0nzH8zPvPRl6y//jDDx9qrGUJ23IAmZdASDnYFoKGjoJRMApGwSgAAIEkOt7/t6HQAAAAAElFTkSuQmCC","orcid":"","institution":"University of Wisconsin–Madison","correspondingAuthor":true,"prefix":"","firstName":"Guanhua","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2026-02-18 22:23:36","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8912605/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8912605/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103506863,"identity":"1b7ac682-f7c7-4c95-b32e-200b452ffec3","added_by":"auto","created_at":"2026-02-26 13:39:47","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6463858,"visible":true,"origin":"","legend":"","description":"","filename":"GenomeBiology.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8912605/v1_covered_768895e2-b450-4c3b-b5da-29bd44950011.pdf"},{"id":103387580,"identity":"e08c9ea7-b216-46cf-826f-798f610b936f","added_by":"auto","created_at":"2026-02-25 07:07:59","extension":"xlsx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":235868,"visible":true,"origin":"","legend":"","description":"","filename":"Supptable.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-8912605/v1/ce8ef40fd183c09c4722e618.xlsx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Systematic benchmarking of foundation models and classical baselines for microbiome-based disease prediction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"genome-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"gbio","sideBox":"Learn more about [Genome Biology](https://genomebiology.biomedcentral.com/)","snPcode":"13059","submissionUrl":"https://submission.springernature.com/new-submission/13059/3","title":"Genome Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8912605/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8912605/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Microbiome-based disease prediction is often hindered by sparse, compositional features and substantial inter-study heterogeneity. Foundation models and LLM-derived representations could, in principle, improve robustness and cross-cohort generalization, but their utility for microbiome prediction has not been systematically benchmarked.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e We benchmarked classical machine-learning baselines (regularized logistic regression and random forests), standard numerical feature representations, GPT-derived semantic embeddings, and two foundation-model paradigms: a general-purpose tabular foundation model (TabPFN) and a microbiome-specific foundation model (MGM). Using 83 publicly curated case–control cohorts spanning 20 diseases profiled by 16S rRNA sequencing and shotgun metagenomics, we assessed performance under three settings: intra-cohort cross-validation, cross-cohort transfer (train on one cohort, test on others), and leave-one-study-out (LOSO) validation. GPT-derived semantic embeddings consistently underperformed standard numerical representations. TabPFN achieved strong out-of-the-box performance and competitive cross-cohort robustness, but did not consistently outperform well-tuned classical baselines across cohorts. MGM’s performance was disease-dependent and generally lagged behind the strongest tabular baselines, suggesting that current microbiome-specific pretraining at genus resolution does not yet confer a consistent advantage under study heterogeneity. Batch-effect correction methods provided limited and non-uniform improvements in LOSO evaluations.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e In this large-scale benchmark, current foundation-model approaches offer, at best, modest gains over strong classical baselines for microbiome-based disease prediction. Our results highlight that standard numerical representations remain difficult to beat, general-purpose tabular foundation models can provide strong out-of-the-box performance under domain shift, and microbiome-specific foundation models may require advances in pretraining scale, taxonomic resolution, and architecture to translate pretraining into reliable cross-study generalization.\u003c/p\u003e","manuscriptTitle":"Systematic benchmarking of foundation models and classical baselines for microbiome-based disease prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-25 07:07:49","doi":"10.21203/rs.3.rs-8912605/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-05-17T12:56:41+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-01T00:01:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"317510098886957357107731353111051504120","date":"2026-04-24T01:15:16+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"249296365501203201836566432025333919822","date":"2026-04-20T10:33:44+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-18T19:51:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"155769623858082283678377102637298027860","date":"2026-03-12T14:51:13+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"37530307820047660563062590891768585082","date":"2026-03-09T12:50:01+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-06T22:17:43+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-20T18:29:25+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-19T10:19:48+00:00","index":"","fulltext":""},{"type":"submitted","content":"Genome Biology","date":"2026-02-18T22:21:19+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"genome-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"gbio","sideBox":"Learn more about [Genome Biology](https://genomebiology.biomedcentral.com/)","snPcode":"13059","submissionUrl":"https://submission.springernature.com/new-submission/13059/3","title":"Genome Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5720f09b-cd03-4098-ae4e-00bb5523457e","owner":[],"postedDate":"February 25th, 2026","published":true,"recentEditorialEvents":[{"type":"editorInvitedReview","content":"","date":"2026-05-17T12:56:41+00:00","index":27,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-01T00:01:14+00:00","index":26,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-03-06T22:23:15+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-25 07:07:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8912605","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8912605","identity":"rs-8912605","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}