BiomeGPT: A foundation model for the human gut microbiome

The human gut microbiome encodes rich information about host health, yet current analysis pipelines remain narrowly optimized for individual tasks. This limits our ability to gain a thorough view of how the microbiome impacts health and disease. Here we introduce BiomeGPT, a transformer-based foundation model pretrained on over 13,300 human gut metagenomes spanning 32 phenotypes—including healthy and 31 diverse diseases—to learn context-aware, species-level gut microbiome representations. The model captures quantitative compositional structure and intricate cross-species dependencies embedded within community profiles. When fine-tuned for predicting host health status, BiomeGPT accurately distinguishes healthy from diseased microbiomes and resolves individual disease states across a broad clinical spectrum. Furthermore, its attention patterns reveal biologically plausible microbial signatures, highlighting both shared and disease-specific microbial species linked to host phenotypes. By providing a unified, scalable framework for species-level gut microbiome representation learning and prediction, BiomeGPT enables new avenues for biomarker discovery, disease stratification, and microbiome-driven precision medicine.

Microbiome; Generative AI; Foundation models; Precision medicine. .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint

The human gut microbiome is a dynamic and complex ecosystem that plays a central role in host health as it shapes a broad spectrum of physiologic processes including digestion of complex nutrient molecules, modulation of the immune and metabolic system, and even the development of neurobehavior. Disruptions in the gut microbiome's composition and function—commonly referred to as gut dysbiosis—have been implicated in a broad continuum of diseases, including gastrointestinal conditions like inflammatory bowel disease (IBD) and colorectal cancer, systemic conditions such as obesity, type 2 diabetes, and cardiovascular disease, and neurodegenerative conditions 1, 2. Due to its far-reaching e\ects on human physiology and disease, the gut microbiome has emerged as a prime area of investigation for uncovering disease mechanisms and developing new therapies. Because the microbes within a microbiome function as a tightly connected network rather than isolated players, a holistic, systems-level perspective is essential to unravel how shifts in community structure translate into health or disease. Over the past decade, the computational toolkit for microbiome analysis has significantly expanded with a wide array of tools developed for multivariate analysis 3, differential abundance analysis 4, 5, and microbiome-based phenotype prediction, among others, to characterize microbiome alterations and their association with host health. However, existing computational tools are rather scattered, each specializing in a specific analytical task. Likewise, the datasets used for each tool are typically task- specific and limited in scale and scope. As a result, microbiome studies often rely on ad hoc combinations of independent pipelines, leading to fragmented analytical workflows that can potentially introduce biases, hinder reproducibility, and complicate data integration across studies. Critically, this compartmentalized approach can limit the ability to capture complex relationships that span di\erent analytical tasks—such as how microbial community structure influences clinical outcomes and relates to disease risk across heterogeneous cohorts. Capturing these cross-task relationships is essential for advancing a systems-level understanding of the complex interactions within microbiomes and between microbiomes and their human hosts. Generative AI and transfer learning have transformed fields such as natural language processing, by enabling the development of general-purpose foundation models that can be adapted to diverse tasks through a two-stage process: pretraining on massive unlabeled datasets, followed by fine-tuning for specific tasks. This paradigm o\ers substantial advantages in transferability, scalability, and performance. Inspired by these advances, a key question emerges for microbiome research: could a similar framework support a unified representation-learning approach for microbiome-based modeling and prediction across diverse phenotypes, rather than relying on separate, task-specific models? .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint Foundation models have already begun to reshape life-science research. A series of foundation models have been developed recently across multiple domains of biology, each pretrained on massive “languages” of biological data and then fine-tuned to specific tasks. In structural biology, AlphaFold series of models 6-8 have set a new standard by accurately predicting protein 3D structures from sequence alone, with accuracy rivaling experimental methods. More recently, RNA-focused models such as RhoFold+ 9, RNA-FM 10, and RNAErnie 11 have extended this paradigm to RNA, showing that large-scale pretraining on millions of RNA sequences enables accurate prediction of RNA 3D structure, secondary structure, and interhelical geometry directly from sequence alone. In protein modeling, large-scale generative language models have proven capable of learning the sequence– function relationships of proteins. ProGen 11, 12, for instance, was trained on over 280 million raw protein sequences and can generate functional proteins with catalytic e\iciency comparable to natural enzymes12. In single-cell transcriptomics, foundation models such as scBERT 13, Geneformer 14, and scGPT 15 have been trained on tens of millions of scRNA- seq profiles, capturing the high-dimensional structure of cellular expression data. These models illustrate the power of transfer learning, shared representations, and generative architectures to unify diverse tasks across biology. All these models operate at the level of individual molecules or cells. Yet, none address the higher-order, community-scale properties of the microbiome—highlighting the need for a microbiome-specific foundation model that unifies microbiome and clinical prediction tasks. Very recently, two microbiome-specific foundation models have been proposed 16, 17. However, both remain confined to genus-level taxonomic resolution and rely on rank-based input encodings, without leveraging the richer species-level abundance information. In this study, we sought to develop a microbiome foundation model that captures fine- grained community taxonomic structure from large-scale metagenomic data and demonstrate its utility across a range of clinical prediction tasks. To this end, we introduce BiomeGPT, a foundation model for the human microbiome—pretrained on metagenomes spanning multiple body sites—and focus here on its gut, species-level adaptation (referred to as BiomeGPT-Gut-Species). For the gut species domain adaptation, the model was further pretrained on species-level taxonomic profiles from over 13,300 human gut metagenomes spanning diverse clinical conditions. BiomeGPT was subsequently fine- tuned on multiple clinical prediction tasks, including classification of healthy vs. diseased samples and di\erentiation across 26 disease types, achieving strong predictive performance across these tasks. In addition, analysis of the model’s attention scores provides interpretable links between key microbial taxa and host phenotypes. Together, these results highlight the potential of a scalable foundation model to advance species- level gut microbiome modeling and support microbiome-based precision medicine. .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint

To evaluate the potential of a foundation model for microbiome-based diagnostics, we developed BiomeGPT, a transformer-based architecture pretrained on species-level taxonomic profiles from over 13,500 human gut metagenomes. Drawing inspiration from language models, we treated microbial abundance profiles as structured sequences and trained the model to learn context-aware representations of microbiomes through masked modeling of species abundances. This pretraining strategy enables the model to capture rich compositional patterns and cross-sample signals that are often missed by task- specific tools. We then fine-tuned BiomeGPT on a series of clinical classification tasks to predict host health status. Below, we describe the pretraining procedure and assess the model’s predictive performance across both internal and external datasets. Model pretraining on gut metagenomes Dataset overview: To train BiomeGPT, we assembled a dataset of 14,451 human metagenomic samples curated from three publicly available resources: the curatedMetagenomicData (cMD) repository 18, the Gut Microbiome Health Index (GMHI) 19, and the Gut Microbiome Wellness Index 2 (GMWI2) 20. All samples were taxonomically profiled at species-level resolution using MetaPhlAn 21, 22, ensuring consistent taxonomic assignment across the entire pretraining cohort. For longitudinal studies, only the sample from the initial time point per subject was retained. We further excluded samples with incomplete or unclassified phenotypes, as well as those with over 25% unclassified (unknown) taxa 20. After filtering, the dataset included both healthy samples and those representing 35 disease conditions from 95 independent studies. We partitioned this dataset into a training set of 13,524 samples and a validation set of 927 samples. The training set was drawn from 86 studies and encompassed 35 diseases and 2,234 unique microbial species, while the external validation set originated from 12 studies and included five diseases (Crohn’s disease, Rheumatoid Arthritis, Ankylosing Spondylitis, Parkinson’s disease, and Colorectal Cancer). To enable rigorous evaluation of generalization, all samples present in the external validation set came exclusively from studies not represented within the training set. Since the curatedMetagenomicData repository also contains non-gut (non-stool) samples, we further created a gut-specific dataset by filtering out 175 non-gut/stool samples and associated phenotypes (Mucositis, Periodontitis, Peri-implantitis, Psoriasis, Arthritis,), along with 66 associated species from the training dataset. The resulting gut-specific training dataset included 13,349 samples, 2,168 species, and 32 phenotypes (including healthy and 31 diseases). The validation dataset contained only gut/stool samples, and no further filtering was required. A breakdown of sample counts per disease for the gut-specific training and validation .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint Fig. 1 | Overview of the gut metagenomic dataset used for pretraining and external validation of BiomeGPT. Number of samples per phenotype, separated by training and validation cohorts, is shown for all phenotypes with 10 or more samples. Sample size distribution .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint datasets are provided in Fig. 1, with full study-level details available in Supplementary Table 1 and summarized in Supplementary Fig. 1. Abundance binning: To prepare species-level taxonomic profiles for pretraining, we converted continuous relative abundances (from MetaPhlAn 21, 22) into discrete abundance bins. This transformation—adapted from discretization strategies used in models such as scBERT 13 and scGPT15—captures the relative magnitude of species abundance within each sample. Specifically, species were sorted by abundance and assigned to one of the 𝑁 bins, with undetected species (i.e., those with zero relative abundance) assigned to bin 0 (see Methods). This quantile-based binning approach preserves relative scale of abundance within each sample—rather than retaining only the positional ordering of taxa in rank- based methods—enabling the model to learn compositional patterns with greater biological fidelity, while also mitigating technical batch e\ects (e.g., those arising from variations in sequencing platforms, sequencing depth, and library preparation). The resulting binned abundance matrix served as the model input for masked modeling during pretraining. Embedding strategy for microbial species and abundance: To model gut microbiomes using a transformer-based architecture, we embedded each sample as a sequence of microbial species with their binned abundance values, resulting in an ordered list of (species, abundance) pairs, referred to as “sample sequence” hereafter. This served as an ordered representation of a microbiome sample structured for transformer input, analogously to input tokens in a language model, Each microbial species was associated with a learnable embedding vector, and each discrete abundance bin was mapped to an embedding via a lightweight multilayer perceptron (MLP). These embeddings were normalized separately using layer normalization to ensure they lie in a comparable representational space and then combined additively to produce the final input token for each (species, abundance) pair (see Fig. 2A). To enable downstream classification, we introduced a special token at the beginning of each sample sequence to represent sample-level context (e.g., healthy or diseased), following the common practice in language and omics foundation models 13, 15 . This token had a dedicated learnable embedding and was always assigned an abundance bin value of 0 (Fig. 1A). The token does not represent a biological feature but serves as a placeholder for aggregating information across the entire sample through attention. .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint Transformer architecture and pretraining details: BiomeGPT employs a transformer- based architecture to learn latent representations of the gut microbiomes via masked modeling. During pretraining, we adopted a non-autoregressive masked attention strategy to accommodate the unordered nature of microbial species. In each sample, a random subset of species was masked, and the model was trained to predict their binned abundance values using information from the unmasked species only. Attention was restricted to occur only between unmasked species, and from masked species to unmasked species and to themselves (Fig. 2B), allowing the model to learn co-occurrence and abundance relationships without imposing any arbitrary sequence or positional bias. To avoid introducing noise from sparsely represented taxa, species with zero abundance were also excluded from attention and loss computation. This strategy encouraged the model to focus on the informative microbial signals present in each sample. A lightweight MLP head then maps the per-species output embeddings back to discrete abundance bins, and the model is trained to minimize mean-squared error on the masked positions. We pretrained BiomeGPT in two phases. In the first phase, the model was trained on the larger dataset of 13,524 human metagenomic samples spanning both gut and non-gut body sites. This was followed by a gut adaptation phase, in which the pretrained model was further refined on our gut-specific dataset with 13,349 samples to specialize its representations for the intestinal microbiome (see Methods for details). Unsupervised evaluation of sample-level representations learned by BiomeGPT: To assess whether BiomeGPT learns biologically meaningful structure prior to supervised fine-tuning for health status prediction, we evaluated the pretrained model’s sample-level representations using the token embeddings derived from both gut and non-gut metagenomic samples (i.e., before gut adaptation). Low-dimensional visualization using UMAP revealed a clear separation between gut and non-gut samples in the learned embedding space (Supplementary Fig. 2), indicating that body-site information is captured at the community level without explicit supervision. Consistent with this observation, simple probing classifiers trained on the embeddings achieved strong discrimination performance between gut and non-gut samples. Specifically, a linear logistic regression probe reached a mean macro accuracy of 0.925, a macro F1 score of 0.948, and a macro AUROC of 0.999 in 5-fold cross-validation evaluation. A 𝑘-nearest neighbors (𝑘NN) classifier yielded comparable performance too (mean macro accuracy = 0.897, macro F1 = 0.917, macro AUROC = 0.957). These results indicate that BiomeGPT encodes robust, body-site–specific structure in its sample-level representations, supporting the model’s ability to learn generalizable microbiome features during pretraining. .CC-BY-NC-ND 4.0 International licenseavailable under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprintthis version posted January 5, 2026. ; https://doi.org/10.64898/2026.01.05.697599doi: bioRxiv preprint SpeciesRelative Abundance SAMPLE 1 SpeciesRelative Abundance SAMPLE 2 SpeciesRelative Abundance SAMPLE N z SAMPLES SPECIESSpecies abundances (from MGX data)Binned abundance matrix z (A) Species embeddingBinned abundance zMLPAbundance embedding Z=+LayerNorm LayerNorm (species, abundance) embedding z (, 0) embedding (Species 1, Abundance) embedding (Species 2, Abundance) embedding Sample embeddings (Species, Abundance) embeddings(Species 3, Abundance) embedding. . . (B) Z= Binned Abundance Encoding and Input Embeddings Masked Attention During PretrainingMasked attention mechanism (C) Multi-head attention layers z MLP z Predicted masked abundances Species 1Species 2Species 3Species 4Species 5Species 6Sample embeddings Binned abundances02004034 z Species 1Species 2Species 3Species 4Species 5Species 6 41 20 Masked Binned Abundance Prediction z Loss function: Mean Squared Error (MSE) on masked abundance values 120345042524410310486322750123490 23 Species 1Species 2Species 3Species 4Species 5Species 6 Randomly mask a portion of species with non-zero abundance in each epoch Species 1Species 2Species 3Species 4Species 5Species 6Species 1Species 2Species 3Species 4Species 5Species 6 Binned abundance020450402234 MasedUnmasked Sample’s embedding Fig. 2 | Schematic overview of input representation, masked attention mechanism, and pretraining architecture of BiomeGPT. (A) Microbiome samples are represented as sequences of (species, binned abundance) pairs, with species and abundance embeddings combined to form input tokens. (B) During pretraining, a subset of non-zero abundance species is randomly masked, and masked attention is applied to restrict attention between unmasked tokens and from masked tokens to unmasked ones and to themselves. (C) The model predicts the binned abundances of masked species using multi-head attention layers and an output MLP , with loss computed only at masked positions. Fine-tuning for health status prediction Following pretraining, we evaluated the utility of BiomeGPT for supervised clinical inference by fine-tuning the model on a range of classification tasks designed to infer host health status from gut microbiome profiles. Specifically, we explored classification tasks including Healthy vs. Diseased and Individual Disease vs. Healthy across diverse diseases. Dataset preprocessing: Before fine-tuning: we rigorously preprocessed the gut-specific training dataset to ensure biologically meaningful input microbiome profiles across all the classification tasks. This included the removal of blacklisted species identified as environmental contaminants, and filtering out borderline species defined as those that may transiently occur in the gut but are also frequently detected as common environmental or skin-associated contaminants. These species were removed when their prevalence and abundance in gut samples were insu\icient to establish gut specificity. Finally, we applied per-phenotype abundance and prevalence thresholds to retain taxa consistently represented across samples within each phenotype (see Methods for details), The final dataset contained 496 distinct species (Supplementary File 1). These preprocessing steps was essential to reduce noise and prevent spurious signal propagation during model fine-tuning. Model architecture: For all tasks, we appended a downstream classification head to the pretrained model and applied the same model architecture. The learned sample-level token from the transformer was provided as input to this classifier (see Fig. 3A and Methods), We leveraged transfer learning to support generalization in data-limited settings. Furthermore, to address pronounced class imbalance—particularly in disease-specific tasks—we generated augmented samples for minority classes (see Methods for details). Below, we summarize the performance across the prediction tasks noted above. Distinguishing heathy vs. diseased microbiomes: To assess the capacity of BiomeGPT in distinguishing between healthy and diseased microbiomes, we fine-tuned the pretrained model on a binary classification task. In 10-fold cross-validation on the training dataset, the model achieved a mean accuracy of 0.851, a F1-score of 0.852, and an AUROC of 0.921, indicating strong internal generalization (Fig. 3B). These results suggest that the model e\ectively captures broad alterations in microbial composition associated with disease status. Fig. 3 | Fine-tuning BiomeGPT for health status prediction tasks. (A) For all downstream tasks, a classification head was appended to the pretrained model, using the learned sample-level token from the transformer as input. (B) Performance metrics for BiomeGPT in healthy vs diseased and specific-disease vs healthy classification across 26 diseases with ≥ 20 samples (including Crohn’s disease and Ulcerative Colitis, separated from the broader IBD category), evaluated by 10-fold cross-validation. (C) BiomeGPT on healthy vs diseased and specific disease vs healthy prediction for five diseases in external the validation dataset and comparison with XGBoost and Random Forest. (A) Multi-head attention layers z MLP classifierSamples embeddings z Fine-tunning model configuration 0.985 0.999 0.985 0.999 0.996 1.000 00.20.40.60.81 MelanomaPremature Born Other Diseases 0.998 1.000 1.000 0.998 1.000 1.000 0.999 1.000 1.000 00.20.40.60.81 Soil-Transmitted Helminths Clostridioides Difficile InfectionAcute Diarrhea Infectious and Parasitic Diseases 0.994 0.996 0.994 0.996 0.998 0.998 00.20.40.60.81 Parkinson’s DiseaseSchizophrenia Neurological and Neuroinflammatory Diseases 0.974 1.000 1.000 1.000 0.974 1.000 1.000 1.000 0.993 1.000 1.000 1.000 00.20.40.60.81 Colorectal CancerColorectal AdenomaPancreatic CancerLiver Cirrhosis GI and Hepatic Cancers/Disorders 0.988 0.993 0.998 0.988 0.993 0.998 0.997 0.997 0.999 00.20.40.60.81 Coronary Artery Disease Atherosclerotic Cardiovascular…Heart Failure Cardiometabolic Diseases 0.959 0.989 1.000 0.960 0.989 1.000 0.989 0.997 1.000 00.20.40.60.81 Type 2 Diabetes Impaired Glucose ToleranceNonalcoholic Fatty Liver Disease Metabolic Disorders 0.976 0.998 0.997 0.985 0.995 0.996 0.999 0.999 0.998 0.976 0.998 0.997 0.985 0.996 0.996 0.999 0.999 0.998 0.993 0.999 0.999 0.996 0.999 0.998 0.999 1.000 0.999 00.20.40.60.81 Inflammatory Bowel DiseaseCrohn's DiseaseUlcerative ColitisMultiple SclerosisRheumatoid ArthritisAnkylosing SpondylitisGraves’ DiseaseBehcet’s DiseaseMyalgic Encephalomyelitis /… Immune-Mediated Diseases BiomeGPTXGBoostRandom Forest(B) (C)AccuracyF1AUROC10-fold Cross-ValidationExternal-Validation MyalgicEncephalomyelitis/ Chronic Fatigue Syndrome 0.854 0.688 0.911 0.670 0.651 0.871 0.809 0.645 0.893 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCRheumatoid Arthritis 0.940 0.612 0.952 0.905 0.602 0.958 0.883 0.560 0.940 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCCrohn’s Disease 0.799 0.760 0.901 0.632 0.646 0.858 0.800 0.739 0.908 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCAnkylosing Spondylitis 0.512 0.502 0.438 0.437 0.453 0.486 0.460 0.448 0.536 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCParkinson’s Disease 0.835 0.745 0.897 0.805 0.710 0.893 0.798 0.665 0.838 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCColorectal Cancer Atherosclerotic Cardiovascular Disease z z 0.762 0.702 0.897 0.712 0.634 0.895 0.711 0.633 0.860 0.00.20.40.60.81.0 Macro AccuracyMacro F1Macro AUROCHealthy vs. Diseased 0.9210.8520.851 0.00.20.40.60.81.0 Healthy vs. DiseasedHealthy vs. Diseased Healthyvs.DiseasedHealthyvs.Disease XHost health status and disease state prediction Predicting disease type from microbiome profiles across diverse clinical conditions: We next sought to evaluate the model’s capacity to resolve finer-grained diagnostic classes by predicting specific disease types from gut microbiome taxonomic composition. This e\ort was designed to emulate real-world diagnostic uncertainty—where disease status is unknown and may correspond to any one of many possible conditions—and to test the model’s ability to support di\erential diagnosis from microbiome data. To this end, we fine-tuned the model to distinguish specific diseases from healthy individuals. This enabled a comprehensive evaluation of the model’s capacity to resolve a broad spectrum of clinical phenotypes. We conducted this analysis for 26 diseases with ≥ 20 samples and trained 26 binary classifiers to di\erentiate individual diseases from healthy samples. BiomeGPT achieved high predictive performance across all disease types, with accuracy, F1-scores, and AUROCs all exceeding 0.970 in 10-fold cross-validation, except for type 2 diabetes (T2D), where accuracy and F1-score were 0.959 and 0.960, respectively (Fig. 3B). This underscores the model’s capacity to detect intricate disease-specific alterations in microbial composition. External validation: To evaluate the generalizability of BiomeGPT beyond the studies seen during pretraining and fine-tuning, we performed external validation using an independent dataset comprising 927 samples from studies not included during model pretraining. For this evaluation, the model architecture and pretrained weights remained unchanged; however, the fine-tuning step was repeated using the full training dataset (rather than performing fine-tuning separately within each cross-validation fold). For the healthy vs. diseased classification task, the model maintained strong predictive performance, achieving a macro accuracy of 0.762, a macro F1-score of 0.702, and a macro AUROC of 0.897 (Fig. 3C). These results demonstrate that BiomeGPT generalizes well to heterogeneous clinical studies and is robust to cohort-specific variation in microbiome composition for this prediction task. For the disease type prediction tasks—Specific Disease vs. Healthy—BiomeGPT demonstrated strong generalizability for several conditions, achieving notably high performance for Rheumatoid Arthritis, Crohn’s Disease, Ankylosing Spondylitis, and Colorectal Cancer (Fig. 3C). Macro accuracies for these diseases ranged from 0.799 to 0.940, with corresponding macro F1-scores between 0.612 and 0.760 and macro AUROC between 0.897 and 0.952, indicating reliable discrimination of case–control labels in unseen datasets. In contrast, performance for Parkinson’s Disease was markedly lower (macro accuracy = 0.512, macro F1 = 0.502; macro AUROC = 0.438), reflecting substantial distributional divergence between the external Parkinson’s Disease samples and those present in the training dataset. Comparison to baseline methods: To contextualize the performance of BiomeGPT, we benchmarked its fine-tuned classification results against a range of baseline methods. First, to assess the added value of BiomeGPT relative to classical machine learning (ML) approaches, we benchmarked its performance against Random Forest and XGBoost using the same input features and identical external validation cohorts. In 10-fold cross-validation, all three approaches performed comparably across healthy vs. diseased and disease-specific classification tasks, indicating that the strong in-distribution performance of BiomeGPT is consistent with classical ML models (see Supplementary File 2). However, BiomeGPT demonstrated consistently stronger generalization to external cohorts. For healthy vs. diseased classification, BiomeGPT achieved higher macro accuracy (0.762) and macro F1-score (0.702) than both XGBoost (0.712 and 0.634) and Random Forest (0.711 and 0.633), with similarly strong and lightly higher AUROC. A similar pattern was observed across disease type predictions on the external cohort: BiomeGPT exceeded or matched the baseline models in Rheumatoid Arthritis, Crohn’s Disease, Ankylosing Spondylitis, and Colorectal Cancer, and performed stronger in macro accuracy and macro F1-score for Parkinson’s Disease despite substantial cohort divergence. These results demonstrate BiomeGPT’s ability to adapt to several heterogeneous disease contexts while also highlighting phenotypes for which external cohort alignment may be more challenging. Next, we compared BiomeGPT to existing rank-based microbiome foundation models, including and that from Pope et. al 16 and MGM 17, both of which were pretrained on large-scale microbiome datasets using taxonomic rank representations. BiomeGPT achieved superior performance on the same prediction tasks across internal and external evaluations. Specifically, Pope et. al 16 reported an AUROC of 0.687 while MGM recorded an AUROC of 0.829 and 0.844 17 for classifying IBD versus healthy microbiomes. In comparison, BiomeGPT achieved AUROCs of 0.993, 0.999, and 0.999 for broader IBD, Crohn’s Disease (CD), and Ulcerative Colitis (UC) vs. healthy classification, respectively, in 10-fold cross-validation. Additionally, for external validation, the model achieved an AUROC of 0.944 and a macro AUROC of 0.952 for CD vs. healthy classification (Fig. 3). Together, these comparisons highlight the benefit of BiomeGPT’s foundation model design, which leverages large-scale pretraining on species-level profiles and abundance-aware embedding strategy to enable robust performance in clinical microbiome prediction tasks. Microbial biomarkers driving health and disease prediction: insights from BiomeGPT attention analysis: To identify microbial species that BiomeGPT deemed most influential for health and disease prediction, we analyzed attention scores from the Healthy vs. Diseased and Specific Disease vs. Healthy fine-tuning tasks. For each task, we computed the mean attention each species received from the token across samples, attention heads, and layers in the model trained on the entire training dataset. To facilitate interpretation, we aggregated the results for five major categories—immune-mediated diseases, GI and hepatic cancers/disorders (non-autoimmune), metabolic disorders, cardiometabolic diseases, and neurological and neuroinflammatory disorders (non-autoimmune). Complete attention analysis results for each specific disease can be found in Supplementary File 3. The top species with the highest attention scores identified for Healthy vs. Diseased and within each disease category for Specific Disease vs. Healthy classification are summarized in Fig. 4. The highest-attention species for Healthy vs. Diseased classification included species that are concentrated within specific disease phenotypes— such as Vibrio cholerae and Raoultella sp T31, both enriched in Acute Diarrhea—as well as those frequently observed in healthy or early-life (preterm born) contexts (e.g., Staphylococcus epidermidis, Mixta calida, Cutibacterium avidum, and Staphylococcus hominis). Classic gut commensals were less represented among the high-attended taxa, although Bifidobacterium angulatum exhibited high attention and broad presence across multiple phenotypes. These results indicate that attention is not solely driven by prevalence. For example, infrequent taxa with phenotype-restricted enrichment, such as V. c h o l e ra e s and Zymomonas mobilis, receive high attention due to their concentrated occurrence in certain diseases and consequently their strong discriminative signal for these diseases. Conversely, broadly represented commensals such as Bifidobacterium angulatum and Bifidobacterium breve (ranked 24th; not shown in the figure; see Supplementary File 3) also show high attention despite wide distribution across phenotypes, indicating that BiomeGPT prioritizes species based on their relevance for distinguishing health states, rather than on prevalence or abundance alone. At the disease-category level, analysis of the species receiving high attention from the token (Fig. 4) shows that BiomeGPT draws on both a shared set of broadly informative taxa, primarily enriched in the Healthy state, and a complementary group of disease category-specific signals. Notably, a small group of species—S. epidermidis, S. hominis, Candida albicans, Staphylococcus aureus, C. avidum, B. breve, Haemophilus haemolyticus, Enterococcus faecalis, and Klebsiella oxytoca—recur as high-attention species across all five categories. Several of these species are strongly enriched in Healthy and early-life samples: for example, S. epidermidis shows non-zero abundance in 556 samples, of which 515 are Healthy and 23 are Premature Born, while S. hominis and C. avidum are present predominately in Healthy samples (250 and 77 samples, respectively). In Specific Disease vs. Healthy fine-tuning tasks, BiomeGPT can therefore reuse these taxa as markers of healthy microbiome across many distinct diseases (other than premature-born), which explains their frequent selection among the top-10 attended species within and across disease categories. By contrast, taxa such as E. faecalis, B. breve, and K. oxytoca exhibit broad presence across Healthy and multiple disease phenotypes. For example, E. faecalis appears in 783 Healthy plus numerous samples from cardiometabolic, metabolic, oncologic, autoimmune, and neurological conditions. The high attention to these species therefore likely reflects not simple prevalence, but rather the phenotype-specific shifts in their abundance patterns captured in the model’s embedding space. This allows these widely distributed taxa to encode variation in host–microbiome state rather than single-disease specificity. Beyond this shared core, there are taxa whose high attention is restricted to specific disease categories or specific diseases within those categories. Examples of these species include Clostridium spiroforme, Lactococcus virus bIL67, and Clostridioides diEicile, in immune-mediated diseases; Clostridium aldenense, Hafnia alvei, and Hungatella hathewayi in metabolic disorders; Enterobacter aerogenes, Clostridium perfringens, and Roseburia faecis in GI and hepatic cancers/disorders; and Flavonifractor plautii and Actinomyces oris in neurological and neuroinflammatory disorders. The presence of these category-restricted taxa—alongside more broadly informative species—indicates that BiomeGPT integrates both cross-cutting and disease-specific microbial signals during classification, reflecting context-dependent axes of host–microbiome variation. Relationship between model attention and diPerential abundance: To assess whether BiomeGPT’s attention scores primarily reflect classical di\erential abundance signals, we examined the correlation between species-level attention and log-fold changes in relative abundance across all fine-tuning tasks (Supplementary File 4). For each task (Healthy vs. Diseased and each Specific Disease vs. Healthy comparison), we computed the log2 fold change in mean abundance between disease and healthy groups for all 496 species and then calculated the Spearman correlation coe\icient between the absolute value oflog2 fold change| and the mean attention score from the token. Across tasks, we observed modest (Spearman 𝜌 ≥ 0.40) but statistically significant (FDR-adjusted p < 0.05) positive correlations in a number of individual Disease vs. Healthy classification tasks including Impaired Glucose Tolerance (Spearman 𝜌 = 0.46), Type 2 Diabetes (𝜌 = 0.44), Coronary Artery Disease (𝜌 = 0.44), Multiple Sclerosis (𝜌 = 0.43), Melanoma (𝜌 = 0.40), and Atherosclerotic Cardiovascular Disease (𝜌 = 0.40). This indicates that species undergoing larger abundance shifts tend, on average, to receive higher attention from the model. However, for many disease-specific comparisons, correlations were weak (0.1 ≤ Spearman 𝜌 < 0.40), near zero (i.e., uncorrelated: 𝜌 < 0.1, e.g., Liver Cirrhosis: 𝜌 = 0.05), negative (e.g., Premature Born: 𝜌 = - 0.05), or nonsignificant (e.g., Premature Born), suggesting that BiomeGPT does not rely exclusively on species with large univariate abundance changes. Instead, BiomeGPT integrates both di\erential abundance and more complex, multivariate relationships among taxa when forming its internal representations of microbiome states. Fig. 4 | Species most emphasized by BiomeGPT during fine-tuning classification tasks. Top-ranked taxa by model attention in the Healthy vs. Diseased as well as Specific Disease vs. Healthy fine-tuning tasks. Only five major disease categories for disease type prediction tasks are shown. Values show the mean attention each species receives from the token, averaged across all transformer heads, layers, and samples in which that species has non-zero abundance. High-attention species are ranked in the heatmaps by coverage—that is, by the number of diseases in the disease category in which the species appears among the top-10—in descending order. Only the 15 species with the highest coverage from the union of species appearing in the top 10 high-attention species are shown. If there are fewer than 15 unique species in the union, additional species that receive high attention but do not appear in the top 10 list are included. Asterisks show if a species is among the top 10 high-attention list for a given phenotype for Specific Disease vs. Healthy classification tasks. Species that do not meet the prevalence and abundance thresholds within at least one of the two groups compared were filtered out. Healthy vs. DiseasedImmune-mediated diseasesMetabolic disorders Cardiometabolic diseasesGI and hepatic cancers/disordersNeurological/Neuroinflammatory disorders

The growing complexity and scale of microbiome datasets have outpaced the capabilities of traditional analysis pipelines, which remain largely scattered and optimized for a narrow range of task-specific applications. In this study, we introduced BiomeGPT, a transformer-based foundation model trained on over 13,300 species-level human gut metagenomic profiles, o\ering a generalizable framework for microbiome analysis such as microbiome-based host health status prediction. The model demonstrated robust classification performance for distinguishing Healthy vs. Diseased samples and across 26 disease states, including immune-mediated, gastrointestinal, metabolic, cardiometabolic, neurological, and oncological conditions, highlighting its adaptability across a broad clinical spectrum. Beyond predictive accuracy, BiomeGPT’s attention-based architecture enabled interpretable insights into microbe-disease associations across these disease categories. These findings position BiomeGPT as a foundational advance toward scalable, interpretable species-level microbiome analysis and prediction, and provide a proof-of-concept that transformer-based pretraining on large gut metagenomic cohorts can learn transferable, context-aware microbiome representations. Recent e\orts to apply foundation modeling to microbiome data have begun to explore the potential of transfer learning in this domain, yet existing approaches remain constrained in both resolution and design. Notably, recent models such as that proposed by Pope et. al 16 and MGM 17, while both represent important and promising initial steps toward applying large-scale pretraining in the microbiome domain, have relied on genus-level taxonomic inputs and rank-based encodings. Although computationally tractable, these models do not fully capture the granularity of microbial community structure at the species level. Furthermore, these rank-based representations may discard critical quantitative abundance information, limiting the model’s ability to capture biologically meaningful abundance gradients across taxa. Additionally, while both models demonstrated cross-cohort utility in selected settings, their broader generalizability and adaptability across heterogeneous disease landscapes remain to be fully explored. BiomeGPT addresses these limitations through a combination of architectural and representational advances. By operating directly on species-level abundance profiles and embedding discrete abundance bins rather than taxonomic ranks, the model preserves both taxonomic resolution and quantitative compositional data. Its masked modeling approach during pretraining, adapted from language models, enables the model to learn cross-species relationships by inferring co-occurrence and context-aware abundance patterns. These architectural features contribute to BiomeGPT’s strong predictive performance across a range of evaluation settings, with consistently high scores in cross-validation and competitive results on external datasets for both health status and disease type prediction tasks. These advances establish BiomeGPT as a more expressive, generalizable, and biologically attuned modeling framework than its predecessors. Interestingly, our cross-validation revealed that classification tasks targeting individual disease labels tend to outperform the Healthy vs. Diseased classification task (Fig. 3B). A similar pattern was observed in our external validation analysis (except for Parkinson’s disease; Fig. 3C). This observation suggests that BiomeGPT is particularly e\ective at capturing disease-specific microbial signatures rather than generic markers of dysbiosis. One contributing factor may be the substantial heterogeneity within our healthy cohort, which in our training dataset includes samples from 61distinct studies (Supplementary Fig. 1)—far more than any single disease class. This diversity likely introduces greater intra-class variance among both healthy and diseased samples, complicating the learning of a consistent representation of health. In addition, the aggregated “Diseased” group encompasses a wide array of distinct conditions from 57 distinct studies (Supplementary Fig. 1), each with its own microbial alterations, which results in a di\use and overlapping structure in the embedding space, further reducing the model’s ability to cleanly separate healthy from diseased states. In contrast, individual diseases often present more internally consistent microbial patterns for one or a limited number of studies, which likely makes them easier for the model to distinguish from healthy samples. These findings highlight the model’s strength in learning disease-specific representations while pointing to the importance of accounting for heterogeneity in both training and evaluation. In addition to in-distribution performance, our external validation analyses further highlight the advantages of BiomeGPT over traditional ML approaches. While Random Forest and XGBoost classifiers performed competitively within cross-validated training cohorts (see Supplementary File 2), their performance declined appreciably when evaluated on independent studies, particularly in macro F1 scores, reflecting reduced robustness to cohort-specific variation—a pervasive challenge in microbiome research and the broader ML area. In contrast, BiomeGPT maintained stronger and more stable predictive accuracy across multiple external disease cohorts, indicating its ability to generalize across heterogeneous microbiome profiles. At the same time, certain phenotypes, such as Parkinson’s disease, exhibited reduced cross-cohort transferability across all models, suggesting that cross-cohort alignment and biological heterogeneity remain critical factors shaping external transferability. These results indicate that although conventional classifiers can su\ice in controlled, in-distribution settings, transformer-based foundation models such as BiomeGPT o\er superior resilience to dataset shifts, a key requirement for reliable deployment in clinical or multi-cohort translational applications. Beyond predictive performance, a major strength of BiomeGPT lies in its interpretability, which enables deeper biological insight into the microbiome-disease axis. The attention patterns learned for the Healthy vs. Diseased classification task recapitulate plausible disease-linked and health-associated commensal microbial signals (Fig. 4). For example, the high attention ranking of V. cholerae aligns with its concentrated occurrence in acute diarrhea within our dataset. Notably, prior studies have documented that V. cholera infection induces reproducible ecological perturbations in the gut microbiota, including transient blooms of Proteobacteria and disruptions of commensal taxa 24, 25. Similarly, elevated attention to Proteobacteria such as Haemophilus influenzae reflects the well-documented surges of Proteobacteria characteristic of inflammatory and diarrheal states 26, 27. Conversely, skin- and oral-associated taxa—including S. epidermidis and C. avidum—are emphasized primarily in healthy samples within our training dataset, but also appear in other phenotypes such as preterm infants, perhaps due to early-life body-site microbiome exchange 28. In parallel, some established commensal gut species such as B. angulatum and B. longum appear with high attention scores (ranked 10th and 24th, respectively). These findings indicate that BiomeGPT leverages attention to encode both disease-associated microbial spikes and core commensal species signatures of a healthy gut ecosystem in distinguishing healthy from diseased states. Attention patterns from specific Disease vs. Healthy classification tasks o\er insight into how the large pretrained foundation model adapts to distinct pathological contexts. BiomeGPT consistently identified a core group of broadly informative species—many enriched in healthy or early-life microbiomes—while also elevating species whose importance is largely confined to specific diseases or disease categories (Fig. 4). This balance between shared and disease-restricted signals suggests that the model internalizes both global axes of microbial community variation and localized, context-dependent microbial shifts. Notably, widely distributed taxa such as E. faecalis, B. breve, and K. oxytoca appear to contribute not through simple presence or abundance, but through intricate, phenotype-specific patterns captured in the embedding space. This is consistent with biological evidence supporting their roles as context-dependent pathobionts—highlighting the advantage of representation learning in resolving subtle but biologically relevant variation across phenotypes. For example, E. faecalis, in addition to healthy samples, is also enriched in our IBD samples. This species has been reported before to induce IBD as well as rectal dysplasia and adenocarcinoma in IL-10–deficient mice 29. Likewise, K. oxytoca is enriched in Crohn’s Disease samples in our dataset in addition to healthy samples. Klebsiella species have been linked to Crohn’s disease and K. oxytoca is a known cause of antibiotic-associated hemorrhagic colitis 30, 31. These observations support the notion that BiomeGPT assigns high attention to pathobionts with established roles in intestinal inflammation and mucosal injury, even when they are not confined to a single phenotype. Additionally, the emergence of species receiving high attention scores by the model only within specific diseases or disease categories demonstrates the model’s capacity to capture disease-linked microbial signals in the training dataset. As an example, C. diEicile emerges as a high-attention species predominantly within immune-mediated disease contexts—especially in IBD—aligning with clinical data showing that IBD and related conditions confer a markedly elevated risk of C. diEicile infection (CDI) and that CDI–IBD co-infection is associated with more severe flares and worse clinical outcomes 32. These attention-derived insights highlight BiomeGPT’s ability to disentangle both shared and disease-specific microbiome signatures across diverse clinical phenotypes, reinforcing the biological interpretability of its predictions. The correlation analysis between attention scores and di\erential abundance provides additional insight into how BiomeGPT extracts microbiome features across phenotypes. The modest positive correlations observed in Healthy vs. Diseased and multiple disease-specific classification tasks indicate that the model is partially sensitive to classical di\erential abundance signals: species that undergo larger shifts between healthy and diseased states often receive elevated attention, aligning BiomeGPT with established microbiome analysis paradigms. However, the absence of strong correlations as well as weak, near zero (uncorrelated), or even negative correlations across several other disease-specific tasks argue against a simplistic interpretation that attention merely recapitulates univariate fold changes in abundance. Rather, these patterns support that BiomeGPT encodes higher-order, multivariate structure in the microbiome—such as co-occurrence patterns, compositional gradients, and shared dysbiosis axes—that extend beyond the scope of traditional di\erential abundance frameworks. This behavior is desirable in a foundation model: it demonstrates that the model learns a richer, more context-dependent representation of host–microbiome variation through its embedding space beyond what can be explained by marginal abundance shifts alone, thereby providing complementary insight to conventional di\erential abundance analyses instead of duplicating them. This representational richness encoded within this foundation model is essential not only for biological interpretability but also for enabling clinically meaningful applications. While pretraining allows BiomeGPT to learn context-aware embeddings of microbiome composition, its practical value lies in its adaptability to clinically relevant tasks. It provides a versatile platform for translating microbiome insights into clinical applications. Its strong performance in distinguishing healthy from diseased states, combined with its ability to resolve specific disease types, suggests it could serve as a non-invasive, high-throughput stool-based assay for early detection or monitoring of microbiome-associated conditions. Moreover, the model’s self-attention scores uncover key microbial taxa linked to host health and disease state classification, o\ering a data-driven route to biomarker discovery that both complements and often surpasses traditional di\erential-abundance methods. These attention-derived “priority species” highlight taxa whose roles in health and disease warrant further mechanistic investigation. Finally, because BiomeGPT is pretrained on thousands of diverse gut microbiome samples—thereby encoding rich, disease-relevant structure—it can be rapidly fine-tuned to new clinical endpoints, such as predicting treatment response or stratifying patients by disease risk, with only modest additional labeled data. In this way, BiomeGPT o\ers a reusable foundation model that accelerates the development of tailored microbiome-based tools in translational research. While BiomeGPT establishes a strong foundation for species-level modeling of the gut microbiome, a number of limitations merit consideration. This study focused exclusively on species-level taxonomic profiles derived from metagenomic data, without yet integrating functional and strain-level features that could o\er complementary layers of insights. Additionally, we observed decline in performance in external evaluations compared to cross-validations. However, this challenge is not unique to our approach; it rather reflects a broader limitation of ML/AI models when applied to unseen cohorts. Such performance variation underscores the importance of assessing how well a new dataset aligns with the training distribution before applying the model to external cohorts. Our evaluation of diseases that showed significant reduced performance on external validation—such as Parkinson’s Disease—using t-SNE visualizations indicates that these external datasets are indeed poorly aligned with their corresponding training data (see Supplementary Fig. 3). Looking ahead, extending this framework to capture other microbiome features beyond species-level taxonomic profiles will be an important next step. More broadly, BiomeGPT represents a foundational step toward more scalable and generalizable approaches to microbiome modeling. Its architecture is designed to flexibly accommodate a range of phenotypes and contexts, o\ering interpretable outputs that highlight microbial taxa associated with model predictions and may reflect biologically meaningful patterns. Furthermore, the robustness of the representations learned by BiomeGPT is reinforced by its two-phase pretraining strategy, which initially leverages a diverse catalogue of metagenomes spanning both gut and non-gut body sites before gut-specific adaptation. Consistent with this design, unsupervised evaluation of the learned sample-level representations shows that gut and non-gut microbiome samples occupy distinct regions of the embedding space (Supplementary Fig. 2), indicating that the pretrained model captures broad body-site structure at the community level even without explicit supervision and fine-tuning. This hierarchical learning process and first learning from the broader microbial landscape, allows the model to capture fundamental species co-occurrence patterns and abundance dependencies that are not tied to a single anatomical niche. Although the present study focuses on gut-associated phenotypes, this broad pretraining foundation suggests that the pretrained model captures cross-body-site structure in its species-level microbiome embeddings. This provides a natural starting point for extending the framework to microbiomes from multiple body sites, or integrations thereof, within a single computational model in future work. Likewise, while this study focused on a defined set of conditions, the same framework can be extended to additional diseases not included in this study as well as more complex clinical endpoints such as treatment response and longitudinal health trajectories. As foundation models continue to reshape the landscape of biology and biomedical AI, BiomeGPT demonstrates how such models can distill complex, high-dimensional microbiome data into actionable insights. By grounding future analyses within a unified representation space, this approach has the potential to reduce methodological compartmentalization, enhance reproducibility, and accelerate the translation of microbiome science into clinical impact. Methods Taxonomic binning strategy: Prior to model training, we discretized species-level relative abundances into a sample-by-species matrix 𝐴, where rows represent samples and columns correspond to microbial species. For abundance binning, species with non-zero abundance within each sample were sorted in descending order and divided evenly into 𝐵 bins. The most abundant 𝑁/𝐵 species (where 𝑁 is the number of species with non-zero relative abundance in the sample) were assigned to bin 𝐵, the next 𝑁/𝐵 to bin 𝐵−1, and so on. Species not detected in each sample (those with zero relative abundance) were assigned a bin value of 0. For samples with fewer than 𝐵 non-zero abundance species, species were distributed proportionally across bins 1 through 𝐵 to ensure that low-diversity samples still spanned the full binned range. This avoided overconcentration of values in the highest bins, which could otherwise lead to underutilization of the lower bins during training and limit the model’s ability to generalize across samples with varying species diversity. Input embeddings and sample representation: For each metagenomic sample, we constructed an input sequence of microbial species paired with their binned abundance values. Let 𝐴 denote the sample-by-species matrix, where each entry 𝑎!"∈{1,2,…,𝐵} represents the binned abundance of species 𝑗 in sample 𝑖. Each microbial species 𝑗 was mapped to a learnable embedding vector via a PyTorch embedding layer, denoted 𝑒𝑚𝑏#$%&'%#". To embed the abundance bin values 𝑎!", we used a shallow MLP with ReLU activation, denoted by 𝑒𝑚𝑏()*+,(+&%(𝑎!"). The species and abundance embeddings were independently normalized using layer normalization. The final input token embedding for species 𝑗 in sample 𝑖 was computed as: 𝑡!"=LayerNormC𝑒𝑚𝑏#$%&'%#"D+LayerNormC𝑒𝑚𝑏()*+,(+&%F𝑎!"GD. (1) To produce a sample-level representation for downstream prediction tasks, we prepended a special token to each sample sequence. This token had its own learnable species embedding 𝑒𝑚𝑏#$%&'%#() and was paired with an abundance bin value of 0. The resulting input embedding for was defined as follows (Fig. 1A): 𝑡!,./012=LayerNormF𝑒𝑚𝑏#$%&'%#./012G+LayerNormF𝑒𝑚𝑏()*+,(+&%(0)G. (2) The full sample sequence, consisting of the token followed by all microbial species with their binned abundances, {(,0),(species3,𝑎3!),(species4,𝑎4!),…}, was used to generate the corresponding embedding sequence U𝑡!,.&5#2 ,𝑡!3,𝑡!4,…V, which was then passed as input to the transformer model. Notably, species within each sample were presented in an arbitrary order determined by the set of non-zero species for that sample. No positional embeddings were applied, ensuring permutation invariance with respect to species order. Transformer configuration and masked pretraining procedure: BiomeGPT was implemented using the TransformerEncoderLayer module from PyTorch. The model architecture consisted of 8 stacked transformer layers, each containing 8 attention heads and a feedforward sublayer with a hidden dimension of 512. The input to the transformer was the embedded sequence for each sample. Let 𝑡!(7) denote the input embedding sequence for sample 𝑖. The transformer computed updated representations layer-by-layer as: 𝑡!(0)=TransformerBlockC𝑡!(093)D,for 𝑙=1,2,…,𝐿. (3) After the final layer, the output at the position, 𝑡!,./012(:), was extracted as the sample-level embedding. This embedding was passed to a task-specific prediction head during fine-tuning. Pretraining was performed using a masked modeling objective. At each epoch, 25% of the non-zero-abundance species in each sample were randomly selected for masking, and the masked subset was selected independently across samples and epochs. Attention was computed only between unmasked species and between masked species and unmasked species; masked species could not attend to each other but were allowed to attend to themselves (Fig. 2). After the transformer stack, each token’s final-layer embedding is passed through a three-layer MLP (hidden dimension = 512, ReLU activations) that outputs one value per abundance bin. The model is trained to reconstruct the true bins of masked species by minimizing mean-squared error (MSE) between the MLP’s predicted bin values and the ground-truth bins—computed only over masked positions and excluding zero-abundance species. We used an embedding dimension of 512 and trained the model for 30 epochs on a single NVIDIA A100 GPU, accessed through a Google Colaboratory notebook, using a batch size of 64. The pretraining dataset (containing 13,524 samples from both the gut and other body sites) was split 90–10 into training and validation subsets for convergence monitoring. After training for 30 epochs, the model went through a domain adaptation step where it was trained for an additional 10 epochs on the gut-specific dataset with 13,349 samples. Data preprocessing pipeline applied before fine-tuning: Prior to fine-tuning, all species-level abundance profiles in the gut-specific training dataset underwent systematic preprocessing to enhance gut specificity and reduce noise. First, we excluded species appearing in a predefined blacklist containing 22 microbial species known to be exclusively non-gut or environmental species, representing potential contaminants or sequencing artifacts (see Supplementary File 1 for the full list). Second, we applied abundance and prevalence filtering on a per-phenotype basis to ensure that retained species were robustly represented within at least one phenotype. Specifically, a species was retained if its relative abundance exceeded 0.1% in at least 3% of samples in one or more phenotypes. The 3% prevalence cuto\ was chosen after empirical evaluation of several thresholds, with 3% providing an optimal balance between retaining core gut taxa and excluding low-prevalence or non-gut/spurious species. Finally, we filtered borderline species, defined as those belonging to genera that may be occasionally present in the gut transiently or during disease but are also common contaminants originating from skin, oral, or environmental sources (see Supplementary File 1 for the list of these genera). For each of these genera, all corresponding species detected in our dataset were evaluated across phenotypes (healthy and specific disease). A borderline species was retained only if it appeared with a relative abundance of ≥0.5% in ≥1% of samples within at least one phenotype. The higher abundance threshold for borderline taxa reflects their potential contaminant origin, ensuring that only consistently abundant, gut-associated representatives were retained. This multi-tiered filtering strategy minimized contamination and sparsity while preserving biologically meaningful taxa for fine-tuning classification tasks. Fine-tuning model architecture: For all the fine-tuning tasks, the same transformer backbone used in pretraining was retained, and an MLP classification head was appended to it. The learned sample-level token embedding from the final transformer layer served as input to this classifier. During fine-tuning, all weights in the transformer backbone were initialized from the pretrained BiomeGPT model. The input to the transformer consisted of full species abundance embeddings without masking, allowing all tokens to attend to each other. The classifier output the probability of each disease or health status. Addressing class imbalance for fine-tunning tasks: In binary disease prediction tasks—especially for Specific Disease vs. Healthy—we frequently encountered class imbalance. To address this imbalance, we applied sample-level augmentation to the minority class using Gaussian noise injection. Specifically, for each minority-class sample, we generated an augmented sample by adding a noise vector, 𝜖~𝑁(𝜇,𝜎), drawn from a Gaussian distribution with mean 𝜇=0 and standard deviation 𝜎=0.1 for cross-validation and 5 for external validation studies. Noise was applied elementwise to only the non-zero entries of the sample abundance vector, preserving the original sparsity and compositional structure. The resulting augmented sample was then modified using the clip() function of numpy to replace negative values with 0. The standard deviation of 𝜎=5 was selected empirically by progressively increasing 𝜎 from 0.1 and visualizing the augmented and original samples via t-SNE projections, choosing the largest value that preserved distinct separation between classes without overlap. The final training set was constructed by combining the original majority-class samples, the minority-class samples, and the augmented minority-class set. The number of augmented samples was chosen such that the new size of the minority class matched the size of the majority class. Transfer learning for diseases with limited training samples: For disease-type prediction tasks with limited sample sizes, we employed an additional layer of transfer learning to enhance model generalization. Diseases were grouped into eight clinically related categories (see Fig. 3 and Supplementary File 1), and the largest disease dataset in each group was used to first train a model for that disease. For other diseases in the group, model weights were initialized from the trained model for the most prevalent disease rather than from the base pretrained backbone, allowing parameter reuse from phenotypically related tasks. Performance evaluation: Fine-tuning performance was evaluated using 10-fold cross-validation on the internal training dataset, with each fold consisting of a 90/10 training/validation split. External validation was performed using a held-out dataset of 927 samples from studies excluded during pretraining. During cross-validation, since class imbalance was addressed through sample augmentation (see above), accuracy, F1-score, and area under the receiver operating characteristic curve (AUROC) were calculated using the predicted vs. true labels for the positive class. For external validation, where class imbalance was preserved in the external cohort, both standard and macro-averaged metrics were recorded. Macro metrics were computed by evaluating each class (positive and negative) independently and averaging the resulting scores, ensuring a balanced assessment that does not favor the majority class. All metrics were computed using the torchmetrics library (v1.3.0) of PyTorch. Specific functions included Accuracy, F1Score, and AUROC, with task=“multiclass” and average= "macro" specified for macro-averaged calculations. Attention analysis: To identify species with high model-assigned importance for each fine-tuning task, we computed the mean attention received by each microbial species from the token. Let 𝐴𝑡𝑡𝑛!,.&5#2→"(<0) denote the attention from the token to species 𝑗 in sample 𝑖 (i.e., from token 𝑡!.&5#2= to token 𝑡!"), head ℎ, and layer 𝑙. For each species 𝑗, we define, 𝐴𝑡𝑡𝑛dddddd.&5#2→", its aggregated attention score across 𝑁" samples where it is present with non-zero abundance as: 𝐴𝑡𝑡𝑛dddddd.&5#2→"=164𝑁" ggg𝐴𝑡𝑡𝑛!,./012→"( 0?3 @! !?3 ∀𝑗∈{Species}. (4) Attention-abundance correlation analyses: To assess whether higher attention aligns with larger abundance fold changes, we computed log₂ fold-change in abundance for every species per task as log4kA̅"#$%&$%C37'(A̅)%&*+,-C37'(l, where 𝑥̅DE0FG<H and 𝑥̅I!1EF1E are mean relative abundance within healthy and disease groups and10⁻⁶ is to prevent division by zero. We then calculated Spearman’s rank correlation (scipy.stats.spearmanr) between each species’ mean attention and absolute value of log₂ fold-change. Resulting p-values were adjusted for multiple comparisons using the Benjamini–Hochberg procedure (statsmodels.stats.multitest. multipletests, method='fdr_bh'). Author contributions ARZ conceived the study and, together with NM, analyzed and interpreted the results and drafted the manuscript. NM performed all computational analyses except for the unsupervised evaluation of sample-level representations and the attention-abundance correlation analyses, which were conducted by SZ and ARZ, respectively All authors have read and approved the final manuscript. Funding This project was unfunded. Conflict of interests The authors declare no competing interests. References 1. Cho I, Blaser MJ. The human microbiome: at the interface of health and disease. Nat Rev Genet. 2012;13(4):260-70. doi: 10.1038/nrg3182. PubMed PMID: 22411464; PMCID: PMC3418802. 2. Hou K, Wu ZX, Chen XY , Wang JQ, Zhang D, Xiao C, Zhu D, Koya JB, Wei L, Li J, Chen ZS. Microbiota in health and diseases. Signal Transduct Target Ther. 2022;7(1):135. Epub 20220423. doi: 10.1038/s41392-022-00974-4. PubMed PMID: 35461318; PMCID: PMC9034083. 3. Mallick H, Rahnavard A, McIver LJ, Ma S, Zhang Y , Nguyen LH, Tickle TL, Weingart G, Ren B, Schwager EH, Chatterjee S, Thompson KN, Wilkinson JE, Subramanian A, Lu Y , Waldron L, Paulson JN, Franzosa EA, Bravo HC, Huttenhower C. Multivariable association discovery in population-scale meta-omics studies. PLoS Comput Biol. 2021;17(11):e1009442. Epub 20211116. doi: 10.1371/journal.pcbi.1009442. PubMed PMID: 34784344; PMCID: PMC8714082. 4. Lin H, Peddada SD. Analysis of compositions of microbiomes with bias correction. Nat Commun. 2020;11(1):3514. Epub 20200714. doi: 10.1038/s41467-020-17041-7. PubMed PMID: 32665548; PMCID: PMC7360769. 5. Mandal S, Van Treuren W, White RA, Eggesbø M, Knight R, Peddada SD. Analysis of composition of microbiomes: a novel method for studying microbial composition. Microb Ecol Health Dis. 2015;26:27663. Epub 20150529. doi: 10.3402/mehd.v26.27663. PubMed PMID: 26028277; PMCID: PMC4450248. 6. Tunyasuvunakool K, Adler J, Wu Z, Green T, Zielinski M, Žídek A, Bridgland A, Cowie A, Meyer C, Laydon A, Velankar S, Kleywegt GJ, Bateman A, Evans R, Pritzel A, Figurnov M, Ronneberger O, Bates R, Kohl SAA, ..., Hassabis D. Highly accurate protein structure prediction for the human proteome. Nature. 2021;596(7873):590-6. Epub 20210722. doi: 10.1038/s41586-021-03828-1. PubMed PMID: 34293799; PMCID: PMC8387240. 7. Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, Tunyasuvunakool K, Bates R, Žídek A, Potapenko A, Bridgland A, Meyer C, Kohl SAA, Ballard AJ, Cowie A, Romera-Paredes B, Nikolov S, Jain R, Adler J, ..., Hassabis D. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596(7873):583-9. Epub 20210715. doi: 10.1038/s41586-021-03819-2. PubMed PMID: 34265844; PMCID: PMC8371605. 8. Abramson J, Adler J, Dunger J, Evans R, Green T, Pritzel A, Ronneberger O, Willmore L, Ballard AJ, Bambrick J, Bodenstein SW, Evans DA, Hung CC, O'Neill M, Reiman D, Tunyasuvunakool K, Wu Z, Žemgulytė A, Arvaniti E, ..., Jumper JM. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature. 2024;630(8016):493-500. Epub 20240508. doi: 10.1038/s41586-024-07487-w. PubMed PMID: 38718835; PMCID: PMC11168924. 9. Shen T, Hu Z, Sun S, Liu D, Wong F , Wang J, Chen J, Wang Y , Hong L, Xiao J, Zheng L, Krishnamoorthi T, King I, Wang S, Yin P , Collins JJ, Li Y . Accurate RNA 3D structure prediction using a language model-based deep learning approach. Nat Methods. 2024;21(12):2287-98. Epub 20241121. doi: 10.1038/s41592-024-02487-0. PubMed PMID: 39572716; PMCID: PMC11621015. 10. Chen J, Hu Z, Sun S, Tan Q, Wang Y , Yu Q, Zong L, Hong L, Xiao J, Shen T. Interpretable RNA foundation model from unannotated data for highly accurate RNA structure and function predictions. arXiv preprint arXiv:220400300. 2022. 11. Shen J, Yu Q, Chen S, Tan Q, Li J, Li Y . Unbiased organism-agnostic and highly sensitive signal peptide predictor with deep protein language model. Nat Comput Sci. 2024;4(1):29-42. Epub 20231213. doi: 10.1038/s43588-023-00576-2. PubMed PMID: 38177492. 12. Madani A, Krause B, Greene ER, Subramanian S, Mohr BP , Holton JM, Olmos JL, Xiong C, Sun ZZ, Socher R, Fraser JS, Naik N. Large language models generate functional protein sequences across diverse families. Nat Biotechnol. 2023;41(8):1099-106. Epub 20230126. doi: 10.1038/s41587-022-01618-2. PubMed PMID: 36702895; PMCID: PMC10400306. 13. Yang F , Wang W, Wang F , Fang Y , Tang D, Huang J, Lu H, Yao J. scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data. Nature Machine Intelligence. 2022;4(10):852-66. doi: 10.1038/s42256-022-00534-z. 14. Theodoris CV , Xiao L, Chopra A, Cha\in MD, Al Sayed ZR, Hill MC, Mantineo H, Brydon EM, Zeng Z, Liu XS, Ellinor PT. Transfer learning enables predictions in network biology. Nature. 2023;618(7965):616-24. Epub 20230531. doi: 10.1038/s41586-023-06139-9. PubMed PMID: 37258680; PMCID: PMC10949956. 15. Cui H, Wang C, Maan H, Pang K, Luo F , Duan N, Wang B. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat Methods. 2024;21(8):1470-80. Epub 20240226. doi: 10.1038/s41592-024-02201-0. PubMed PMID: 38409223. 16. Pope Q, Varma R, Tataru C, David MM, Fern X. Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome data. PLoS Comput Biol. 2025;21(5):e1011353. Epub 20250507. doi: 10.1371/journal.pcbi.1011353. PubMed PMID: 40334224; PMCID: PMC12058177. 17. Zhang H, Zhang Y , Kang Z, Song L, Yang R, Ning K. MGM as a large-scale pretrained foundation model for microbiome analyses in diverse contexts. bioRxiv. 2025:2024.12.30.630825. doi: 10.1101/2024.12.30.630825. 18. Pasolli E, Schi\er L, Manghi P , Renson A, Obenchain V , Truong DT, Beghini F , Malik F , Ramos M, Dowd JB, Huttenhower C, Morgan M, Segata N, Waldron L. Accessible, curated metagenomic data through ExperimentHub. Nat Methods. 2017;14(11):1023-4. doi: 10.1038/nmeth.4468. PubMed PMID: 29088129; PMCID: PMC5862039. 19. Gupta VK, Kim M, Bakshi U, Cunningham KY , Davis JM, Lazaridis KN, Nelson H, Chia N, Sung J. A predictive index for health status using species-level gut microbiome profiling. Nat Commun. 2020;11(1):4635. Epub 20200915. doi: 10.1038/s41467-020-18476-8. PubMed PMID: 32934239; PMCID: PMC7492273. 20. Chang D, Gupta VK, Hur B, Cobo-López S, Cunningham KY , Han NS, Lee I, Kronzer VL, Teigen LM, Karnatovskaia LV , Longbrake EE, Davis JM, Nelson H, Sung J. Gut Microbiome Wellness Index 2 enhances health status prediction from gut microbiome taxonomic profiles. Nat Commun. 2024;15(1):7447. Epub 20240828. doi: 10.1038/s41467-024-51651-9. PubMed PMID: 39198444; PMCID: PMC11358288. 21. Blanco-Míguez A, Beghini F , Cumbo F , McIver LJ, Thompson KN, Zolfo M, Manghi P , Dubois L, Huang KD, Thomas AM, Nickols WA, Piccinno G, Piperni E, Punčochář M, Valles-Colomer M, Tett A, Giordano F , Davies R, Wolf J, ..., Segata N. Extending and improving metagenomic taxonomic profiling with uncharacterized species using MetaPhlAn 4. Nat Biotechnol. 2023;41(11):1633-44. Epub 20230223. doi: 10.1038/s41587-023-01688-w. PubMed PMID: 36823356; PMCID: PMC10635831. 22. Segata N, Waldron L, Ballarini A, Narasimhan V , Jousson O, Huttenhower C. Metagenomic microbial community profiling using unique clade-specific marker genes. Nat Methods. 2012;9(8):811-4. Epub 2012/06/10. doi: 10.1038/nmeth.2066. PubMed PMID: 22688413; PMCID: PMC3443552. 23. Pope Q, Varma R, Tataru C, David M, Fern X. Learning a deep language model for microbiomes: the power of large scale unlabeled microbiome data. bioRxiv. 2023:2023.07.17.549267. doi: 10.1101/2023.07.17.549267. 24. Monira S, Nakamura S, Gotoh K, Izutsu K, Watanabe H, Alam NH, Nakaya T, Horii T, Ali SI, Iida T, Alam M. Metagenomic profile of gut microbiota in children during cholera and recovery. Gut Pathog. 2013;5(1):1. Epub 20130201. doi: 10.1186/1757-4749-5-1. PubMed PMID: 23369162; PMCID: PMC3574833. 25. David LA, Weil A, Ryan ET, Calderwood SB, Harris JB, Chowdhury F , Begum Y , Qadri F , LaRocque RC, Turnbaugh PJ. Gut microbial succession follows acute secretory diarrhea in humans. mBio. 2015;6(3):e00381-15. Epub 20150519. doi: 10.1128/mBio.00381-15. PubMed PMID: 25991682; PMCID: PMC4442136. 26. Bäumler AJ, Sperandio V . Interactions between the microbiota and pathogenic bacteria in the gut. Nature. 2016;535(7610):85-93. doi: 10.1038/nature18849. PubMed PMID: 27383983; PMCID: PMC5114849. 27. Winter SE, Bäumler AJ. Why related bacterial species bloom simultaneously in the gut: principles underlying the 'Like will to like' concept. Cell Microbiol. 2014;16(2):179-84. Epub 20131216. doi: 10.1111/cmi.12245. PubMed PMID: 24286560; PMCID: PMC4013256. 28. Younge NE, Araújo-Pérez F , Brandon D, Seed PC. Early-life skin microbiota in hospitalized preterm and full-term infants. Microbiome. 2018;6(1):98. Epub 20180531. doi: 10.1186/s40168-018-0486-4. PubMed PMID: 29855335; PMCID: PMC5984431. 29. Balish E, Warner T. Enterococcus faecalis induces inflammatory bowel disease in interleukin-10 knockout mice. Am J Pathol. 2002;160(6):2253-7. doi: 10.1016/S0002-9440(10)61172-8. PubMed PMID: 12057927; PMCID: PMC1850822. 30. Rashid T, Ebringer A, Wilson C. The role of Klebsiella in Crohn's disease with a potential for the use of antimicrobial measures. Int J Rheumatol. 2013;2013:610393. Epub 20131010. doi: 10.1155/2013/610393. PubMed PMID: 24223596; PMCID: PMC3810322. 31. Högenauer C, Langner C, Beubler E, Lippe IT, Schicho R, Gorkiewicz G, Krause R, Gerstgrasser N, Krejs GJ, Hinterleitner TA. Klebsiella oxytoca as a causative organism of antibiotic-associated hemorrhagic colitis. N Engl J Med. 2006;355(23):2418-26. doi: 10.1056/NEJMoa054765. PubMed PMID: 17151365. 32. Bai M, Guo H, Zheng XY . Inflammatory bowel disease and. Therap Adv Gastroenterol. 2023;16:17562848231207280. Epub 20231128. doi: 10.1177/17562848231207280. PubMed PMID: 38034098; PMCID: PMC10685799.