Structural bias in machine learning-guided peptide design

(250 words) Machine learning continues to accelerate peptide and protein design through the rapid prediction and generation of sequences with desired characteristics. Many applications focus on predicting properties, functions, and structures, as well as generating point mutations and de novo designs. Nevertheless, many models prove less generalizable than initially claimed. Most predictors and generators are trained on sequential datasets, where imbalances can be addressed during preprocessing. In contrast, structural bias, a subtype of algorithmic bias arising from uneven representation of structural classes in training datasets, and the

of early protein structure predictors ha ve frequently remained undetected and uncorrected. The recent surge in powerful protein structure prediction tools, such as the AlphaFold and RosettaFold series and their variants, now presents opportunities to mitigate this issue. We hypothesize that such structural sampling biases influence the downstream performance of ML models . Using antimicrobial peptides as a case study, we audited the structural biases in 16 state-of-the-art predictors for antimicrobial activity and tested whether structural information constrains their predicti ons. Our analysis revealed that models explicitly trained on s equential data still produce predictions biased by uneven fold representations and data leakage . These findings highlight the importance of integrating balanced structural data or implementing bias -mitigating strategies to develop agnostic models that maximize bioactive protein discovery and multi-objective optimization.

Protein design, machine learning, antimicrobial peptides, algorithmic bias, data leakage. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 2 1. INTRODUCTION Facial recognition systems that misidentify people with darker skin tones or women1, lending and credit scoring models that disadvantage minority or low -income communities2, and image generators that reproduce stereotypes embedded in the training data3 illustrate the pervasive nature of algorithmic bias in our society. These examples demonstrate that bias can emerge from human decision -making and the computational systems we design, often mirroring and exacerbating pre-existing inequalities. Algorithmic bias refers to systematic biases in the outputs of computational models or decision -making systems, arising from the data on which they are trained, the algorithms they use, or the contexts in which they operate. The consequences of algorithmic bias in domains such as finance, law enforcement, and media production are increasingly recognized, prompting governments and private organizations to implement policies and guardrails that support the safe and responsible use of artificial intelligence (AI) technologies4,5. By comparison, the effects of algorithmic bias in the natural sciences, including computational biology, have received less attention. Over the past decade, the field of protein biology has benefited from machine learning (ML) methods for engineering, structure prediction, functional annotation, and de novo design6–13. However, because these models are trained on protein sequence or structural databases , they inherit the historical, methodological, and experimental biases embedded in these resources, shaped by decades of research funding priorities, experimental detection limits, taxonomic preferences, and domain- specific data availability14. Recognizing and addressing such biases early is essential to develop ing robust, generalizable, and reproducible applications in protein engineering, drug discovery, and synthetic biology15. It may also present significant biosecurity risks. In line with these objectives, the Responsible AI x Biodesign initiative provides a global framework that outlines community values, guiding principles, and commitments to ensure sound AI-guided protein design practices16. Early evidence of algorithmic bias in protein biology has emerged across sequence-, structure-, and function -prediction ML models. Sequence-based approaches rely on training datasets with phylogenetic and annotation biases that overrepresent certain model organisms, protein families, and conserved domains, thereby reducing model performance on underrepresented sequences17–20. Structure-based models exhibit performance variations due to imbalanced structure classes, fold-switching proteins, and intrinsically disordered regions in structural repositories 21–25. Functional annotation models propagat e historical misannotations from homology -based methods, overlooking novel functions 26–28. These biases can distort predictions and .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 3 generations, narrow the exploration of the sequence-function landscape to dominant protein classes, and limit the discovery of rare folds29–31,24. To examine structural bias, a subtype of algorithmic bias, we recently applied a set of protein structure predictors, including AlphaFold29, to map the structural landscape of medium-to-large-sized datasets using the GRAMPA repository of 5,980 antimicrobial peptides32. Our structural mapping revealed that most peptides (65.1%) adopted loose helices, whereas fewer formed random coils (17.8%) or b-stranded or mixed structures ( 17.1%) – mimicking folds observed in X-ray crystallography and aqueous NMR solutions . Because many state -of-the-art antimicrobial peptide predictors are trained on GRAMPA or similar repositories1, we evaluated 16 models to assess whether the latent structural composition influences their predictions and to explore how structural features shape predictive bias. 2. MATERIALS AND METHODS 2.1. Datasets. GRAMPA. We obtained the peptide sequences from the GRAMPA repository (Giant Repository of AMP Activities), a robust database established in 2018 containing sequences of 5,980 peptides ranging from 5 to 50 residues.33 The GRAMPA repository and detailed information are available at https://github.com/zswitten/Antimicrobial- Peptides. Non-GRAMPA. We retrieved 3,385 peptide sequences ranging from 5 to 50 residues from the UniProt database 34. These sequences had no reported antimicrobial activity and originated from taxa known to produce antimicrobial peptides: Arthropoda (n=1,867), Mammalia (n=501), Violaceae (n=114), Amphibia (n=322), Rubiaceae (n=49), Cucurbitaceae (n=60), and Mollusca (n=472). The search excluded entries labeled as partial, putative, or predicted, and those annotated with keywords Antimicrobial [KW- 0929], Antiviral, Anticancer, Antibiotic [KW-0044], or Fungicide [KW-0295], while restricting results to reviewed records (reviewed: yes). 2.2. Preprocessing. Structural predictions. We used PEP2D (https://webs.iiitd.edu.in/raghava/pep2d)35 to predict the secondary structures of all peptides. Each peptide is represented as a vector of three-state values (H: helices, E: extended strand/β -sheets, and C: coils) expressed as percentages. The collection of peptide structure predictions forms the corresponding structural landscape, depicted in a ternary plot. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 4 Structural classes. We defined seven structural regions (or classes) using the three states (H, E, C) and arbitrary borders at 80:0:20, 20:0:80, and 20:80:0. Removal. We removed duplicated sequences , those containing non-canonical or unknown amino acids (represented as X), a nd sequences common to both the GRAMPA and non-GRAMPA datasets. Redundancy. We used the CD-HIT36 web server (https://github.com/weizhongli/cdhit- web-server) to remove highly redundant sequences from both datasets, grouping sequences with a 70% identity threshold and retaining only the most representative sequence from each group. 2.3. Model selection, evaluation, and interpretation. AMP predictors . We selected 16 predictive models for antimicrobial activity : AMP scanner v2 37, amPEPpy38, AMPlify variants39, CAMPr3 variants40, DBAASP41, IAMPE variants42, iAMPpred43, PepNet44, and Sense the Moment 45 – see Table S1. Evaluating the model performances. Each model was evaluated as a binary classifier, with antimicrobial peptides (AMPs) as signed to class 1 and non -antimicrobial peptides (non-AMPs) to class 0. Depending on the architecture, the model outputs were interpreted as either discrete class labels, class probabilities (e.g., p(sequence X | AMP) = 0.67), or both. For models that produced only class probabilities, sequences with values greater than 0.50 were labeled as AMPs. For each structural class, we computed the confusion matrix, recording the number of True Positives (TP; correctly predicted active AMP sequences), True Negatives (TN; correctly predicted non-AMP sequences as inactive), False Positives (FP; non-AMP sequences incorrectly predicted as AMPs), and False Negatives (FN; AMP sequences incorrectly predicted as non-AMPs). From these, we derived standard performance metrics, including accuracy, precision, specificity, sensitivity, and Matthews Correlation coefficient (MCC). (1) Accuracy, the fraction of correct predictions obtained by the model, defined as: 𝐴𝑐𝑐. = 𝑇𝑃 + 𝑇𝑁 𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁 (2) Precision, the fraction of positive predictions correctly identified: 𝑃𝑟𝑒𝑐. = 𝑇𝑃 𝑇𝑃 + 𝐹𝑃 (3) Specificity, the fraction of negative predictions correctly identified: .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 5 𝑆𝑝𝑒𝑐. = 𝑇𝑁 𝑇𝑁 + 𝐹𝑁 (4) Sensitivity, the fraction of true positives correctly identified: 𝑆𝑒𝑛𝑠. = 𝑇𝑃 𝑇𝑃 + 𝐹𝑁 (5) Matthews Correlation coefficient integrates information from all four elements of the confusion matrix, yielding a value between -1 and 1. Values of 1 and -1 indicate perfect and completely erroneous classification, respectively, whereas a value of 0 reflects random prediction. 𝑀𝐶𝐶 = 𝑇𝑃 ∗ 𝑇𝑁 − 𝐹𝑃 ∗ 𝐹𝑁 5(𝑇𝑃 + 𝐹𝑃) ∗ (𝑇𝑃 + 𝐹𝑁) ∗ (𝑇𝑁 + 𝐹𝑃) ∗ (𝑇𝑁 + 𝐹𝑁) Embeddings and Dimensionality reduction of peptide datasets. We encoded the structures of the training sets (amPEPpy, AMP scanner v2 , AMPlify, PepNet) and the external validation sets (GRAMPA and non -GRAMPA) using ProstT 46, a sequence -structure encoder that generates 3Di tokens as proxies for three -dimensional representations. The resulting structural embeddings were projected onto a two-dimensional UMAP47 manifold optimized to separate the structural classes – see Figure S6. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 6 3. RESULTS Machine learning algorithms have become increasingly complex, gaining in performance but often at the expense of interpretability. Many existing models lack detailed descriptions of the training data and algorithms, which limits their reproducibility and contributes to their perception as “black boxes” . This limited transparency complicates the evaluation of potential biases and the robustness of model predictions. To address these concerns, explainable AI techniques have gained traction, and external (post-hoc) validation on independent datasets is a robust strategy for uncovering inherent biases. In our study, we assembled two curated datasets to critically assess the performance and potential biases of existing binary classifiers for predicting antimicrobial peptide activity. The independent validation set comprised 5,980 antimicrobial peptide sequences from the GRAMPA repository 33 and 3,385 non-antimicrobial peptide sequences from the UniProt database (see Materials and Methods ). To explore the sequence-structure relationships with these sets, we predicted the secondary structure of all peptides using the PEP2D web server35. We then assigned each peptide to one of seven structural classes based on defined thresholds for the fractions of helix ( H), extended strand/β-sheet (E), and coil (C). 3.1. Structural composition of the validation sets. The GRAMPA repository is enriched in α-helical and coiled structures, consistent with PEP2D and AF2 predictions, which were experimentally validated, as previously reported32. To assess structural differences from non-antimicrobial peptides, we predicted the secondary structures of the non-GRAMPA dataset. Initial analysis revealed a limited presence of β-sheet conformations in non-GRAMPA sequences. To better capture this structural class , we supplemented the dataset with 53 sequences explicitly annotated as β-sheet structures, bringing the total to 3,438. We subsequently removed duplicate sequences and those containing non-canonical or ambiguous residues, yielding 2,511 representative non-GRAMPA peptides. Both GRAMPA and non-GRAMPA datasets were further subjected to redundancy reduction via CD-HIT36 clustering at 70% sequence identity . This process reduced GRAMPA to 1,731 sequences and non-GRAMPA to 1,185 sequences , thereby eliminating highly homologous peptides while preserving sequence diversity. Importantly, this reduction did not substantially alter the distribution of structural classes across the datasets. The GRAMPA subset remained dominated by α -helices and coils, accounting for 64.6% of sequences, with 15.1% belonging to β -sheet/coiled .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 7 structures and 14.1% to coils ( Table 1 ). Conversely, the non -GRAMPA subset exhibited a more balanced structural composition, comprising 35.9% helices, 32.7% β- sheets/coils, and 25.6% coils. The structural landscapes of both datasets are illustrated in Figure 1. Table 1. Distributions of predicted structural classes (1)-(7) for the GRAMPA and non- GRAMPA datasets using protein secondary structure predictor PEP2D. Structural class GRAMPA non-GRAMPA N=5980 % N=1731 % N=2511 % N=1185 % 1 Helices and coils 3892 65.1 1119 64.6 749 29.8 425 35.9 2 Mostly helices 88 1.5 23 1.4 5 0.2 5 0.2 3 Helices and strands 0 0.0 0 0.0 0 0.0 0 0.0 4 Mostly strands (β-sheets) 1 0.0 1 0.0 0 0.0 0 0.0 5 Strands and coils 754 12.6 261 15.1 767 30.6 387 32.7 6 Mostly coiled structures 1063 17.8 244 14.1 862 34.3 302 25.6 7 Mixed structures 182 3.0 83 4.8 128 5.1 66 5.6 Figure 1. Predicted structural landscapes of (A) GRAMPA and (B) non-GRAMPA. 3.2. Performance of AMP predictors across structural classes. To investigate whether the structural distributions of GRAMPA and non-GRAMPA subsets influence antimicrobial peptide (AMP) prediction models, we evaluated 16 state-of-the-art predictors published between 2016 and 2024 (Table S1). These models vary in machine learning architectures and training set sizes : AMP scanner v2 37, amPEPpy38, AMPlify variants39, CAMPr3 variants40, DBAASP41, IAMPE variants42, 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 % Helix (H) % Strand (E) % Coil (C) Density 05 1 0 1 5 A. GRAMPA 5 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 % Helix (H) % Strand (E) % Coil (C) Density 05 1 0 1 5 B. Non−GRAMPA 1 2 3 4 7 6 .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 8 iAMPpred43, PepNet44, and Sense the Moment 45. Predictions were stratified into the four dominant structural classes, as shown in Table 1: helices and coils (1), strands and coils (5), mostly coils (6), and mixed structures (7). For each model and structural class, we calculated accuracy, precision , sensitivity, and specificity from the confusion matrices shown in Figures 2A-D and Figures S1-S4. For the helical and coil dataset (1, Figures 2A and S1), most classifiers achieved accuracy and precision consistently above 0.80, indicating robust classification of both AMPs and non-AMPs across all models. Here, the Sense the Moment model is a relative underperformer with an accuracy dropping to nearly 0.70. In addition, nearly all models also showed high sensitivity (~0.80 -0.95) but lower specificity (~0.60 -0.80), suggesting a bias toward predicting the positive (AMP) class. As such , the model IAMPE (SVM) exhibited the largest gap between sensitivity (~0.95) and specificity (~0.42). The models AMPlify and AMPlify (imbalanced) offer the best balance with relatively high accuracy, precision, and sensitivity without low specificity. This sensitivity-specificity imbalance persists across other structural classes (5, 7) and classifiers, suggesting a systemic bias in the benchmark and reflecting the composition of the training data. Most predictors in the strands and coils class (5; Figures 2B and S2) underperformed, with low accuracy, precision (~0.40), and specificity (~0.25), but exhibited the highest sensitivity (>0.90). Interestingly, three classifiers – AMPlify, AMPlify (imbalanced), and PepNet – displayed all classification metrics above 0.75, demonstrating strong predictive performance for this structural class. We observed similar trends in the mixed structures class (7, Figures 2D and S4), where predictors performed moderately well, with accuracy and precision in the 0.60 -0.70 range, specificity averaging 0.30, and sensitivity above 0.90. Finally, several classifiers for the coiled class (6, Figures 2C and S3) achieved near -equivalent performance across all metrics, suggesting either a more tractable structural class or a smaller, less discriminating test set. The IAMPE (SVM) and IAMPE (XGB) exhibited the largest sensitivity-specificity gaps, indicating a bias toward coiled AMPs. The same three classifiers – AMPlify, AMPlify (imbalanced), and PepNet – outperformed the other models, suggesting they capture features relevant to stranded and coiled peptide datasets that others do not. Despite these observations, confusion-matrix metrics may overestimate performance on imbalanced datasets48. To ensure fairer comparisons across structural classes, we also evaluated performance using the Matthews Correlation Coefficient (MCC )49, which balances predictions across both positive and negative classes (see Materials and Methods). As shown in Figure 2E, MCC values varied across structural classes. The AMPlify models are the most robust classifiers across all four structural classes, with AMPlify (imbalanced) peaking at 0.82 on coiled structures. The two models .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 9 maintained strong MCC values even on the hardest classes (5: 0.74/0.79 and 7: 0.66/0.73), where other models collapsed. The model PepNet is arguably the third most reliable classifier, with moderate MCC values of 0.70 ( strands), 0.65 (coils), 0.63 (helices), and 0.59 (mixed structures). In contrast, the model Sense the Moment (StM) exhibited the (near -)lowest MCC values in three of the four structural classes: 0.19 (strands), 0.12 (coils), 0.34 (helices), and 0.26 (mixed structures). Its moderate accuracy (Figure S2) on helical peptides may be due to class bias. All IAMPE variants and the iAMPpred model exhibited near -random MCC values on stranded and mixed structures (5 and 7), confirming the high-sensitivity/low-specificity pattern. Overall, these results confirm that peptide structural features influence model predictions, with β-sheet-containing classes (5 and 7) consistently performing worse than helical and coiled classes (1 and 6). This pattern suggests an algorithmic bias that hinders the classification of AMPs versus non -AMPs within strand-rich structural space. Structural class 1 (helices and coils) achieved the highest performance across evaluation approaches, confirming reliable classification across models. In contrast, strand-rich classes (5 and 7) exhibited high sensitivity but low specifi city, and their low MCC values revealed limited overall discriminatory reliability. This MCC -based evaluation aligns with the confusion matrix analyses ( Figures 2A-D and S1-S4), and MCC provides a more comprehensive representation of model efficacy than accuracy alone, particularly in skewed datasets where sensitivity -specificity trade -offs can obscure predictive power48. Among the 16 classifiers evaluated, AMPlify (both models) and PepNet showed the strongest and most consistent performance across all structural classes. Notably, the AMPlify (imbalanced) variant achieved the highest single MCC value across the entire benchmark (0.82 on coiled structures) and maintained strong discriminative power on the helical and stranded classes alike, suggesting that accounting for structural class imbalance during model training contributes to its robustness. With the exception of these models, our combined analyses indicate that most classifiers struggle to reliably discriminate AMP from non -AMP sequences containing β-sheet motifs, a limitation that accuracy-based metrics alone would have underrepresented. Figure 2. Model performance varies across structural classes: (A) class 1 (helices/coils), (B) class 5 (strands/coils), (C) class 6 (predominantly coiled), and (D) class 7 (mixed). Evaluation is based on confusion-matrix metrics (specificity, sensitivity, accuracy, precision), with global performance captured by Matthews correlation coefficients (E). .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 10 Next, we assessed per-sequence model performance by calculating the proportion of sequences correctly classified as GRAMPA or non -GRAMPA across different structural classes (Figure 3). The figure displays the predictions – true positives (TP), false negatives ( FN), false positives ( FP), and true negatives ( TN) – for the 16 AMP classifiers tested against the four structural classes (1, 5, 6, and 7). Most models achieve high true-positive rates for GRAMPA peptides (dominant dark blue bars on the left) but also produce significant false positives among non -GRAMPA peptides (light red bars on the right). This further confirms the systemic bias in traini ng toward known AMPs. The right side of each panel shows that a few models, such as AMPlify and PepNet, consistently achieve high true -negative rates (dark red bars) acro ss all four structural classes. 0.74 0.82 0.49 0.65 0.120.620.44 0.21 0.35 0.46 0.33 0.350.36 0.38 0.460.37 0.69 0.74 0.51 0.63 0.340.600.58 0.54 0.56 0.60 0.54 0.490.56 0.48 0.580.51 0.74 0.79 0.43 0.70 0.190.260.10 0.17 0.12 0.22 0.16 0.080.06 0.04 0.060.12 0.66 0.73 0.21 0.59 0.260.370.27 0.22 0.27 0.45 0.36 0.140.12 0.13 0.230.11Mixed structures (7) Coiled structures (6) Strands and coils (5) Helices and coils (1) AMP scanner v2 amPEPpy AMPlify AMPlify ( imbalanced) CAMPr 3 ( ANN)CAMPr 3 ( D AC)CAMPr 3 ( RF)CAMPr 3 ( SVM) DBAASPIAMPE (kNN)IAMPE (RF)IAMPE (SVM)IAMPE (XG B)iAMPpred PepNet Sense t he Moment Classifiers Structural Class Metric E Matthews Correlation coefficients (MCC) A C Coiled structures (6) D B 0.91 0.88 0.92 0.77 0.92 0.90 0.94 0.77 0.90 0.77 0.76 0.79 0.89 0.85 0.90 0.72 0.85 0.69 0.69 0.69 0.89 0.84 0.89 0.70 0.88 0.83 0.88 0.69 0.88 0.81 0.86 0.69 0.88 0.82 0.87 0.69 0.90 0.83 0.87 0.74 0.88 0.81 0.85 0.71 0.87 0.78 0.82 0.69 0.88 0.83 0.89 0.67 0.82 0.81 0.95 0.44 0.86 0.84 0.93 0.60 0.86 0.81 0.88 0.62 AMP scanner v2 amPEPpyAMPlify AMPlify ( imbalanced) CAMPr 3 ( ANN)CAMPr 3 ( D AC)CAMPr 3 ( RF)CAMPr 3 ( SVM) DBAASPIAMPE (kNN)IAMPE (RF)IAMPE (SVM)IAMPE (XG B)iAMPpredPepNet Sense the Moment 0.80 0.87 0.91 0.85 0.81 0.89 0.96 0.85 0.81 0.73 0.43 0.93 0.76 0.85 0.90 0.81 0.60 0.63 0.26 0.88 0.47 0.55 0.91 0.32 0.42 0.44 0.95 0.10 0.45 0.52 0.86 0.28 0.43 0.49 0.86 0.24 0.45 0.52 0.92 0.26 0.45 0.51 0.85 0.28 0.42 0.45 0.93 0.12 0.41 0.44 0.94 0.09 0.41 0.42 0.97 0.04 0.41 0.43 0.95 0.08 0.43 0.47 0.92 0.16 Helices and coils (1) Specificity Sensitivity Strands and coils (5) 0.82 0.83 0.89 0.76 0.83 0.87 0.95 0.76 0.68 0.59 0.49 0.71 0.78 0.80 0.88 0.70 0.81 0.57 0.30 0.91 0.67 0.69 0.88 0.45 0.62 0.64 0.93 0.27 0.62 0.62 0.83 0.36 0.64 0.64 0.83 0.41 0.69 0.72 0.90 0.50 0.67 0.68 0.86 0.47 0.59 0.59 0.88 0.23 0.59 0.58 0.87 0.23 0.58 0.58 0.95 0.12 0.60 0.62 0.94 0.21 0.58 0.58 0.86 0.23 0.84 0.87 0.87 0.87 0.85 0.91 0.95 0.87 0.86 0.74 0.51 0.93 0.80 0.83 0.82 0.83 0.54 0.58 0.32 0.78 0.74 0.81 0.87 0.75 0.60 0.69 0.89 0.53 0.54 0.60 0.64 0.56 0.63 0.68 0.68 0.68 0.68 0.73 0.75 0.72 0.62 0.66 0.66 0.67 0.59 0.66 0.78 0.57 0.59 0.66 0.80 0.56 0.55 0.63 0.95 0.37 0.60 0.69 0.93 0.50 0.60 0.67 0.81 0.56 Mixed structures (7) Accuracy Precision Specificity Sensitivity Accuracy Precision Specificity Sensitivity Accuracy Precision Specificity Sensitivity Accuracy Precision AMP scanner v2 amPEPpyAMPlify AMPlify ( imbalanced) CAMPr 3 ( ANN)CAMPr 3 ( D AC)CAMPr 3 ( RF)CAMPr 3 ( SVM) DBAASPIAMPE (kNN)IAMPE (RF)IAMPE (SVM)IAMPE (XG B)iAMPpredPepNet Sense the Moment AMP scanner v2 amPEPpyAMPlify AMPlify ( imbalanced) CAMPr 3 ( ANN)CAMPr 3 ( D AC)CAMPr 3 ( RF)CAMPr 3 ( SVM) DBAASPIAMPE (kNN)IAMPE (RF)IAMPE (SVM)IAMPE (XG B)iAMPpredPepNet Sense the Moment AMP scanner v2 amPEPpyAMPlify AMPlify ( imbalanced) CAMPr 3 ( ANN)CAMPr 3 ( D AC)CAMPr 3 ( RF)CAMPr 3 ( SVM) DBAASPIAMPE (kNN)IAMPE (RF)IAMPE (SVM)IAMPE (XG B)iAMPpredPepNet Sense the Moment 0.00 0.25 0.50 0.75 1.00 .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 11 Figure 3 Mapping the proportions of predictions across classifiers and structural classes - (A) 1: Helices and coils, (B) 5: Strands and coils, (C) 6: Mostly coiled structures, and (D) 7: Mixed structures. In Figure 3A, models perform confidently overall against the helical structural class (1), as expected, since α-helical peptides are likely to dominate AMP training databases, such as GRAMPA 32. Most classifiers output high TP rates (about 75-95%) and moderate TN rates for non-GRAMPA peptides. IAMPE (RF) has the lowest TN rate; the model may struggle with helical non-AMPs. Sense the Moment (StM) performs well on helices, consistent with its design to distinguish AMP sequences from randomized variants that maintain the hydrophobic moment but lack sequence order and secondary structure. The authors of StM recommend using their tool as a complementary model, primarily for sequences prone to form helical motifs 45. The model performance declines in other structural classes. Figure 3C displays greater variation across models, with performance differences falling between those of the other structure classes. Several classifiers show low TP rates (<60%) and high false-negative rates for GRAMPA peptides, especially DBAASP and StM, indicating that coiled structures are often misclassified as non-AMPs – likely due to underrepresentation or structural ambiguity in the training sets. Among the AMPscanner v2 amPEPpy AMPlify AMPlify (imbalanced) CAMPr3 (ANN) CAMPr3 (DAC) CAMPr3 (RF) CAMPr3 (SVM) DBAASP IAMPE (kNN) IAMPE (SVM) IAMPE (RF) IAMPE (XGB) iAMPpred PepNet Sense the Moment 100 75 50 25 0 25 50 75 100 Proportion of predictions (%) 100 75 50 25 0 25 50 75 100 Proportion of predictions (%) 100 75 50 25 0 25 50 75 100 Proportion of predictions (%) 100 75 50 25 0 25 50 75 100 Proportion of predictions (%) A Helices and coils (1) B Strands and coils (5) C Coiled structures (6) D Mixed structures (7) True Positive (GRAMPA) False Negative (GRAMPA) False Positive (Non −GRAMPA) True Negative (Non −GRAMPA) AMPscanner v2 amPEPpy AMPlify AMPlify (imbalanced) CAMPr3 (ANN) CAMPr3 (DAC) CAMPr3 (RF) CAMPr3 (SVM) DBAASP IAMPE (kNN) IAMPE (SVM) IAMPE (RF) IAMPE (XGB) iAMPpred PepNet Sense the Moment AMPscanner v2 amPEPpy AMPlify AMPlify (imbalanced) CAMPr3 (ANN) CAMPr3 (DAC) CAMPr3 (RF) CAMPr3 (SVM) DBAASP IAMPE (kNN) IAMPE (SVM) IAMPE (RF) IAMPE (XGB) iAMPpred PepNet Sense the Moment AMPscanner v2 amPEPpy AMPlify AMPlify (imbalanced) CAMPr3 (ANN) CAMPr3 (DAC) CAMPr3 (RF) CAMPr3 (SVM) DBAASP IAMPE (kNN) IAMPE (SVM) IAMPE (RF) IAMPE (XGB) iAMPpred PepNet Sense the Moment .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 12 IAMPE variants, differences in performance on coiled structures appear to be driven more by model architecture than by training data, since all variants use the same training set. IAMPE (RF) achieves a high TP rate for GRAMPA coils but has the lowest true-negative rate among IAMPE variants , suggesting that the random forest architecture tends to overfit AMPs for this structural class. The other IAMPE models (kNN, SVM, XGB) attain higher TN rates despite lower TP rates, indicating that these architectures are more conservative in predicting coiled structures. A similar pattern is observed among CAMPr3 variants, with the random forest model achieving the highest TP and TN rates in the series. AMPlify, AMPlify (imbalanced), amPEPpy, and PepNet performed best on this structural class , achieving the highest combined true- positive and true-negative rates. In Figures 3B and 3D, overall performance drops notably, reflecting that stranded AMPs (class 5) and mixed structures (class 7) are underrepresented in the training data. Many models – including AMP scanner v2 , CAMPr3 variants, and IAMPE variants – misclassify non-GRAMPA strands and mixed structures as AMPs, resulting in higher false -positive rates despite maintaining high TP rates for GRAMPA sequences. This simultaneous rise in both true-positive and false-positive rates suggests that these models recognize broad physicochemical features common to helical AMPs rather than structure-specific signatures. Conversely, DBAASP and StM show the opposite pattern – with higher false -negative and true -negative rates – consistent with a training bias toward helical AMPs and, in the case of StM, its original helix-centric design. Amon g all tested models, AMPlify, AMPlify (imbalanced) , and PepNet perform best across these two structural classes, exhibiting relatively higher TP and TN rates, which aligns with their strong performance shown in Figure 3C. To better understand model confidence, we examined class probability distributions for nine out of the 16 classifiers that provided probability outputs (Figure S5). Overall, GRAMPA peptides (shown in blue) had median probabilities above 0.5 across all structural subsets, consistent with their correct labe lling as AMPs. In contrast, non - GRAMPA sequences (depicted in red) exhibited wider distributions and greater uncertainty. Structural classes lacking stranded motifs (classes 1 and 6) typically fell below the 0.5 threshold, while strand-rich subsets (classes 5 and 7) often exceeded it— consistent with the higher false positive rates seen in Figure 3. Model confidence also mirrored structural bias: probability distributions are tighter and higher for helical GRAMPA peptides, gradually broadening across coiled, stranded, and mixed structural classes, reinforcing the training bias shown in Figure 3. Among the nine models, AMPlify and AMPlify (imbalanced) produce the clearest separation between GRAMPA and non -GRAMPA probability distributions across all structural classes, with tight distributions approaching 1.00 for GRAMPA sequences and 0.00 for non - GRAMPA sequences. PepNet displayed similarly well -separated distributions for .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 13 most structural classes , though notably more dispersed probabilities for non- GRAMPA helices and mixed structures. In contrast, non -GRAMPA sequences with stranded or mixed structures were assigned intermediate class probabilities (0.25-0.75) across most remaining models - including amPEPpy, AMP scanner v2 , CAMPr3 variants, and iAMPpred - indicating that the ir decision boundaries are poorly calibrated for non-helical structures. In summary, structural class strongly influences classifier performance, with helical peptides being the most consistently detected and β -sheet-rich subsets (5 and 7) presenting the greatest challenge. Most models exhibit a high false-positive rate across structural classes, reflecting limited specificity for non -helical AMPs and suggesting that current classifiers generalize poorly beyond the helical sequences that dominate their training data. Few models perform consistently across all four structural classes; AMPlify, AMPlify (imbalanced), and PepNet demonstrated the most balanced profiles in Figure 3, and the probability distributions in Figure S5 further corroborate their superior discrimination across all structural classes. 3.3. Understanding structural bias in predictions. To elucidate the origi ns of observed prediction biases , particularly the misclassification of non-GRAMPA sequences containing β-sheet motifs as AMPs, we investigated two potential contributing factors: structural imbalance and data leakage in training sets. We first assessed whether structural diversity was adequately represented across both sequence classes. W e encoded the structures of both external GRAMPA and non - GRAMPA validation sets using ProstT546, a sequence-structure encoder that generates 3Di tokens as computationally efficient proxies for three-dimensional representations. The approach circumvented the need for explicit structure predictors such as PEP2D and AlphaFold2. The resulting structural embeddings were projected onto a two - dimensional UMAP 47 manifold fine -tuned to maximize the separation of the four structural classes under consideration (folds 1, 5, 6, 7) – see Figure S6. As shown in Figure 4 , the two datasets are distributed across the structural space, forming two clusters: a broad, continuous lower manifold (bottom center -right) and a compact upper cluster (top left). The first cluster contains the four predicted folds, predominantly dominated by structural classes 1 and 6, whereas the second cluster includes only structural classes 5 and 7. Comparing the datasets fold by fold, GRAMPA helices assigned to fold 1 (light green, panel A) are abundant, whereas their non-GRAMPA counterparts (yellow, panel B) remain underrepresented. Conversely, non-GRAMPA sequences are enriched in strands and coils (5 and 6, red and orange) .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 14 relative to their GRAMPA counterparts . Th ese asymmetries mirror the structural imbalance documented in Table 1 , namely , the overrepresentation of helical folds among AMPs and of strands and coils among non-AMPs – a bias that likely underlies the misclassification of stranded and coiled non-GRAMPA sequences as AMPs. Figure 4. Structural diversity of GRAMPA and non -GRAMPA external validation sets projected onto a UMAP structural space. UMAP projections of ProstT5-derived 3Di structural embeddings for the GRAMPA (A) and non -GRAMPA (B) validation sets, colored by predicted structural fold (1, 5, 6, 7). The manifold was fine -tuned to maximize the separation of the four structural classes. We then compared the structural distributions of the two external validation sets with those of the available training sets for four classifiers : GRAMPA against the positive training sets and non-GRAMPA against the negative training sets. The classifiers were AMP scanner v2 (panels A and B) , amPEPpy (panels C and D) , AMPlify, and PepNet sharing identical training sets (panels E and F). The results are illustrated in Figure 5. Across all four classifiers, a large fraction of GRAMPA sequences overlap s with the positive training sets (dark blue, “shared”), indicating varying degrees of data leakage (defined here as sequence identity) into them. The shared sequences are not confined to a specific structural region but are distributed broadly across the UMAP manifold, suggesting the leakage is uniform across structural classes. The proportion of shared sequences is notable : 626/1707 for AMP scanner v2 , 594/1707 for amPEPpy, and 695/1707 for AMPlify/PepNet, representing roughly one third of GRAMPA sequences are shared with the respective positive training sets. Of note, AMPlify (imbalanced) shares the same positive training set as AMPlify and PepNet and therefore exhibits an identical positive leakage (695/1707). A GRAMPA (N=1707) B Non-GRAMPA (N=1180) .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 15 On the negative side, AMP scanner v2 (panel B) and amPEPpy (panel D) are effectively leakage-free; both show zero and 11 shared sequences, respectively, between the non- GRAMPA validation set and the negative training sets. AMPlify/PepNet (panel F) includes 576 shared sequences out of 1180, representing nearly half of the non - GRAMPA validation set that overlaps with the negative training data. AMPlify (imbalanced) retains this negative leakage and expands to over 102,000 peptide sequences (not depicted). Consequently, the fraction of non-GRAMPA sequences is at least as high as in panel F, increasing the likelihood that additional non -GRAMPA sequences are covered. The models AMP scanner v2 and amPEPpy exhibit negligible negative leakage but perform poorly on non-GRAMPA strands and mixed structures (5 and 7 ; Figures 3B and 3D). The negative training sets (light red) are heavily concentrated in the lower broad cluster, dominated by helices and coils (1 and 6). The non-GRAMPA validation sequences (dark red, “unique”), however, are distributed across both the lower manifold and the upper cluster, which is occupied by folds 5 and 7 (strands and mixed structures). AMP scanner v2 (panel B) contained no negative stranded and mixed examples, whereas amPEPpy (panel D) spans a broader, more continuous structural space, partially covering the upper cluster with fewer helical folds (1) - yet this partial coverage remains insufficient to prevent misclassification of stranded and mixed structures. This s tructural underrepresentation in the negative training sets causes both models to misclassify out-of-distribution non-GRAMPA sequences from folds 5 and 7 as AMPs by default. The broader negative set coverage visible in panel F (Figure 5 ) offers a structural explanation for the apparent superior performance of AMPlify/PepNet across structural classes, including greater representation of stranded and mixed structures in the upper cluster. However, t he data leakage observed in panels E (~41% of GRAMPA) and F (~49% of non -GRAMPA) indicates that both validation sets are contaminated with sequences from the AMPlify, AMPlify (imbalanced), PepNet training datasets. All three models may simply be recognizing examples seen during training (memorization) rather than learning structurally generalizable discriminating features. In the case of AMPlify (imbalanced), the disproportionate size of the negative training set increases the likelihood that non -GRAMPA validation sequences are represented in training. Their superior performance across all four structural classes may therefore be inflated by this leakage, making it difficult to determine how much is attributable to learning versus memorization of identical training examples. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 16 Figure 5 Structural overlap between external validation sets and classifier training sets in UMAP space. UMAP projections of ProstT5 -derived 3Di structural embeddings compare the structural distributions of GRAMPA (Val1, blue shades) and non-GRAMPA (Val2, red shades ) with the positive and negative training sets (light shades) for AMP scanner v2 (A, B), amPEPpy (C, D), and AMPlify/PepNet (E, F) , respectively. Darker points denote the validation sequences shared with the corresponding training set. Sample sizes are indicated in the legends. A AMP scanner v2 (+) B AMP scanner v2 (-) C amPEPpy (+) D amPEPpy (-) E F AMPlify (-) / PepNet (-)AMPlify (+) / PepNet (+) .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 17 To disentangle data leakage from genuine pan -fold learning, we re -evaluated classification performance on non -overlapping subsets of GRAMPA (N=1,012) and non-GRAMPA sequences (N=604). Sustained superior discrimination on these subsets would support genuine model quality, whereas a performance drop would implicate data leakage as a major confounding factor. We report prediction results – true positives (TP), false negatives (FN), false positives (FP), and true negatives (TN) – for the AMPlify and PepNet models, both before ( Tables S2 and S4) and after removing duplicate sequences (Tables S3 and S5). These results are summarized in Figure 6. The removal of shared sequences between the training and validation sets resulted in significant losses for both GRAMPA and non -GRAMPA sequences across folds. GRAMPA lost 37.4% of helices (fold 1), half (52.5%) of stranded AMPs (fold 5), 41.8% coils (fold 6) , and 44.6% mixed structures (fold 7) . Similarly, non -GRAMPA diminished 44% of helices, 46% of stranded non-AMPs, nearly two-thirds (62.6%) of coils, and one-third of mixed structures. After removing leakage, AMPlify consistently outperforms PepNet across all folds in both sensitivity and specificity (Figure 6 ). For AMPlify, FN counts are exactly preserved across all folds, and FP counts are nearly identical (fold 1: 96 to 93) , indicating that the performance drops are almost entirely denominator -driven; consistent with the near-complete memorization of the leaked sequences. For PepNet, both FN and FP counts decrease slightly after de -leaking, indicating that a small number of leaked sequences were misclassified and that memorization was therefore less complete than in AMPlify. Sensitivity (DTP%) drops 5-11% for both models across folds, while the false positive rate (DFP%) rises by 13-22% for AMPlify (Figure 6A) and 13-28% for PepNet (Figure 6B). Specificity is more affected than sensitivity across all folds and both models, suggesting that the non -AMP decision boundary has learned less from genuine sequence features and is more vulnerable to leakage removal. Examining performance by structural class reveals consistent patterns. Fold 1 (helices) retains the highest sensitivity for both models ( AMPlify 86.7%, PepNet 84.8%) and shows the smallest sensitivity drop ( DTP% = –5.0% for both), suggesting that helical AMPs carry more discriminative sequence features beyond the training set. Fold 5 (stranded AMPs – upper panels) exhibits greater sensitivity loss relative to baseline (DTP% = –10.1% and –10.6%), consistent with the largest proportion of leaked sequences (52.5%). Fold 7 (mixed structures) shows more modest declines in either sensitivity or specificity across both models, despite its small sample size. The largest divergence between models occurs in fold 6 (coiled structures), where both models fall below 75% sensitivity. With PepNet, the proportion of true positives decreases from 81.6% to 70.4% after leakage removal ( Tables S4 and S5). AMPlify retains approximately 8% higher sensitivity and 9% lower FP rate than PepNet. Overall, these .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 18

indicate that both models had inflated performance metrics due to data leakage, with coiled AMPs representing a structurally distinct class that neither model has learned to reliably distinguish from non-AMPs on novel sequences. Figure 6. Classification proportions before and after data leakage removal for AMPlify (A) and PepNet (B). The positive set (GRAMPA/AMP) is shown in the upper panels as stacked proportions of TP (dark blue) and FN (light blue). The negative set (non-GRAMPA/non-AMP) is shown in the lower panels as stacked proportions of FP (light red) and TN (dark red). Solid ba rs represent performance before leakage removal; hatched bars represent performance after removing shared sequences between training and validation sets. Delta values (DTP%, DFP%) indicate the change in proportion after leakage removal. Folds correspond to structural classes: 1 (helices), 5 (strands), 6 (coils), and 7 (mixed structures). A AMPlify B PepNet 100 60 80 20 40 0 100 60 80 20 40 0 100 60 80 20 40 0 100 60 80 20 40 0 Proportions (%) Proportions (%) Proportions (%) Proportions (%) True Positive (GRAMPA) False Negative (GRAMPA) False Positive (Non-GRAMPA) True Negative (Non-GRAMPA) Before removal After removal fold 1 fold 5 fold 6 fold 7 -5.0 -10.6 -11.1 -9.7-5.0ΔTP (%) ΔFP (%) -10.1 -9.1 -8.7 +17.0 +13.5 +27.7 +12.9+16.5 +13.0 +22.1 +12.1 fold 1 fold 5 fold 6 fold 7 fold 1 fold 5 fold 6 fold 7 fold 1 fold 5 fold 6 fold 7 .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 19 4. DISCUSSION Our study demonstrates that current antimicrobial peptide (AMP) predictive models exhibit structural bias arising from the uneven representation of peptide structural classes in training datasets. Most models achieve high predictive accuracy for sequences adopting helical folds and coils , but perform poorly on β-sheet motifs , thereby limiting their applicability across the broader AMP structural space . These observations are consistent with independent work by Dean and co -workers, who reported that computationally predicted and generated AMPs against E. coli, S. aureus, and P. aeruginosa predominantly folded into α-helices50. This mirrors the inherent bias towards helical motifs in existing AMP datasets used for activity prediction and generation. Comparable biases have been reported across other protein classes51 and computational tasks, resulting in skewed predictions in functional annotation 52,53, protein-protein interaction networks 27, protein stability54,55, protein structure32,56, and in sequence or structure generation57,58. This work contributes to a growing body of studies showing that biases in training data inflate performance metrics in protein prediction and generation . Multiple strategies have been proposed to mitigate systematic bias in ML models, including class resampling, clustering-based methods , synthetic data augmentation via generative algorithms , feature augmentation, and optimized classification algorithms55,59. In our recent work, subset selection enabled the development of structure-specific models with superior predictive performance, whereas targeted data reduction led to information loss in structure-agnostic models60, underscoring the need for structural diversity and rigorous curation in the next generation of ML models56. The structural biases documented here also carry underappreciated consequences for prospective experimental campaigns. Poor model recall of β-sheet AMPs leads to fewer stranded candidates being synthesized and tested, fewer representatives in public databases, and continued training imbalance in future models – a self - reinforcing loop analogous to chemotype bias in drug discovery 61–63. Breaking this cycle requires treating structural diversity as an explicit experimental design criterion, prioritizing β-sheet and mixed -fold candidates for synthesis over relying solely on predicted activity scores. The observed differences in classification performance across structural classes are attributable to training imbalance and variation in learned (physicochemical) features across folds. α-Helical AMPs possess well-characterized cationic charge distribution, hydrophobicity, and amphipathicity that are captured by the hydrophobic moment, a vector sum of side -chain hydrophobicities projected along the helical axis and the basis of the StM model45,64. These properties are amenable to extraction by both .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 20 traditional machine learning and neural network architectures. By comparison, stranded and mixed AMPs include structural components (e.g., hydrogen bonding, strand orientation, disulfide bridges) that are non-local and poorly captured by sequence-only encodings. The superior sensitivity retained for helical AMPs after leakage removal (fold 1: AMPlify 86.7%, PepNet 84.8%) reflects the stronger discrimination in sequence space for this class and points to an intrinsic limitation of sequence-only models for stranded and coiled structures, motivating the development of structure-aware feature representations for pan-fold discrimination of AMPs. Beyond the structural imbalance, the de -leaking analysis further reveals that the two top-performing models exhibited inflated performance metrics driven by near- complete memorization of the leaked sequences rather than generalization. Some parallels can be drawn with large generative models, where training on insufficiently diverse data may lead to memorization that masks poor generalization. The accompanying sensitivity loss (5 -11%) and increase in false positive rate (13 -28%) highlight that neither model has learned to reliably distinguish non-AMPs from novel sequences in the coiled structural space. A principled mitigation strategy is to develop selective classifiers that abstain from making predictions when an input falls outside the structural training distribution 65, for which the ProstT5 -UMAP projections reported here provide a natural basis. Finally, our benchmarking analysis compared models of increasing algorithm ic complexity: early models ( pre-2022) primarily relied on SVM and Random Forest classifiers, wh ile recent models incorporate CNNs or transformers. Greater complexity has improved predictive performance but has not resolved the underlying sampling bias. Joint sequence-structure embeddings (e.g., Prost T546, Saprot66) and co- generation approaches that pair sequence and structure data are promising avenues, though they currently risk biasing exploration to ward well-represented folds . As ProstT5 was trained on AlphaFold2 predictions that were filtered for high structural confidence, enriched in well -structured helical proteins, and excluded disordered sequences, the derived embeddings should be treated with caution. With sufficiently diverse and balanced training data , these approaches hold the potential to expand exploration across broader protein structural space67. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 21 5. CONCLUSION Our study highlights structural bias in predictive models for identifying novel antimicrobial peptides. The combination of uneven structural representation in training data, data leakage, and the intrinsic learnability of helical versus non -helical folds limits the generalization and reliability of current AMP classifiers across the full structural space. While current efforts to integrate sequence, structure, and function into machine learning -guided frameworks offer more controllable peptide and protein d esign with improved success rates, they may also constrain novelty by reinforcing exploration of familiar structural territory. Addressing structural bias – through curated, structurally diverse training data, selective prediction strategies, and joint seq uence-structure modeling – will be essential to unlocking the ‘dark’ peptidome and proteome, creating opportunities for novel discoveries that could drive the next generation of therapeutics, biopesticides, and biomaterials. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 22 Data and code availability Datasets include p eptide sequences, ProsT5 embeddings, activity, and fold predictions. R and Python scripts to reproduce ternary plots and UMAP -ProsT5 projections are all available at https://github.com/plissonf/AMP-structural-bias-audit. Declaration of competing interest V.D.A.-B. declares no competing interests. F.P. is the founder and scientific director of Ingenie Bio , a company that provides consulting and services in computational peptide design. Acknowledgments The authors thank the Mexican Secretaría de Ciencias, Humanidades, Tecnología e Innovación (SECIHTI, formerly CONAHCYT), for financial support through grant A1- S-32579 (2019-2023) supporting this study . V.D.A. -B. acknowledges a national postgraduate scholarship awarded by CONAHCYT during the funded period. Author contributions F.P. conceptualized the investigation. V.D.A.-B. and F.P. carried out the investigation, including methodology, data curation, benchmarking, and bioinformatics analysis. Both authors wrote, edited, and reviewed the manuscript. .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 23

1. Buolamwini, J. & Gebru, T. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. in Proceedings of Machine Learning Research vol. 81 1–15 (2018). 2. Bartlett, R., Morse, A., Stanton, R. & Wallace, N. Consumer-lending discrimination in the FinTech Era. J. Financ. Econ. 143, 30–56 (2022). 3. Bianchi, F. et al. Easily Accessible Text -to-Image Generation Amplifies Demographic Stereotypes at Large Scale. in 2023 ACM Conference on Fairness, Accountability, and Transparency 1493–1504 (ACM, Chicago IL USA, 2023). doi:10.1145/3593013.3594095. 4. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act). (2024). 5. Artificial Intelligence Risk Management Framework (AI RMF 1.0). (2023). 6. Yang, K. K., Wu, Z. & Arnold, F. H. Machine -learning-guided directed evolution for protein engineering. Nat. Methods 16, 687–694 (2019). 7. Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M. Unified rational protein engineering with sequence-based deep representation learning. Nat. Methods 16, 1315–1322 (2019). 8. Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature 577, 706–710 (2020). 9. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). 10. Bileschi, M. L. et al. Using deep learning to annotate the protein universe. Nat. Biotechnol. 40, 932–937 (2022). 11. Watson, J. L. et al. De novo design of protein structure and function with RFdiffusion. Nature 620, 1089–1100 (2023). 12. Abramson, J. et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493–500 (2024). 13. Ahern, W. et al. Atom-level enzyme active site scaffolding using RFdiffusion2. Nat.

23, 96–105 (2026). 14. Albanese, K. I., Barbe, S., Tagami, S., Woolfson, D. N. & Schiex, T. Computational protein design. Nat. Rev. Methods Primer 5, 13 (2025). 15. Eid, F.-E. et al. Systematic auditing is essential to debiasing machine learning in biology. Commun. Biol. 4, 183 (2021). 16. Community Values, Guiding Principles, and Commitments for the Responsible Development of AI for Protein Design. (2023). 17. Schnoes, A. M., Ream, D. C., Thorman, A. W., Babbitt, P. C. & Friedberg, I. Biases in the Experimental Annotations of Protein Function and Their Effect on Our Understanding of Protein Function Space. PLoS Comput. Biol. 9, e1003063 (2013). 18. Radivojac, P. et al. A large-scale evaluation of computational protein function prediction. Nat. Methods 10, 221–227 (2013). 19. Plisson, F., Ramírez -Sánchez, O. & Martínez -Hernández, C. Machine learning -guided discovery and design of non-hemolytic peptides. Sci. Rep. 10, 16581 (2020). .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 24 20. Rádai, Z., Kiss, J. & Nagy, N. A. Taxonomic bias in AMP prediction of invertebrate peptides. Sci. Rep. 11, 17924 (2021). 21. Hiranuma, N. et al. Improved protein structure refinement guided by deep learning based accuracy estimation. Nat. Commun. 12, 1–11 (2021). 22. Varadi, M. et al. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein -sequence space with high -accuracy models. Nucleic Acids Res. 50, D439–D444 (2022). 23. Omidi, A., Møller, M. H., Malhis, N., Bui, J. M. & Gsponer, J. AlphaFold -Multimer accurately captures interactions and dynamics of intrinsically disordered protein regions. Proc. Natl. Acad. Sci. 121, e2406407121 (2024). 24. Chakravarty, D., Lee, M. & Porter, L. L. Proteins with alternative folds reveal blind spots in AlphaFold-based protein structure prediction. Curr. Opin. Struct. Biol. 90, 102973 (2025). 25. Graber, D. et al. Resolving data bias improves generalization in binding affinity prediction. Nat. Mach. Intell. 7, 1713–1725 (2025). 26. Tsishyn, M., Pucci, F. & Rooman, M. Quantification of biases in predictions of protein – protein binding affinity changes upon mutations. Brief. Bioinform. 25, bbad491 (2023). 27. Lannelongue, L. & Inouye, M. Pitfalls of machine learning models for protein –protein interaction networks. Bioinformatics 40, btae012 (2024). 28. Yılmaz, S., Yorgancioglu, K. & Koyutürk, M. Bias -aware training and evaluation of link prediction algorithms in network biology. Proc. Natl. Acad. Sci. 122, e2416646122 (2025). 29. Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M. & Church, G. M. Low -N protein engineering with data-efficient deep learning. Nat. Methods 18, 389–396 (2021). 30. Batra, R. et al. Machine learning overcomes human bias in the discovery of self-assembling peptides. Nat. Chem. 14, 1427–1435 (2022). 31. Wankowicz, S. A. Modeling Bias Toward Binding Sites in PDB Structural Models. Preprint at https://doi.org/10.1101/2024.12.14.628518 (2024). 32. Aldas-Bulos, V. D. & Plisson, F. Benchmarking protein structure predictors to assist machine learning-guided peptide discovery. Digit. Discov. 2, 981–993 (2023). 33. Witten, J. & Witten, Z. Deep learning regression model for antimicrobial peptide design. 692681 Preprint at https://doi.org/10.1101/692681 (2019). 34. The UniProt Consortium et al. UniProt: the Universal Protein Knowledgebase in 2025. Nucleic Acids Res. 53, D609–D617 (2025). 35. Singh, H., Singh, S. & Singh Raghava, G. P. Peptide Secondary Structure Prediction using Evolutionary Information. Preprint at https://doi.org/10.1101/558791 (2019). 36. Huang, Y., Niu, B., Gao, Y., Fu, L. & Li, W. CD-HIT Suite: a web server for clustering and comparing biological sequences. Bioinformatics 26, 680–682 (2010). 37. Veltri, D., Kamath, U. & Shehu, A. Deep learning improves antimicrobial peptide recognition. Bioinformatics 34, 2740–2747 (2018). 38. Lawrence, T. J. et al. amPEPpy 1.0: a portable and accurate antimicrobial peptide prediction tool. Bioinformatics 37, 2058–2060 (2021). 39. Li, C. et al. AMPlify: attentive deep learning model for discovery of novel antimicrobial peptides effective against WHO priority pathogens. BMC Genomics 23, 77 (2022). 40. Waghu, F. H., Barai, R. S., Gurung, P. & Idicula -Thomas, S. CAMPR3  : a database on sequences, structures and signatures of antimicrobial peptides. Nucleic Acids Res. 44, D1094–D1097 (2016). .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 25 41. Pirtskhalava, M. et al. DBAASP v3: database of antimicrobial/cytotoxic activity and structure of peptides as a resource for development of new therapeutics. Nucleic Acids Res. 49, D288–D297 (2021). 42. Kavousi, K. et al. IAMPE: NMR -Assisted Computational Prediction of Antimicrobial Peptides. J. Chem. Inf. Model. 60, 4691–4701 (2020). 43. Meher, P. K., Sahu, T. K., Saini, V. & Rao, A. R. Predicting antimicrobial peptides with improved accuracy by incorporating the compositional, physico-chemical and structural features into Chou’s general PseAAC. Sci. Rep. 7, 42362 (2017). 44. Han, J., Kong, T. & Liu, J. PepNet: an interpretable neural network for anti-inflammatory and antimicrobial peptides prediction using a pre -trained protein language model. Commun. Biol. 7, 1198 (2024). 45. Porto, W. F., Ferreira, K. C. V., Ribeiro, S. M. & Franco, O. L. Sense the moment: A highly sensitive antimicrobial activity predictor based on hydrophobic moment. Biochim. Biophys. Acta BBA - Gen. Subj. 1866, 130070 (2022). 46. Heinzinger, M. et al. Bilingual language model for protein sequence and structure. NAR Genomics Bioinforma. 6, lqae150 (2024). 47. Healy, J. & McInnes, L. Uniform manifold approximation and projection. Nat. Rev.

Primer 4, 82 (2024). 48. Chicco, D. Ten quick tips for machine learning in computational biology. BioData Min. 10, 35 (2017). 49. Chicco, D. & Jurman, G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21, 6 (2020). 50. Dean, S. N., Alvarez, J. A. E., Zabetakis, D., Walper, S. A. & Malanoski, A. P. PepVAE: Variational Autoencoder Framework for Antimicrobial Peptide Generation and Activity Prediction. Front. Microbiol. 12, 725727 (2021). 51. Ding, F. & Steinhardt, J. Protein language models are biased by unequal sequence sampling across the tree of life. Preprint at https://doi.org/10.1101/2024.03.07.584001 (2024). 52. Liu, X. Deep Recurrent Neural Network for Protein Function Prediction from Sequence. https://doi.org/10.1101/103994 (2017) doi:10.1101/103994. 53. Bonetta, R. & Valentino, G. Machine learning techniques for protein function prediction. Proteins Struct. Funct. Bioinforma. 88, 397–413 (2020). 54. Pucci, F., Bernaerts, K. V., Kwasigroch, J. M. & Rooman, M. Quantification of biases in predictions of protein stability changes upon mutations. Bioinformatics 34, 3659 –3665 (2018). 55. Fang, J. The role of data imbalance bias in the prediction of protein stability change upon mutation. PLOS ONE 18, e0283727 (2023). 56. Derry, A., Carpenter, K. A. & Altman, R. B. Training data composition affects performance of protein structure analysis algorithms. Pac. Symp. Biocomput. Pac. Symp. Biocomput. 27, 10–21 (2022). 57. Tang, X. et al. A survey of generative AI for de novo drug design: new frontiers in molecule and protein generation. Brief. Bioinform. 25, bbae338 (2024). 58. Lu, T., Liu, M., Chen, Y., Kim, J. & Huang, P. -S. Assessing generative model coverage of protein structures with SHAPES. Cell Syst. 16, 101347 (2025). 59. Jiang, J. et al. A review of machine learning methods for imbalanced data challenges in chemistry. Chem. Sci. 16, 7637–7658 (2025). .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint 26 60. Aguilera-Puga, M. D. C. & Plisson, F. Structure -aware machine learning strategies for antimicrobial peptide discovery. Sci. Rep. 14, 11995 (2024). 61. Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S. & Hopkins, A. L. Quantifying the chemical beauty of drugs. Nat. Chem. 4, 90–98 (2012). 62. Saldívar-González, F. I., Aldas -Bulos, V. D., Medina -Franco, J. L. & Plisson, F. Natural product drug discovery in the artificial intelligence era. Chem. Sci. 13, 1526–1546 (2022). 63. Van Den Broek, R. L., Patel, S., Van Westen, G. J. P., Jespers, W. & Sherman, W. In Search of Beautiful Molecules: A Perspective on Generative Modeling for Drug Design. J. Chem. Inf. Model. 65, 9383–9397 (2025). 64. Eisenberg, D., Weiss, R. M. & Terwilliger, T. C. The hydrophobic moment detects periodicity in protein hydrophobicity. Proc. Natl. Acad. Sci. 81, 140–144 (1984). 65. Rabanser, S. & Papernot, N. What Does It Take to Build a Performant Selective Classifier? Preprint at https://doi.org/10.48550/ARXIV.2510.20242 (2025). 66. Su, J. et al. Democratizing protein language model training, sharing and collaboration. Nat. Biotechnol. https://doi.org/10.1038/s41587-025-02859-7 (2025) doi:10.1038/s41587 -025- 02859-7. 67. Wang, C., Alamdari, S., Domingo -Enrich, C., Amini, A. P. & Yang, K. K. Toward deep learning sequence–structure co-generation for protein design. Curr. Opin. Struct. Biol. 91, 103018 (2025). .CC-BY 4.0 International licensemade available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is The copyright holder for this preprintthis version posted May 8, 2026. ; https://doi.org/10.64898/2026.05.06.721805doi: bioRxiv preprint