ATLAS: Population-Level Disease Locus Discovery via Differential Attention in Genomic Language Models

Understanding how genetic variants affect gene activity is central to linking genotype to disease. Variants such as missense single-nucleotide polymorphisms (SNPs), insertions, and deletions can alter protein structure, disrupt regulatory elements, or perturb transcriptional regulation, contributing to Mendelian diseases, inher- ited metabolic disorders, and complex diseases includ- ing cancer and neurodevelopmental disorders Cutting (2015); Turner and Eichler (2019); Pagel et al. (2019); Montella et al. (2025). Accurately identifying such vari- ants remains a central challenge in human genetics, with broad implications for disease prediction, mecha- nism discovery, and precision medicine. Genome-wide association studies (GWAS) are the most widely used framework for identifying disease- associated variants, enabling the discovery of thou- sands of loci linked to complex traits through large- scale genotype–phenotype correlation analyses Risch and Merikangas (1996); Klein et al. (2005); Sollis et al. (2023). However, GWAS provide limited resolution at the haplotype level and are not optimized for non-SNP variants such as insertions and deletions Tewhey et al. (2011); Alkan et al. (2011). Their statistical power also depends strongly on allele frequency and effect size, often requiring very large cohorts to detect modest ef- fects Visscher et al. (2017). Complementary bioinfor- matics tools predict variant effects using biological fea- tures such as sequence conservation and protein prop- erties, avoiding large sample size requirements, but are typically restricted to predefined features and individual variants Adzhubei et al. (2013); Vaser et al. (2016). Recent advances in genomic large language models (gLMs) provide a new paradigm for sequence-based variant analysis. Trained on massive genomic corpora, gLMs capture long-range dependencies and achieve state-of-the-art performance in regulatory annotation, variant effect prediction, and functional genomicsDalla- Torre et al. (2025); Brixi et al. (2025); Lin et al. (2025). Unlike GWAS, gLMs operate directly on raw DNA se- quences and naturally accommodate heterogeneous variants, including substitutions and indels. However, most gLM-based applications focus on supervised pre- diction at the individual level Avsec et al. (2021); Bene- gas et al. (2023), leaving population-level sequence comparisons—analogous to GWAS but in a learned em- bedding space—largely unexplored. In this paper, we propose ATLAS (Attention-based Lo- cus Analysis System), an efficient attention-based ex- planation framework built on the genomic language model Genos Lin et al. (2025). We hypothesize that at- tention weights encode sequence-level importance and that functionally relevant loci manifest as statistically sig- nificant attention distribution shifts between case and control populations. By directly contrasting these at- tention patterns, ATLAS enables the identification of disease-associated loci without relying on explicit vari- Liu et al. | | February 9, 2026 | 1–22 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint ant annotations or large sample sizes. Our main contri- butions are as follows. •End-to-end disease locus discovery. We pro- pose ATLAS, a framework that directly analyzes genomic sequences from multiple populations to identify disease-associated loci and genes without explicit variant calling. • Robust and flexible interpretation. ATLAS is ca- pable of multi-scale analysis at both gene and base levels with single-haplotype resolution, demon- strating robustness in small cohorts and high sen- sitivity to low-frequency variants. • Empirical validation on real-world data. AT - LAS not only recovers known disease-associated loci reported in the literature and GWAS but also identifies additional informative candidates on β- thalassemia datasets. • Revisiting the value of human-centric founda- tion models. By benchmarking diverse archi- tectures, we demonstrate that massive parame- ter scale in generalist models does not guaran- tee performance. Instead, we conclude that ac- curate disease localization critically depends on the synergy between model capacity and extensive human-centric pretraining. Related Work Genomic Language Foundation Models The application of large language models (LLMs) to genomics has shifted sequence analysis from alignment-based statistics to representation learning over raw DNA. These genomic language models (gLMs) are trained on large genomic corpora with self- supervised objectives to capture long-range dependen- cies and contextual sequence semantics. Representa- tive general-purpose gLMs include LucaOne and the Nucleotide Transformer series, which leverage multi- species genomic data to learn transferable sequence representations Dalla-Torre et al. (2025); He et al. (2025). Evo 2 further extends this paradigm by mod- eling genomic sequences across all domains of life with strong generative capabilityBrixi et al. (2025). However, most general-purpose gLMs rely on reference genomes or cross-species consensus signals, limiting their sensi- tivity to human population variation and disease-specific sequence heterogeneity. Therefore, this work builds on Genos, a human-centric genomic language founda- tion model trained on large-scale human population se- quencing data Lin et al. (2025), which has learned rep- resentations that are more sensitive to pathogenic vari- ants and human-specific regulatory patterns. The Attention-based Model Interpretation Attention mechanisms are intrinsic to Transformer ar- chitectures and provide an explicit, quantitative mea- sure of how models weight long-range dependencies across input sequences. In genomics, attention has been widely used as an interpretability tool to reveal biologically meaningful interactions learned during su- pervised training. For example, Enformer demonstrated that attention maps capture long-range regulatory in- teractions underlying gene expression prediction Avsec et al. (2021). Beyond supervised tasks, attention has also been applied to unsupervised structural discovery, including the recovery of transcription factor binding mo- tifs and the reconstruction of three-dimensional chro- matin contact maps directly from sequences Tomaz da Silva et al. (2025); Boshar et al. (2025). While exist- ing attention-based analyses predominantly focus on within-sequence interpretation for individual genomic sequences, our framework shifts attention analysis to- ward population-level comparison. Variant Effect Prediction Models Prior work has largely focused on supervised variant ef- fect prediction, estimating the impact of specific vari- ant classes. Many approaches excel by specializ- ing in defined biological mechanisms. For example, SpliceAI identifies splice-disrupting variants by model- ing sequence context but is limited to splicing regula- tion Jaganathan et al. (2019). Similarly, AlphaMissense predicts missense pathogenicity using protein language models, yet remains restricted to coding regions Cheng et al. (2023). Unlike these variant-centric methods that evaluate mutations individually, ATLAS offers a model- agnostic pipeline capable of detecting diverse events without task-specific supervision or predefined variant classes. Statistical and Bioinformatic Approaches Genome-wide association studies (GWAS) consti- tute the dominant statistical framework for genotype- phenotype analysis, but their power critically depends on allele frequency, effect size, and large cohort sizes, requiring 104–106 samples to reliably detect variants at modest (10%) frequencies Visscher et al. (2017). Region-based aggregation methods such as SKAT par- tially alleviate this limitation by combining signals across predefined genomic regions or genes, yet still require moderate to large sample sizes for stable inference Wu et al. (2011); Zhan et al. (2016). In parallel, classical bioinformatic tools (e.g., SIFT4G, PolyPhen-2) estimate variant effects using curated annotations and prior bi- ological knowledge, but are largely restricted to cod- ing single-nucleotide substitutions and struggle to cap- ture haplotype context or signals in poorly annotated re- gions Adzhubei et al. (2013); Vaser et al. (2016). To- gether, these limitations motivate alternative population- level frameworks that operate directly on sequence rep- resentations and are less dependent on explicit variant enumeration or extensive prior annotation. Liu et al. | | 2 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint S tep 1: Input D ata S tep 2: A ttention Extraction S tep 3: G ene-level A nalysis S tep 4: Base-level A nalysis & Cluster Control Ng0S equences H aplotype1 H aplotype2 Control Input Case Input H aplotype1 Case S equences M utations H aplotype2 gLM s m ult-head Ng0attention Ng0m atrix last layer attn. score colum n sum G ene 1 G ene 2 G ene N Case Control attn 1 attn 2 attn N attn 1 attn 2 attn N D istribution com parison A ttn 1 A ttn 1 A ttn 2 A ttn 2 A ttn N A ttn N Control Case elem ent-w ise Ng0average Control Case p=0.05 G enes -log10（p-value） Candidate gene Candidate gene: D NA sequence input S tatistical tests * * * A ttn-D iff log2FC K now n variants D iiferential Ng0A ttention Ng0A nalysis target Com m on Regions w ith target variants Cluster the high-confident loci log2FC attn. score Figure 1. Overview of the ATLAS workflow. Step 1: ATLAS operates on haplotype-resolved genomic sequences derived from case and control cohorts. Step 2: Each sequence is processed a genomic language model (e.g., Genos). Multi-head attention matrices are extracted from the final layer, averaged element-wise, and aggregated via column-wise summation to obtain nucleotide-level importance scores. Step 3: ATLAS prioritizes candidate genes by performing statistical tests on attention score distributions between cohorts, identifying genes that exhibit significant disease-associated attention fluctuations relative to the background. Step 4: Within candidate genes, the framework calculates differential attention scores to identify specific loci with significant signal divergence. High-confidence sites are finally clustered to delineate discrete risk-associated genomic regions.

In this section, we present the ATLAS pipeline and its benchmarking against GWAS (Figure 1). We first de- scribe how attention scores are extracted from genomic sequences using a pretrained genomic language model and how variable-length inputs are handled. We then in- troduce the gene-level differential attention analysis for identifying candidate genes. Next, we dive into these candidates, detecting disease-associated loci at base- level resolution and clustering them into regions of in- terest. Finally, we summarize the GWAS procedures and describe how GWAS results are used for compar- ative evaluation. The code of ATLAS is available at https://github.com/BGI-HangzhouAI/ATLAS. Retrieve and Calculate Attention Scores We extract attention scores from the final Transformer layer of each genomic language foundation model, with exceptions for specific architectures (e.g., the Evo 2 se- ries Brixi et al. (2025); see Supplementary Note 3). This approach is motivated by evidence that the final layer captures the most globally contextualized sequence in- formation Vig and Belinkov (2019) and serves as the primary interface for downstream biological tasks Marin et al. (2023). Consequently, we consistently employ final-layer attention to maximize the capture of high- level semantic features. Let the input sequence consist of L tokens. For each attention head h, we extract the query and key projec- tions, Q(h) and K(h), and apply Rotary Position Em- beddings (RoPE). We first define the pre-softmax atten- tion matrix A(h) incorporating the causal mask Mcausal. The final saliency score sj for the j-th position is then derived by averaging the attention probabilities across H heads and summing the attention weights received from all query positions (i.e., column-wise summation): A(h) = RoPE(Q(h))RoPE(K(h))⊤ √ d +M causal sj = 1 H L∑ i=1 H∑ h=1 softmax ( A(h) ) i,j (1) To efficiently compute attention scores across vary- ing sequence lengths, we adopt a three-tiered strat- egy based on sequence lengthL. These thresholds are empirically determined to balance scoring latency and GPU memory consumptions on an NVIDIA H100 (80GB) GPU, ensuring numerical stability and signal in- tegrity (see detailed benchmarks in Appendix C). •Vanilla attention (L≤4,096):For short se- quences, we explicitly materialize the full L×L attention matrix. Our benchmarks indicate that within this range, the computational overhead is manageable, allowing full-matrix operations to be Liu et al. | | 3 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint efficiently handled even on GPUs with more con- strained memory capacities, such as the NVIDIA A40 (48GB). •FlashAttention-based (4,096< L≤ 131,072): For sequences up to 128k (where 1k = 1, 024), we employ a FlashAttention-style algorithm Dao et al. (2022). This approach computes exact atten- tion scores using block-wise tiling and streaming softmax, effectively bypassing the O(L2) memory bottleneck while maintaining identical precision to vanilla attention. • Chunked processing (L > 131,072): For even longer sequences exceeding the hardware’s single-pass capacity (128k), where memory con- sumption and computational costs grow quadrati- cally, we apply a sliding-window chunking strategy. We empirically select a chunk size of 8,192 with a 4,096 overlap(Figure S1), as this configuration demonstrates the optimal balance between com- putational throughput and signal quality. To miti- gate boundary effects and preserve continuity, we discard the outer half of each overlap region and retain only the central region of each chunk for fi- nal reconstruction. Gene-wise Differential Attention Analysis We first identify candidate disease-associated genes using a gene-wise differential attention analysis, which compares attention distributions aggregated over de- fined genomic windows between control and disease cohorts. Prior work has shown that sequence variants can induce systematic and biologically meaningful per- turbations in attention patterns of genomic Transformer models, with detectable fluctuations in aggregated at- tention distributions Consens et al. (2025). Motivated by this evidence, we hypothesize that genes involved in disease will exhibit reproducible differences in attention distributions relative to controls. For each gene, we consider the entire gene body. The start and end positions are retrieved directly from the Ensembl database. We summarize the per-base atten- tion scores across these regions for each sample us- ing a set of complementary statistics designed to cap- ture central tendency, extremum, and dispersion (Sup- plementary Note 5). For each summary statistic, we compare the distributions between control and disease cohorts using a two-sided Mann–Whitney U test. To ac- count for multiple comparisons across the genome, we apply the Benjamini-Hochberg procedure to control the False Discovery Rate (FDR). Genes with FDR-adjusted p-values< 0.05are considered significant. Across our evaluations, Shannon Entropy consistently exhibits the strongest discrimination. We attribute this to the model’s sensitivity to risk syntax: rather than dis- persing attention, disease-associated variants act as strong attractors. This causes the model’s focus to systematically concentrate on specific loci, thereby re- ducing the distributional entropy compared to controls. Consequently, Shannon Entropy is emphasized in sub- sequent analyses. Base-wise Differential Attention Analysis To localize potentially disease-associated loci within the candidate genes prioritized in Section , we perform a base-level differential attention analysis. Drawing an analogy to differential expression analysis used to iden- tify disease signatures Rosati et al. (2024), we hypothe- size that high-risk loci induce systematic fluctuations in attention patterns between case and control cohorts. For each genomic position j, we aggregate attention scores across valid samples in each cohort. LetSc and Sd denote the sets of samples where positionj is effec- tively present (i.e., non-deleted and non-padding) in the control and disease groups, respectively. To rigorously handle Indels, we align comparisons to the reference coordinates: for deletions (e.g., AAAT →A), differen- tial attention is quantified at the retained anchor (start) position. Positions falling within the deleted span are excluded from Sg for the affected samples to prevent zero-inflation artifacts. We compute the base-wise log2 fold change (LFCj) as: LFCj = log2 (µd,j +ϵ µc,j +ϵ ) , whereµg,j = 1 |Sg| ∑ i∈Sg Ai,j (2) whereA i,j represents the attention score of sample i at position j, and ϵis a small pseudo-count to ensure numerical stability. The statistical significance of this difference is assessed using a two-sided Mann–Whitney U test, with p-values adjusted via the Benjamini–Hochberg (BH) procedure. A position is identified as ahighly differentiated attention locus if it satisfies two criteria: (1) adjusted p-value< 0.01; and (2) absolute LFCj exceeds a length-adaptive quantile threshold (qL). This dynamic thresholding strat- egy is critical for maintaining a constant false discov- ery rate across varying genomic contexts, as detailed in Section . Clustering and Delineating Disease-associated Re- gions We observe that differential attention signals do not al- ways strictly co-localize with target variants at single- base resolution; rather, they exhibit a proximal enrich- ment pattern, where significant attention fluctuations spatially cluster around disease-associated loci. Con- sequently, we aim to aggregate these discrete high- confidence signals into contiguous candidate risk re- gions. Adaptive Thresholding. To ensure consistent detec- tion sensitivity, the filtering threshold qL introduced in Liu et al. | | 4 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Section is formulated as a function of sequence length L. We empirically calibrate the baseline threshold on 4 kb sequences, identifying the 95th percentile (α= 0.05) as the optimal cutoff for effective signal isolation. How- ever, applying this fixed threshold to longer sequences

in elevated false discovery rates due to the ex- panded search space. To counteract this, we define qL (forL in kb) as: qL = 1−4α L·log10 (L 4 + 9 ) (3) This formulation is strictly grounded in our experimental observations: the term is designed to satisfy the bound- ary conditionq L = 1−α= 0.95at the baseline length (L= 4kb), while imposing progressively stricter filter- ing constraints (asymptotically approaching 1) for longer sequences to rigorously suppress the accumulation of

noise. Density-based Clustering.We apply DBSCAN Ester et al. (1996); Schubert et al. (2017) to the loci satisfy- ing the qL criterion, calculating clusters along 1D ge- nomic coordinates using Euclidean distance. We set the neighborhood radius ϵ= 20 bp, approximating the typical length of transcription factor binding sites or reg- ulatory motifs, and min_samples = 5 to ensure that de- tected clusters represent robust, multi-base signal ac- cumulations rather than isolated artifacts. Benchmark with GWAS To establish a baseline comparison with traditional sta- tistical methods, we performed genome-wide associa- tion studies (GWAS) on genotype data from 1,429 β- thalassemia carriers and patients (details in Supple- mentary Note 2 A). We implemented a stringent quality control (QC) pipeline using VCFtools Danecek et al. (2011). First, multi-allelic variants were split into biallelic records, and chromosome identifiers were standardized. We then applied genotype-level filtering to set genotypes with sequencing depth (DP) < 4 to missing. Subsequently, variant-level filtering was performed to retain only auto- somal loci with a missingness rate < 15% and a minor allele frequency (MAF)≥1%. The post-QC dataset was converted to PLINK binary format using plink2 Chang et al. (2015). We performed a case–control association analysis using a logistic re- gression model to test the additive effect of each vari- ant on the binary disease phenotype. To control for false positives, statistical significance was assessed us- ing the conventional genome-wide significance thresh- old (p< 6.3×10−9). Experiment Design Synthetic Datasets To systematically evaluate ATLAS’s robustness under controlled conditions—particularly for small cohorts and low allele frequencies—we constructed a series of syn- thetic datasets using background noise derived from the realβ-thalassemia cohort (details in Supplementary Note 2 A). We sampled four genomic regions of increasing length (4 kb, 20 kb, 128 kb, and 384 kb) from the reference genome, explicitly excluding the HBB locus to prevent data leakage. Within each region, synthetic target vari- ants were inserted as ground truth, simulating increas- ing complexity: from 8 SNPs in 4 kb regions to a mix- ture of 20 variants (15 SNPs + 5 Indels) in larger win- dows. Genotypes were assigned based on disease sta- tus: cases were modeled as homozygous mutants (1|1) and controls retained the reference genotype (0|0). We designed two experimental settings to stress-test the model: • Allele Frequency (AF) Sensitivity: In the 4 kb re- gion, we simulated lower penetrance by randomly downsampling mutant genotypes (1|1→0|0) in case samples to target frequencies of 70%, 50%, 20%, and 10%, while keeping background sites un- changed. • Sample Efficiency: We varied the balanced case–control cohort sizes from 200:200 down to 10:10, fixing the penetrance of synthetic target vari- ant at 100%. Sequence construction followed a strict coordinate mapping pipeline to handle Indel-induced shifts (see Supplementary Note 2 B). Thalassemia Cohort and Data Preprocessing We designed this real-world evaluation to assess whether ATLAS can recover known risk loci under real- istic population heterogeneity and, crucially, to identify plausible disease-associated signals beyond curated annotations. The β-thalassemia dataset was derived from a cross- sectional whole-genome sequencing study investigating the clinical heterogeneity of hemoglobinopathies com- prising 1,429 individuals. For validation, we referenced the IthaGenes database Kountouris et al. (2014), which documents 512 thalassemia-associated variants in the HBB gene. Within our cohort, 31 of these recorded vari- ants were identified. We selected the 8 most preva- lent variants routinely used in clinical diagnosis as the ground-truth set for our primary performance bench- marks Writing Group For Practice Guidelines For Di- agnosis And Treatment Of Genetic Diseases Medical Genetics Branch Of Chinese Medical Association et al. (2020). Raw genotypes were processed to generate model- ready sequences. First, VCF records were phased using Beagle4 Browning and Browning (2007) to ob- tain haplotype-resolved genotypes. Subsequently, we reconstructed full haplotype sequences for each indi- vidual by applying variants to the GRCh38 reference Liu et al. | | 5 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint genome. Specific rules for handling Indels, coordinate mapping, and reverse-complementation for negative- strand genes are detailed in Supplementary Note 2 B. Genome-Wide Scanning on Chromosome 11.To eval- uate beyond knownHBB mutations, we extended the analysis to all protein-coding genes on chromosome 11. This genome-wide scan aimed to: (i) identify pre- viously uncharacterized loci; and (ii) evaluate method specificity, under the expectation that non–hematology- related genes should exhibit minimal differential atten- tion signals. Evaluation Metric Base-Resolution Localization Accuracy. To evaluate localization performance at single-base resolution, we measure the extent to which attention-derived signals concentrate near known disease-associated variant po- sitions. We posit that informative signals should prefer- entially localize in the immediate vicinity of causal vari- ants; distal signals are considered less actionable for downstream validation. Given the extreme class imbalance between causal variants (rare) and background bases (abundant), we adopt the Area Under the Precision–Recall Curve (AUPRC) as our primary metric. We utilize absolute log2 fold-change (log 2 FC) values as scores, treating bases within fixed windows around each variant as pos- itive labels. To provide a comprehensive assessment of signal quality beyond ranking, we report three comple- mentary metrics: • Signal-to-Noise Ratio (SNR): Measuring the magnitude contrast between signal and back- ground regions. • Fraction of Signal in Windows (FRiW):Quantify- ing the efficiency of attention mass allocation (anal- ogous to ChIP-seq FRiP scores). • Weighted Distance (WDist): Assessing spatial precision without imposing hard window bound- aries. For cross-scale comparisons (e.g., 4 kb vs. 384 kb se- quences), we utilize length-normalized variants of FRiW and WDist, alongside the mean percentile rank of true positions. Detailed mathematical definitions are pro- vided in Supplementary Note 4. Cluster-Level Recovery. We further evaluate perfor- mance at the region level by assessing whether the contiguous clusters identified by our pipeline (Section ) successfully overlap with known variant loci. For syn- thetic datasets, we compute both precision and recall to measure the recovery of predefined variants and the suppression of spurious clusters. For real-world cohorts (e.g., β-thalassemia), where ground-truth an- notations are inherently incomplete, we prioritize re- call—quantifying the coverage of clinically validated loci. Clusters lacking overlap with annotated variants are not strictly penalized as false positives, as they may repre- sent novel, uncharacterized biological signals. Table 1. Biological evidence for top-ranked β-thalassemia associated genes identified by ATLAS. Gene Biological Relevance HBMSβ-thalassemia associated gene inferred by GeneCard Stelzer et al. (2016) and DISEASE databases Grissa et al. (2022). OR52A1 Contains the γ-globin enhancer; variants in this region can affect erythropoiesis and mod- ulate thalassemia phenotypes Himadewi et al. (2021). HBG1 A key modifier gene encoding γ-globin. Up- regulation (↑HbF) ameliorates clinical severity in β-thalassemia. HBD Encodes δ-globin. Co-inherited variants or al- tered HbA2 levels are diagnostic hallmarks for β-thalassemia carriers.

In this section, we present the evaluation of ATLAS in three parts. First, we demonstrate its practical efficacy by identifying disease-associated genes and localizing fine-grained signals in a real-world β-thalassemia co- hort. Second, we validate the method’s robustness un- der controlled conditions using synthetic datasets with varying cohort sizes and signal strengths. Finally, we benchmark different foundation models to reveal how model scale and data diversity impact disease localiza- tion performance. Gene-level differential attention identifies disease- associated genes Our gene-level differential attention analysis success- fully prioritizes the HBB gene as the top candidate on chromosome 11, demonstrating the model’s capacity to distinguish disease-associated signals from the ge- nomic background. As illustrated in Figure 2, HBB dominates the rankings across both haplotypes, achiev- ing significance levels (−log10padj) that are 6.36×and 9.60×higher than the second-ranked candidates on haplotype 1 and haplotype 2, respectively. Specifically, disease samples exhibite a sharp reduction in Shan- non entropy within the HBB locus compared to con- trols (padj< 10−32, Figure 3). This significant drop indi- cates that the model’s attention—typically dispersed in controls—becomes systematically focused on specific disease-associated sites in the disease state. This pat- tern proves robust across multiple distributional metrics, including standard deviation and coefficient of variation (Figure S5 and S6). Beyond the expected HBB signal, the analysis demon- strates high specificity. Among the top candidates, only 15 other genes surpass the significance thresh- old (p adj < 0.05) across the entire chromosome. Cru- Liu et al. | | 6 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint 0 10 20 30 40 -log10(p-value) HBB HMBS OR52A1 OR52E2 OR52E1 OR52J3 OR4D9 HBG1 OR51C1P OR51B4 HBD OR51F1 NDUFC2 CCDC88B TRIM77 OR51Q1 ENSG00000284931 PATL1 OR51I2 OR4D11 Shannon Entropy Haplotype 1 Adjusted p=0.05 0 5 10 15 20 25 30 -log10(p-value) HBB OR52A1 OR51B4 CCKBR ENSG00000254979 OR52J3 FGF19 HMBS OR52E1 OR52E2 PATL1 OR52L1 OR52N4 ZFTA OR51C1P CD5 CEP295 DENND5A OR2AT4 OR52M1 Shannon Entropy Haplotype 2 Adjusted p=0.05 Figure 2.Top-20 genes with differently distributed attentions quantified in en- tropy. The bar plots rank the top-20 protein-coding genes on chromosome 11 by−log10(p)values from Wilcoxon rank-sum tests for Shannon entropy, high- lighting HBB as the most significant gene. cially, four of these genes—HBG1,HBD,HBMS, and OR52A1—are biologically validated as thalassemia modifiers or hemoglobin regulators (Table 1). The recovery of these secondary but functionally relevant genes, amidst a low false-positive background, confirms that differential attention entropy serves as an effective, zero-shot filter for isolating disease-relevant loci prior to fine-grained mapping. Case Control 7.338 7.340 7.342 7.344 7.346 7.348Shannon Entropy HBB Haplotype 1 Case Control 7.338 7.340 7.342 7.344 7.346 7.348 HBB Haplotype 2 Figure 3.Comparison of Shannon entropy values of HBB in case and control groups. Both haplotypes show significantly lower entropy values in the case group compared to the control group. Synthetic validation of base-level localization accu- racy We evaluate ATLAS on synthetic datasets to quantify robustness under varying conditions, reporting metrics averaged across window sizes and haplotypes. The 4 kb dataset with 100% allele frequency and 1,429 sam- ples serves as the primary baseline. ATLAS demonstrates high resilience to data sparsity and rare variants (Table 2, Figure S3). Notably, clear signal concentration around synthetic target sites re- mained observable even at 10% allele frequency or with as few as 10 individuals per group. Base-level metrics (AUPRC, SNR, FRiW) show minimal degrada- tion (<10%) under these challenging conditions, while cluster-level precision and recall consistently remained above 0.80. This confirms that ATLAS can reliably pri- oritize rare risk variants even in small-scale studies. Table 2. Base-level localization and cluster-level recovery performance on 4 kb synthetic sequences under varying cohort sizes and allele frequencies. Cohor t Allele AUPRC↑SNR↑FRiW↑Prec.↑Recall↑Size (N) Freq. Eff ect of Allele Frequency (Cohort Size = 1,429) 1,429 10% 0.350 9.430 0.150 0.845 0.688 1,429 20% 0.369 10.041 0.159 0.875 0.875 1,429 50% 0.385 10.028 0.160 0.889 1.000 1,429 70% 0.384 9.731 0.157 0.889 1.000 1,429 100% 0.382 9.213 0.150 0.845 1.000 Eff ect of Cohort Size (Allele Frequency = 100%) 200 100% 0.382 9.193 0.150 0.845 1.000 100 100% 0.382 9.124 0.149 0.889 1.000 50 100% 0.382 9.111 0.149 0.889 1.000 20 100% 0.380 9.084 0.148 0.800 1.000 10 100% 0.384 9.078 0.148 0.845 1.000 Signal localization scales effectively across genomic contexts. As shown in Table 3, we extend the evaluation to sequences ranging from 4 kb to 384 kb. Length-normalized metrics demonstrate consis- tent performance: FRiW Enrichment scales with se- quence length, indicating effective signal isolation against expanding genomic backgrounds, while Nor- malized Weighted Distance remained consistently low (< 0.4). Notably, for the longest sequences (384 kb), cluster-level recovery achieves near-perfect scores (Precision/Recall≈1.0), validating the pipeline’s capa- bility to detect sparse signals in large genomic windows (Figure S4). Table 3. Performance scaling across increasing sequence lengths (4 kb to 384 kb). Seq. FRiW Normalized Mean Prec.↑Recall↑Length Enrich. ↑ WDist↓ Rank↑ 4 kb 8.03 0.373 0.926 0.844 1.000 20 kb 19.04 0.328 0.908 0.909 1.000 128 kb 63.41 0.252 0.908 0.950 0.950 384 kb 129.82 0.325 0.915 1.000 1.000 Base-level discovery of disease-associated regions We next apply base-level differential attention analysis to top candidate genes, where ATLAS demonstrates su- perior recall of known disease-associated variants in HBBcompared to standard GWAS. We compare the detected attention clusters against both GWAS results (p<6.3×10−9) and the 31 clinical variants reported in IthaGenes. As illustrated in Figure 4, while GWAS iden- tify only a single locus overlapping with known variants, ATLAS detects 4 distinct clusters on haplotype 1 (cov- ering 7 reported sites) and 3 clusters on haplotype 2 (covering 6 reported sites). This confirms that attention- derived signals can recover a substantially larger pro- portion of clinically reported variants that are statistically elusive to standard association mapping. Extending the analysis to the other 15 candidate genes, ATLAS shows strong agreement with standard methods while offering superior spatial resolution. Our method successfully recovers all GWAS-significant loci (e.g., in Liu et al. | | 7 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Figure 4.Comparision of the derived cluster, known loci, and GWAS inference on both haplotypes. Here we visualize the four genes that both GWAS and ATLAS have found. The gray regions are the clusters with their cluster score showed as the colored dots. The red lines are the known loci. The orange star is the GWAS inferred locus. OR52A1,HBD) and identifies latent signals in 6 addi- tional genes. Notably, the detected clusters span an average of only 11.62 bp. This fine-grained localization significantly narrows down the search space for putative risk variants, providing much tighter candidate windows compared to typical LD-based association blocks. Fur- thermore, ATLAS leverages haplotype information to re- veal phase-specific patterns—such as the asymmetric attention clusters inHBD —which are typically obscured in population-based GWAS signals. Impact of model capacity and pretraining diversity on loci discovery Furthermore, we explore how model architecture and pretraining data influence the localization of human dis- ease variants. We benchmark models with varying scales and training domains—including the Evo2 series, Luca series, and Genos family—on the β-thalassemia cohort (Table 4). First, our observations indicate that model capacity is a significant factor in signal quality, particularly when the domain aligns. Within the human-centric Genos fam- ily, scaling up parameters yielded consistently cleaner signals. As shown in Table 4, Genos-v2 substantially improves metrics over Genos-1.2B (AUPRC: 0.173→ 0.265; SNR: 3.24→4.94), suggesting that larger mod- els benefit from enhanced semantic denoising capabili- ties in non-informative regions. Second, the results also suggest that scaling parame- ters alone may not guarantee performance gains if the pretraining data is not sufficiently aligned with the target task. This is notable in the comparison with the gener- alist Evo2-40B model: despite its substantial parameter count, it does not exhibit a commensurate performance advantage in localizing human pathogenic variants (Avg AUPRC≈0.093), performing similarly to smaller base- lines. In contrast, Genos-v2, which benefits from exten- sive human genome diversity, consistently outperforms generalist counterparts. The findings imply that for specific human disease tasks, achieving reliable localization performance likely requires a synergy between sufficient model capacity and diverse human-centric pretraining. Table 4. Foundation-model comparison onβ-thalassemia (HBB) under ATLAS. Model Pretr ain Avg. Avg. Avg. Avg. domain AUC↑AUPRC↑SNR↑FRiW↑ Genos-v2 Human (diverse) 0.807 0.265 4.941 0.169 Genos-v1 Human 0.804 0.258 4.496 0.157 Genos-1.2b Human 0.777 0.173 3.237 0.121 Evo2-7b Generalist 0.712 0.147 2.566 0.096 Evo2-1b Generalist 0.753 0.141 2.782 0.107 Evo2-40b Generalist 0.709 0.093 2.139 0.082 LucaOne Generalist 0.687 0.094 2.148 0.081 LucaVirus Virus 0.695 0.087 1.908 0.074 Discussions Conclusion.We present ATLAS, a framework that es- tablishes a new paradigm for population-level discovery through the lens of genomic language models. By de- coding internal attention patterns, ATLAS bypasses the

of traditional frequency-based statistics, en- abling the discovery of disease-associated loci directly from sequence semantics. Our results confirm that identifying risk variants does not strictly require massive cohorts or clear statistical sepa- ration; rather, actionable signals can be extracted from the model’s intrinsic understanding of genomic syntax. ATLAS thus positions itself not merely as a complement to GWAS, but as a critical instrument for illuminating the "genetic dark matter"—including rare variants, com- plex haplotypes, and non-coding regions—that remains inaccessible to conventional methods. Ultimately, this work paves the way for a future where genomic lan- guage models become the standard engine for decod- ing the complex syntax of human disease. Liuet al.| | 8 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint

and Future Work.First, this study focuses on binary phenotypes driven by major genes; extending ATLAS to complex, polygenic traits remains a key fu- ture direction. Second, our analysis currently relies on reconstructed genomes with inherent processing noise. As high-fidelity sequencing data becomes more acces- sible, ATLAS is expected to leverage these improved in- puts to naturally enhance localization precision without architectural changes. Finally, while ATLAS effectively prioritizes candidate loci, establishing definitive causal- ity warrants further empirical verification. Future work will focus on validating these computational predictions through large-scale functional assays. Ethics Statement Theβ-thalassemia dataset utilized in our research are from the study approved by the ethics committee of Nanfang Hospital, Southern Medical University, and the ethical committees of each local hospital participating in this study (Approval No. NFEC-2019-039). All sub- jects and/or their guardians provided written informed consent. The data supporting the findings of this study will be made available upon reasonable request to the corresponding author, subject to approval by the original data custodian(s).

Adzhubei, I., Jordan, D. M., and Sunyaev, S. R. (2013). Predicting functional effect of human missense mutations using PolyPhen-2.Curr. Protoc. Hum. Genet., Chapter 7(1):Unit7.20. Alkan, C., Coe, B. P ., and Eichler, E. E. (2011). Genome structural variation discovery and genotyping. Nat. Rev. Genet., 12(5):363–376. Avsec, , Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y ., Jumper, J., Kohli, P ., and Kelley, D. R. (2021). Effective gene expression prediction from sequence by integrating long-range interactions. Nat. Methods, 18(10): 1196–1203. Benegas, G., Batra, S. S., and Song, Y . S. (2023). DNA language models are power- ful predictors of genome-wide variant effects. Proc. Natl. Acad. Sci. U. S. A., 120(44): e2311219120. Boshar, S., Evans, B., Tang, Z., Picard, A., Adel, Y ., Lorbeer, F . K., Rajesh, C., Karch, T., Sidbon, S., Emms, D., Mendoza-Revilla, J., Al-Ani, F ., Seitz, E., Schiff, Y ., Bornachot, Y ., Hernandez, A., Lopez, M., Laterre, A., Beguir, K., Koo, P ., Kuleshov, V., Stark, A., de Almeida, B. P ., and Pierrot, T. (2025). A foundational model for joint sequence-function multi-species modeling at scale for long-range genomic prediction. bioRxiv. Brixi, G., Durrant, M. G., Ku, J., Poli, M., Brockman, G., Chang, D., Gonzalez, G. A., King, S. H., Li, D. B., Merchant, A. T., Naghipourfar, M., Nguyen, E., Ricci-Tam, C., Romero, D. W., Sun, G., Taghibakshi, A., Vorontsov, A., Y ang, B., Deng, M., Gorton, L., Nguyen, N., Wang, N. K., Adams, E., Baccus, S. A., Dillmann, S., Ermon, S., Guo, D., Ilango, R., Janik, K., Lu, A. X., Mehta, R., Mofrad, M. R. K., Ng, M. Y ., Pannu, J., Re, C., Schmok, J. C., St. John, J., Sullivan, J., Zhu, K., Zynda, G., Balsam, D., Collison, P ., Costa, A. B., Hernandez-Boussard, T., Ho, E., Liu, M.-Y ., McGrath, T., Powell, K., Burke, D. P ., Goodarzi, H., Hsu, P . D., and Hie, B. (2025). Genome modeling and design across all domains of life with evo 2. bioRxiv. Browning, S. R. and Browning, B. L. (2007). Rapid and accurate haplotype phasing and missing-data inference for whole-genome association studies by use of localized haplo- type clustering. Am. J. Hum. Genet., 81(5):1084–1097. Chang, C. C., Chow, C. C., Tellier, L. C., Vattikuti, S., Purcell, S. M., and Lee, J. J. (2015). Second-generation PLINK: rising to the challenge of larger and richer datasets. Giga- science, 4(1):7. Cheng, J., Novati, G., Pan, J., Bycroft, C., Žemgulyt ˙e, A., Applebaum, T., Pritzel, A., Wong, L. H., Zielinski, M., Sargeant, T., Schneider, R. G., Senior, A. W., Jumper, J., Hassabis, D., Kohli, P ., and Avsec, (2023). Accurate proteome-wide missense variant effect prediction with AlphaMissense. Science, 381(6664):eadg7492. Consens, M. E., Diaz-Navarro, A., Chu, V., Stein, L., He, H. H., Moses, A., and Wang, B. (2025). Interpreting attention mechanisms in genomic transformer models: a framework for biological insights. bioRxiv, pages 2025–06. Cutting, G. R. (2015). Cystic fibrosis genetics: from molecular understanding to clinical application. Nat. Rev. Genet., 16(1):45–56. Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A. H., Oteri, F ., Dallago, C., Trop, E., de Almeida, B. P ., Sirelkhatim, H., Richard, G., Skwark, M., Beguir, K., Lopez, M., and Pierrot, T. (2025). Nucleotide transformer: build- ing and evaluating robust foundation models for human genomics. Nat. Methods, 22(2): 287–297. Danecek, P ., Auton, A., Abecasis, G., Albers, C. A., Banks, E., DePristo, M. A., Handsaker, R. E., Lunter, G., Marth, G. T., Sherry, S. T., McVean, G., Durbin, R., and 1000 Genomes Project Analysis Group. (2011). The variant call format and VCFtools. Bioinformatics, 27 (15):2156–2158. Dao, T., Fu, D. Y ., Ermon, S., Rudra, A., and Ré, C. (2022). FlashAttention: Fast and memory-efficient exact attention with IO-awareness. arXiv [cs.LG]. Ester, M., Kriegel, H., Sander, J., and Xu, X. (1996). A density-based algorithm for discover- ing clusters in large spatial databases with noise. KDD, pages 226–231. Grissa, D., Junge, A., Oprea, T. I., and Jensen, L. J. (2022). Diseases 2.0: a weekly updated database of disease-gene associations from text mining and data integration. Database (Oxford), 2022(baac019). He, Y ., Fang, P ., Shan, Y ., Pan, Y ., Wei, Y ., Chen, Y ., Chen, Y ., Liu, Y ., Zeng, Z., Zhou, Z., Zhu, F ., Holmes, E. C., Y e, J., Li, J., Shu, Y ., Shi, M., and Li, Z. (2025). Generalized biological foundation model with unified nucleic acid and protein language. Nat. Mach. Intell., 7(6): 942–953. Himadewi, P ., Wang, X. Q. D., Feng, F ., Gore, H., Liu, Y ., Yu, L., Kurita, R., Nakamura, Y ., Pfeifer, G. P ., Liu, J., and Zhang, X. (2021). 3’HS1 CTCF binding site in humanβ-globin locus regulates fetal hemoglobin expression. Elife, 10(e70557). Jaganathan, K., Kyriazopoulou Panagiotopoulou, S., McRae, J. F ., Darbandi, S. F ., Knowles, D., Li, Y . I., Kosmicki, J. A., Arbelaez, J., Cui, W., Schwartz, G. B., Chow, E. D., Kanterakis, E., Gao, H., Kia, A., Batzoglou, S., Sanders, S. J., and Farh, K. K.-H. (2019). Predicting splicing from primary sequence with deep learning. Cell, 176(3):535–548.e24. Klein, R. J., Zeiss, C., Chew, E. Y ., Tsai, J.-Y ., Sackler, R. S., Haynes, C., Henning, A. K., SanGiovanni, J. P ., Mane, S. M., Mayne, S. T., Bracken, M. B., Ferris, F . L., Ott, J., Barnstable, C., and Hoh, J. (2005). Complement factor H polymorphism in age-related macular degeneration. Science, 308(5720):385–389. Kountouris, P ., Lederer, C. W., Fanis, P ., Feleki, X., Old, J., and Kleanthous, M. (2014). IthaGenes: an interactive database for haemoglobin variations and epidemiology. PLoS One, 9(7):e103020. Lin, A., Xie, B., Y e, C., Wang, C., Chen, D., Wang, E., Lu, F ., Xue, G., Zhang, H., Zhan, J., Zhang, J., Pang, J., Liang, J., Lin, J., Ma, J., Hu, J., Ma, J., Dong, J., Li, J., Liu, J., Chen, J., Li, J., Ding, K., Deng, K., Chen, K., Wang, L., Liu, L., Guo, L., Xiong, L., Y ang, L., Cheng, M., Chen, N., Chen, R., Sun, S., Li, S., Chen, S., Liu, S., Xie, S., Liu, S., Zhou, T., Tang, W., Zhang, W., Jiang, X., Qi, X., Jin, X., Tan, X., Hu, X., Xu, X., Feng, X., Lu, Y ., Gao, Y ., Shang, Y ., He, Y ., Yuan, Y ., Wang, Y ., Liu, Y ., Xiao, Z., Meng, Z., Li, Z., Zhao, Z., Y ang, Z., and Wang, Z. (2025). Genos: A human-centric genomic foundation model. Gigascience, (giaf132). Marin, F . I., Teufel, F ., Horlacher, M., Madsen, D., Pultz, D., Winther, O., and Boomsma, W. (2023). BEND: Benchmarking DNA language models on biologically meaningful tasks. arXiv [q-bio.GN]. Montella, A., Tirelli, M., Lasorsa, V. A., Aievola, V., Cerbone, V., Manganiello, R., Iolascon, A., and Capasso, M. (2025). Regulatory non-coding somatic mutations as drivers of neuroblastoma. Br. J. Cancer, 132(5):469–480. Pagel, K. A., Antaki, D., Lian, A., Mort, M., Cooper, D. N., Sebat, J., Iakoucheva, L. M., Mooney, S. D., and Radivojac, P . (2019). Pathogenicity and functional impact of non- frameshifting insertion/deletion variation in the human genome. PLoS Comput. Biol., 15 (6):e1007112. Risch, N. and Merikangas, K. (1996). The future of genetic studies of complex human diseases. Science, 273(5281):1516–1517. Rosati, D., Palmieri, M., Brunelli, G., Morrione, A., Iannelli, F ., Frullanti, E., and Giordano, A. (2024). Differential gene expression analysis pipelines and bioinformatic tools for the identification of specific biomarkers: A review. Comput. Struct. Biotechnol. J., 23: 1154–1168. Schubert, E., Sander, J., Ester, M., Kriegel, H. P ., and Xu, X. (2017). DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN. ACM Trans. Database Syst., 42 (3):1–21. Sollis, E., Mosaku, A., Abid, A., Buniello, A., Cerezo, M., Gil, L., Groza, T., Güne¸ s, O., Hall, P ., Hayhurst, J., Ibrahim, A., Ji, Y ., John, S., Lewis, E., MacArthur, J. A. L., McMahon, A., Osumi-Sutherland, D., Panoutsopoulou, K., Pendlington, Z., Ramachandran, S., Stefanc- sik, R., Stewart, J., Whetzel, P ., Wilson, R., Hindorff, L., Cunningham, F ., Lambert, S. A., Inouye, M., Parkinson, H., and Harris, L. W. (2023). The NHGRI-EBI GWAS catalog: knowledgebase and deposition resource. Nucleic Acids Res., 51(D1):D977–D985. Stelzer, G., Rosen, N., Plaschkes, I., Zimmerman, S., Twik, M., Fishilevich, S., Stein, T. I., Nudel, R., Lieder, I., Mazor, Y ., Kaplan, S., Dahary, D., Warshawsky, D., Guan-Golan, Y ., Kohn, A., Rappaport, N., Safran, M., and Lancet, D. (2016). The GeneCards suite: From gene data mining to disease genome sequence analyses. Curr. Protoc. Bioinformatics, 54(1):1.30.1–1.30.33. Tewhey, R., Bansal, V., Torkamani, A., Topol, E. J., and Schork, N. J. (2011). The importance of phase information for human genomics. Nat. Rev. Genet., 12(3):215–223. Tomaz da Silva, P ., Karollus, A., Hingerl, J., Galindez, G. S. T., Wagner, N., Hernandez-Alias, X., Incarnato, D., and Gagneur, J. (2025). Nucleotide dependency analysis of genomic language models detects functional elements. Nat. Genet., 57(10):2589–2602. Turner, T. N. and Eichler, E. E. (2019). The role of DE novo noncoding regulatory mutations in neurodevelopmental disorders. Trends Neurosci., 42(2):115–127. Vaser, R., Adusumalli, S., Leng, S. N., Sikic, M., and Ng, P . C. (2016). SIFT missense predictions for genomes. Nat. Protoc., 11(1):1–9. Vig, J. and Belinkov, Y . (2019). Analyzing the structure of attention in a transformer language model. arXiv [cs.CL]. Visscher, P . M., Wray, N. R., Zhang, Q., Sklar, P ., McCarthy, M. I., Brown, M. A., and Y ang, J. (2017). 10 years of GWAS discovery: Biology, function, and translation. Am. J. Hum. Genet., 101(1):5–22. Writing Group For Practice Guidelines For Diagnosis And Treatment Of Genetic Diseases Medical Genetics Branch Of Chinese Medical Association, Shang, X., Wu, X., Zhang, X., Feng, X., and Xu, X. (2020). Clinical practice guidelines for beta-thalassemia. Zhonghua Yi Xue Yi Chuan Xue Za Zhi, 37(3):243–251. Liu et al. | | 9 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Wu, M. C., Lee, S., Cai, T., Li, Y ., Boehnke, M., and Lin, X. (2011). Rare-variant association testing for sequencing data with the sequence kernel association test. Am. J. Hum. Genet., 89(1):82–93. Zhan, X., Hu, Y ., Li, B., Abecasis, G. R., and Liu, D. J. (2016). RVTESTS: an efficient and comprehensive tool for rare variant association analysis using sequence data. Bioinfor- matics, 32(9):1423–1426. Liu et al. | | 10 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Supplementary Note 1: Implementation Details & Reproducibility To ensure the reproducibility of our experiments and the scalability of our approach to long-context genomic se- quences, we provide detailed specifications of our computational environment, attention extraction algorithms, and hyperparameter optimization strategies. A. Computational Environment All models were trained and evaluated on a high-performance computing node optimized for large-scale deep learn- ing. •Hardware Infrastructure: A single node equipped with 4×NVIDIA A40 GPUs (48GB VRAM per GPU). This setup utilizes the Ampere architecture to support efficient bfloat16 mixed-precision training. • Software Stack: – Framework: PyTorch2.5.1+cu124 with Python 3.12.2. – Acceleration: FlashAttention-2 (v2.8.3) was employed to optimize the memory hierarchy (HBM vs. SRAM), significantly reducing the IO overhead for attention matrix computation. – CUDA Toolkit: Version12.4. B. Attention Score Extraction & Alignment Unlike standard NLP tasks where token indices align linearly with positions, genomic analysis requires precise mapping between attention weights and biological coordinates, especially in the presence of Indels. B.1. FlashAttention-based Extraction Logic. We extract raw per-token attention importance scores directly from the Transformer layers. To handle the quadratic complexity of attention in long sequences, we implement a block-wise extraction algorithm based on FlashAttention logic. The detailed procedure is formally described in Algorithm 1. Algorithm 1FlashAttention-based Per-token Attention Extraction 1:Input:Sequence S, modelfθ, block sizeBr, causal flagC 2: Output: Attention importance vector v∈RL 3: Q∈RB×Hq×L×D, K∈RB×Hkv×L×D←Hook(fθ,S) 4: ifHkv<Hq then 5: K←Repeat(K,Hq/Hkv) {Broadcast KV heads to Match Q} 6: end if 7: Q, K←ApplyRoPE(Q, K,pos_ids) {Apply rotary embeddings} 8: m←[−∞]L, l←[0]L, c←0L 9: fori = 0,Br,2Br,...,L do 10: i′←min(i +Br,L) 11: Si←1√ D Q[:,:,i:i′,:]K⊤ 12:ifC then 13: Si←Mask(Si) 14: end if 15: mnew←max(mi:i′,rowmax(Si)) 16: li:i′←li:i′⊙emi:i′−mnew + rowsum(eSi−mnew) 17: mi:i′←mnew 18: end for 19: fori = 0,Br,2Br,...,L do 20: i′←min(i +Br,L) 21: Pi←exp( 1√ D Q[:,:,i:i′,:]K⊤−mi:i′)⊘li:i′ 22:c← c+∑ head ∑ row Pi 23: end for 24: v←c/Hq 25: return v Liuet al.| Supplementary Information | 11 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint B.2. Dynamic Coordinate Alignment.Post-extraction, the vector v corresponds to token indices. To map these to genomic coordinatesp, we apply the following rules to handle variants (Indels): • Insertions (Shared Coordinates): Tokens generated by an insertion share the genomic coordinate of the preceding anchor base. The attention score for coordinate p is calculated as the aggregation of all tokens mapped top. • Deletions (Placeholder Encoding): Deleted regions retain their genomic keys in the coordinate map but gen- erate no tokens. This ensures that the relative distance of downstream tokens remains biologically consistent. C. Long-Sequence Strategy and Hyperparameter Ablation C.1. Empirical Benchmarking on NVIDIA H100. The strategy transition thresholds (4k and 128k) were established through rigorous benchmarking on a single NVIDIA H100 (80GB) GPU. Scalability Constraints. As detailed in Table5, Vanilla Attentionmemory usage grows quadratically. While efficient for short contexts, it reaches 53.7GB at L = 8,192 and triggers Out-of-Memory (OOM) errors at 32k. Conversely, FlashAttention (Table 6) maintains linear memory scaling, enabling processing up to 131,072 (128k) bp. However, at 128k, the latency per sequence rises to15.47s, and memory usage hits42.9GB. To ensure robust throughput and prevent instability at extreme lengths, we defineL = 128k as the upper bound for exact global attention, switching to chunked processing thereafter. Table 5. Computational Cost of Vanilla Attention (Single H100) # GPUs Length (L) Latency (s) Memory (MB) 1 2,048 0.2359 21,967 1 4,096 0.4221 28,425 1 8,192 1.1600 53,677 1 32,768 OOM OOM Table 6.Computational Cost of FlashAttention (Single H100) # GPUs Length (L) Latency (s) Memory (MB) 1 8,192 0.2272 21,070 1 32,768 1.3582 25,423 1 131,072 15.4657 42,929 C.2. Chunking Hyperparameter Optimization.For the chunking strategy (L> 128k), we performed an ablation study to select the optimal window sizeC and overlapO. We measured the Area Under the Precision-Recall Curve (AUPRC) as a proxy for the model’s Signal-to-Noise Ratio (SNR) in detecting regulatory variants. As shown in Figure S1, we evaluated chunk sizes ranging from 4k to 32k. The configuration of C = 8,192 (Chunk Size) and O = 4,096 (Overlap) yielded the highest AUPRC across multiple sequence lengths (green line). While larger chunks (32k) theoretically offer more context, our analysis suggests they introduce excessive background noise that dilutes the local signal, in addition to higher computational overhead (see Figure S2). Conversely, smaller chunks (4k) fracture long-range dependencies essential for distal regulation. Thus, the 8k/4k setup provides the optimal trade-off between signal fidelity and computational cost. Supplementary Note 2: Data & Experiments Details A. Real Dataset Characteristics A.1. Sample Classification and Ground Truth. We designed this real-world evaluation to assess whether ATLAS can recover known related loci under realistic population heterogeneity and identify plausible disease-associated signals beyond curated annotations. Theβ-thalassemia dataset was retrieved from a cross-sectional whole-genome sequencing study investigating the clinical heterogeneity of hemoglobinopathies. The cohort comprises 1,429 individuals, classified into three groups based on clinical severity: • Carriers (N = 409): Individuals who generally exhibit no overt clinical symptoms, although mild anemia and microcytosis may be observed in hematological tests. Liu et al. | Supplementary Information | 12 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Figure S1. Ablation Study - Performance Analysis (AUPRC).Comparison of variant detection accuracy across varying chunk sizes and overlaps. The 8k chunk size with 4k overlap (green line) consistently shows the optimal trade-off between signal detection and context capture. Figure S2. Ablation Study - Computational Cost Analysis. Total processing time (hours) and chunk generation overhead. While larger chunks (32k) reduce total time, they degrade detection performance (as shown in Figure.S1). The 8k size maintains a reasonable computational cost. • Mild Cases (Thalassemia Intermedia, TI, N = 245): Classified as non-transfusion dependent thalassemia (NTDT), characterized by low or intermittent transfusion dependence and relatively milder clinical manifesta- tions. • Severe Cases (Thalassemia Major, TM, N = 775): Classified as transfusion-dependent thalassemia (TDT), Liu et al. | Supplementary Information | 13 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint characterized by high transfusion dependence and severe disease progression. The clinical classification of these samples serves as the ground truth labels for all subsequent analyses. A.2. Genomic Scope.We analyzed all protein-coding genes located on chromosome 11 with lengths shorter than 512 kb. This comprises a total of 1,031 genes. The length distribution is as follows: •< 64 kb: 1,049 genes (Note: strictly adhering to the <512kb filter). • 64 kb−128 kb: 145 genes. • 128 kb−256 kb: 72 genes. • 256 kb−512 kb: 35 genes. B. Data Processing Pipeline B.1. Phasing and Haplotype Construction. Haplotype phasing was performed on the original VCF files using Beagle v4. Default parameters were applied with genotype probability output enabled (gp=true) and a sliding window size ofwindow=10.0. This procedure phased unphased genotypes into two haplotypes (hap1andhap2) for each sample. B.2. Sequence Construction and Coordinate Mapping (VCF2CSV). We constructed haplotype-specific sequences by mapping VCF variants to the GRCh38 reference genome. For each sample, sequences were processed from the 5’ end to the 3’ end to handle coordinate shifts dynamically. The mapping rules are defined as follows: 1. No Variant: The reference base is retained, and its absolute genomic position is recorded. 2. SNPs: The reference base is replaced by the alternative allele, recording the original reference position. 3. Deletions: Bases are excluded from the sequence, and their positions are not recorded (only remaining bases retain position tags). 4. Insertions: Inserted bases are included sequentially. Crucially, the position of the preceding reference base is repeated for each inserted base to maintain alignment with the reference coordinate system. Strand Handling: For genes located on the negative strand, the generated sequences were reverse-complemented, and the corresponding position arrays were reversed to maintain alignment with the reference genome coordinate order. C. Synthetic Dataset Generation We provide the detailed statistics of the synthetic datasets constructed for robustness evaluation in Table 7. The synthetic generation process involved injecting variants into intergenic regions derived from chromosome 11. Table 7. Statistics of the Synthetic Datasets across different sequence lengths. Seq Length Carriers Mild Severe Target Variants (SNP/Indel) Target Prop. 4 kb 409 245 775 8/0 100% 20 kb 409 245 775 8/2 100% 128 kb 409 245 775 15/5 100% 384 kb 409 245 775 15/5 100% D. Baseline Settings D.1. Foundation Model.We utilize Genos-10b-v1 as the foundation model baseline. The input sequences are padded or truncated to match the model’s maximum context length where necessary. D.2. GWAS Configuration. Genome-Wide Association Studies (GWAS) are conducted using PLINK2. We apply a unified threshold strategy for fair comparison. • Quality Control (QC): Variants with missingness> 1% (–geno 0.01) and Minor Allele Frequency< 1% (–maf 0.01) were removed. • Locus Identification: A GWAS locus is counted as co-identified if its position falls within an attention-derived cluster interval defined as: Liu et al. | Supplementary Information | 14 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Listing 1.GWAS Pipeline Commands (PLINK2) # 1. Quality Control and VCF to BED c onv e r si on plink2 -- vcf / path / to / dataset / a u t o s o m e s _ s p l i t . vcf . gz \ -- make - bed \ -- out / path / to / output / t h a l _ 1 4 2 9 _ a u t o \ -- chr 1 -22 \ -- geno 0.01 \ -- maf 0.01 # 2. A s s o c i a t i o n Analysis ( GLM ) # Note : - -1 ensures output uses 1 - based c o o r d i n a t e s plink2 -- bfile / path / to / output / t h a l _ 1 4 2 9 _ a u t o \ -- glm omit - ref \ -- pheno / path / to / phe n o t y pe s / p h e n o _ b i n a r y . txt \ -- covar / path / to / cov a r i a te s / c o v a r _ f i n a l . txt \ -- out / path / to / results / g w a s _ r e s u l t s \ - -1 –Forward strand:[start−10,end] –Reverse strand: [start,end + 10] The specific PLINK2 commands used for Quality Control (QC) and Association Analysis are listed below: Supplementary Note 3: Baseline Model Configurations To benchmark our approach, we compare it against state-of-the-art genomic language models in different pre-train scopes to verify the influences of model sizes and training data on downstream tasks. Table 8 summarizes the architectures and specifications. Notably, all selected models utilize a single-nucleotide tokenizer, ensuring a fair comparison at the base resolution level. Regarding the Evo 2 series, given its hybrid StripedHyena architecture where attention heads are interspersed with convolution operators, we identified the best-performing layers reported in the original study and selected the nearest attention layer immediately preceding them for analysis. Table 8. Summary of Baseline Models. The "Extraction Layer" column indicates which layer’s attention weights were used for analysis. Model Name Architecture Pre-train Scope Data Volume Context Length Param Size Extraction Layer Ev o2-1B-Base StripedHyena 2 Multispecies 1T bp 8K bp 1.1B Layer 10 Evo2-7B-Base StripedHyena 2 Multispecies 2.1T bp 8K bp 6.5B Layer 24 Evo2-40B StripedHyena 2 Multispecies 9.3T bp 1M bp 40B Layer 17 Genos-1.2B MoE (Mixture of Experts) Human 1600B tokens 1M 1.2B Last Layer Genos-10B MoE (Mixture of Experts) Human 2200B tokens 1M 10B Last Layer Genos-10B-v2 MoE (Mixture of Experts) Human 6286B tokens 1M 10B Last Layer LucaOne Transformer Multispecies 36.95B bp 1k 1.8B Last Layer LucaVirus Transformer Viral sequences 50B bp 3k 1B Last Layer Supplementary Note 4: Evaluation Metrics for Signals A. Overview To evaluate the spatial concentration of differential signals around true variant positions, we develop a set of evalu- ation metrics. These metrics quantify how well the predicted signals localize to regions surrounding known variants. To enable fair comparison across sequences of different lengths (e.g., 4 kb vs. 384 kb), we additionally introduce length-normalized variants of key metrics. B. Notation Lets= (s 1,s 2,...,sN)denote the signal scores (absolute log 2 fold-change of attention) atNgenomic positions, wherepi denotes the genomic coordinate of positioni. Given a set of K true variant positionsV ={v1,v 2,...,vK}, Liu et al. | Supplementary Information | 15 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint we define a binary labelyi∈{0,1}indicating whether positioni falls within a window around any variant: yi = { 1 if∃v∈V:v−wL≤pi≤v +wR 0 otherwise (S1) wherewL andwR are the left and right window boundaries, respectively (tested at 3 bp, 5 bp, 10 bp, and 20 bp). We further defineT ={i:yi = 1}as the set of positions within true variant windows (positives),F ={i:yi = 0}as the set of background positions (negatives), with nT =|T|andnF =|F|denoting their respective counts. For spatial metrics, we definedi = minv∈V|pi−v|as the distance from positioni to the nearest variant. C. Primary Metric: AUPRC We adopt the Area Under the Precision–Recall Curve (AUPRC) as the primary metric to quantify ranking per- formance under severe class imbalance. AUPRC summarizes the precision-recall trade-off across all classification thresholds: AUPRC = ∑ k (Rk−Rk−1)·Pk (S2) wherePk andRk are the precision and recall at thek-th threshold. AUPRC is particularly informative when positive cases are rare. The baseline (random classifier) AUPRC equals the proportion of positives (n T/N); thus, AUPRC values should be interpreted relative to this baseline when comparing across sequences of different lengths. As shown in Figure S7, we evaluated performance across varying window sizes to ensure robustness. D. Complementary Metrics To overcome the limitations of threshold-based metrics, we introduce complementary indicators measuring magni- tude contrast, signal efficiency, and spatial precision. D.1. Signal-to-Noise Ratio (SNR). Categorized as Magnitude Contrast, SNR measures how biologically distinct the variant signal is from the background noise floor: SNR = ¯sT ¯sF = nF nT · ∑ i∈Tsi∑ i∈Fsi (S3) where¯sT and¯sF denote the mean signal in true variant windows and background regions, respectively. SNR > 1 indicates that signals are, on average, stronger near variants than in background regions; higher values indicate better signal specificity. Figure S8 illustrates the SNR distribution. D.2. Fraction of Signal in Windows (FRiW). Categorized as Signal Efficiency, this metric is analogous to the FRiP score in ChIP-seq. It quantifies the proportion of total signal that falls within true variant windows: FRiW = ∑ i∈Tsi ∑N i=1si (S4) A low FRiW implies that despite a potentially high AUPRC, the majority of the model’s attention mass is allocated to regions outside variant windows (see Figure S9). D.3. Signal-Weighted Mean Distance. Categorized as Spatial Precision (Threshold-Free), this metric measures the average distance of signal from the nearest variant, weighted by signal intensity, removing the need for hard window boundaries: Weighted Distance = ∑N i=1si·di ∑N i=1si (S5) Alower score is better, indicating that the attention mass is concentrated physically closer to the causal variants (see Figure S10). E. Length-normalized Metrics To enable fair comparison across sequences of vastly different lengths, we introduce length-normalized variants of the above metrics. These normalized metrics account for the expected baseline values under uniform signal distribution, making them suitable for cross-scale comparisons. Liu et al. | Supplementary Information | 16 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint E.1. FRiW Enrichment.To account for varying sequence lengths and window sizes, we normalize FRiW by the ex- pected fraction under a uniform signal distribution: FRiWexpected = nT N (S6) FRiW Enrichment= FRiW FRiWexpected = N nT · ∑ i∈Tsi ∑N i=1si (S7) This fold-enrichment metric is length-independent. A value of 1 implies a signal distribution equivalent to random expectation (uniform background); values > 1 indicate a significant enrichment of attention mass within variant windows. E.2. Normalized Weighted Distance. To enable comparison across sequences of different lengths, we normalize the weighted distance by the characteristic length scale of the sequence. Under a uniform distribution assumption with K variants, the expected distance to the nearest neighbor scales linearly withL/K. We therefore define: Normalized Weighted Distance = Weighted Distance L/(2K) (S8) This normalization factorL/(2K)serves as a first-order approximation of the expected random distance, facilitating fair comparison of spatial accuracy across varying genomic scales. E.3. Mean Percentile Rank of True Positions. This rank-based metric evaluates where true variant positions fall in the ranked signal distribution: Mean RankT = 1 nT ∑ i∈T R(si) N−1 (S9) whereR(s i)is the rank of si among all scores (0 for lowest, N−1 for highest). This metric ranges from 0 to 1, where 0.5 indicates performance equivalent to random ranking, and values approaching 1 indicate that positions near variants consistently have high signal scores. As a rank-based metric, Mean Percentile Rank is completely length-independent. Supplementary Note 5: Gene-Level Statistical Descriptors The gene-level analysis aims to identify distribution differences of attention scores across entire genes. For a gene withM positions and attention scores s = (s1,s 2,...,sM), we compute 17 descriptors categorized into four groups. In our experiments, Max, Std (σ), Top5%Mean, CV, Median, IQR, and Entropy provide the most significant sepa- ration between the informative (HBB) and control genes, suggesting that both the magnitude of extreme values and the overall distribution shape are informative for distinguishing regulatory patterns. A. Location/Scale (10 metrics) • Mean: The arithmetic mean of all scores: µ= 1 M ∑M j=1 sj (S10) •Median:The middle value when scores are sorted, i.e., Q0.50. • Top5% Mean: Mean of the top 5% highest scores: Top5% Mean= 1 |H| ∑ j∈H sj,whereH={j: sj≥Q0.95} (S11) • Low5% Mean: Mean of the bottom 5% lowest scores: Low5% Mean = 1 |L| ∑ j∈L sj,whereL= {j:sj≤Q0.05} (S12) • Max: The maximum score across all positions: Max = max j∈{1,...,M} sj (S13) Liu et al. | Supplementary Information | 17 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint •Standard Deviation:Measures the spread of scores around the mean: σ= √ 1 M−1 ∑M j=1 (sj−µ)2 (S14) •Coefficient of Variation (CV):Scale-normalized dispersion: CV= σ µ (S15) •Interquartile Range (IQR): The range between the 25th and 75th percentiles: IQR =Q0.75−Q0.25 (S16) • 10th Percentile (Q0.10) and 90th Percentile (Q0.90): Values below which 10% and 90% of scores fall, respec- tively. B. Distribution Shape (3 metrics) • Skewness: The standardized 3rd central moment, measuring distribution asymmetry: Skewness = 1 M ∑M j=1 (sj−µ σ )3 (S17) Positive skewness indicates a right-tailed distribution; negative skewness indicates a left-tailed distribution. •Kurtosis:The standardized 4th central moment, measuring tail heaviness: Kurtosis = 1 M ∑M j=1 (sj−µ σ )4 −3(S18) Values> 0 (leptokurtic) indicate heavy tails; values< 0 (platykurtic) indicate light tails. • Mode: The most frequent value, computes via a fixed-bin histogram as the center of the bin with maximum count. C. Peak Structure (3 metrics) To capture local regulatory motifs and identify regions of concentrated attention: • Peak Count: Number of local maxima identified, where positionj is a local maximum ifsj>sj−1andsj>sj+1. • Peak Density: PeakCount normalized by gene length: Peak Density = PeakCount M (S19) •Peak Mean:Mean attention score at peak summits: Peak Mean = 1 |P| ∑ j∈P sj (S20) wherePis the set of positions identified as local maxima. D. Information (1 metric) •Shannon Entropy: Measures the sparsity or concentration of the attention distribution. After normalizing scores to a probability distributionpj =sj/∑M k=1sk, the entropy is computed as: H(s) =− ∑M j=1 pj log(pj +ϵ) (S21) whereϵis a small constant for numerical stability. Lower entropy indicates more concentrated (sparse) attention; higher entropy indicates more uniform distribution. Liu et al. | Supplementary Information | 18 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Figure S3. ATLAS results under different allele frequencies and cohort sizes. Red vertical lines are the positions of the synthetic variants. Dots are the bases with significant attention differences. The gray areas are the clusters. Figure S4. ATLAS results under different sequence lengths Notations are the same as Figure S3. The gray areas may not be clearly visible in long sequences. Liu et al. | Supplementary Information | 19 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint 0 1 2 3 4 5 6 7 8 -log10(P-value) OR51V1 HMBS OR51M1 CLNS1A UBQLN3 LDHA NDUFC2 HBG1 PATL1 OR52J3 NUP98 NDUFC2-KCTD14 OR51F1 HBB Mean (14 Significant Genes) P=0.05 0 2 4 6 8 10 -log10(P-value) OR51B4 ENSG00000284931 UBQLN3 HBD OR51Q1 OR51C1P OR56A4 SCGB2A2 OR52I2 OR52J3 OR51E2 Median (11 Significant Genes) P=0.05 0 1 2 3 4 5 -log10(P-value) HMBS OR52E2 HBG1 OR51C1P OR52J3 UBQLN3 OR51B4 Top 5% Mean (7 Significant Genes) P=0.05 0 1 2 3 4 5 6 7 8 -log10(P-value) HBB OR51B4 OR51I2 UBQLN3 OR51E2 OR51B6 OR4D9 SERPING1 ENSG00000284931 UBQLNL Low 5% Mean (10 Significant Genes) P=0.05 0 2 4 6 8 10 -log10(P-value) HBB CLNS1A HBD HBG1 OR52E2 HMBS UBQLN3 SCGB2A2 OR51F1 ELP4 OR51B4 Max (11 Significant Genes) P=0.05 0 2 4 6 8 -log10(P-value) HMBS OR51V1 HBG1 HBD OR52E2 OR52J3 UBQLN3 CLNS1A OR51B4 OR52E1 OR51C1P OR51Q1 Standard Deviation (12 Significant Genes) P=0.05 0 1 2 3 4 5 6 7 8 -log10(P-value) HMBS OR51V1 HBD HBG1 OR52E2 OR52J3 UBQLN3 CLNS1A OR51B4 UBQLNL NDUFC2 OR52E1 PATL1 OR51Q1 Coefficient of Variation (CV) (14 Significant Genes) P=0.05 0 1 2 3 4 5 -log10(P-value) OR52E2 HBG1 OR51B4 OR51B6 HMBS HBD LRP4 SCGB2A2 NDUFC2 UBQLN3 OR52E5 HBB B3GAT3 SERPING1 OR51V1 Interquartile Range (IQR) (15 Significant Genes) P=0.05 0 5 10 15 20 25 -log10(P-value) HBB OR51B4 HBD OR52J3 ENSG00000284931 10th Percentile (5 Significant Genes) P=0.05 0 1 2 3 4 -log10(P-value) NUP98 HBD ENSG00000284931 HBG1 OR51I2 OR51B6 HMBS OR52E2 OR52E1 LDHA OR51B4 OR51Q1 90th Percentile (12 Significant Genes) P=0.05 0 1 2 3 4 5 6 7 8 -log10(P-value) OR51B4 HBB OR52E1 OR52J3 OR51F1 HBG1 OR51C1P OR52E2 Skewness (8 Significant Genes) P=0.05 0 2 4 6 8 10 12 -log10(P-value) HBB OR51B4 OR52E1 OR52E2 OR52J3 FGF19 HBG1 OR51C1P OR51F1 HMBS OR52I2 Kurtosis (11 Significant Genes) P=0.05 0 2 4 6 8 10 -log10(P-value) HBD HBB OR51V1 HBG1 ENSG00000284931 OR51B4 HARBI1 OR51F1 Mode (8 Significant Genes) P=0.05 0 2 4 6 8 -log10(P-value) OR51V1 ENSG00000284931 OR56A4 OR52E1 UBQLN3 OR51E2 OR52J3 HBB PGA3 CWF19L2 OR51I1 OR51I2 SCGB1A1 CCDC153 HMBS Peak Count (15 Significant Genes) P=0.05 0 2 4 6 8 -log10(P-value) OR51V1 ENSG00000284931 HBG1 OR56A4 OR52E1 HBB UBQLN3 OR51E2 OR52J3 CWF19L2 HMBS PGA3 OR51I1 CCDC153 SCGB1A1 OR51I2 CCDC88B Peak Density (17 Significant Genes) P=0.05 0 1 2 3 4 5 6 7 -log10(P-value) ENSG00000284931 HMBS OR52E1 HBG1 OR51C1P OR56A4 OR51E2 UBQLN3 OR52J3 OR52A1 Peak Mean (10 Significant Genes) P=0.05 0 5 10 15 20 25 30 35 40 -log10(P-value) HBB HMBS OR52A1 OR52E2 OR52E1 OR52J3 HBG1 OR4D9 OR51B4 OR51C1P HBD OR51F1 NDUFC2 Shannon Entropy (13 Significant Genes) P=0.05 Figure S5. The significant genes selected by the 17 distribution descriptor in the gene-level analysis. The results are retrieved from the haplotype 1 attention scores 0 1 2 3 4 5 6 -log10(P-value) CLNS1A HMBS OR51V1 Mean (3 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 -log10(P-value) OR51E2 OR51L1 OR52J3 OR51C1P OR52M1 Median (5 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 -log10(P-value) OR51B4 DENND5A HMBS OR52J3 Top 5% Mean (4 Significant Genes) P=0.05 0 1 2 3 4 5 -log10(P-value) HBB OR51E2 Low 5% Mean (2 Significant Genes) P=0.05 0 1 2 3 4 5 6 7 8 -log10(P-value) HBB CLNS1A HMBS OR51B4 Max (4 Significant Genes) P=0.05 0 1 2 3 4 5 6 -log10(P-value) HMBS OR51V1 OR51B4 CCKBR OR52J3 FGF19 CCDC153 CLNS1A OR52E1 Standard Deviation (9 Significant Genes) P=0.05 0 1 2 3 4 5 6 -log10(P-value) HMBS OR51V1 OR51B4 OR52J3 CCKBR CCDC153 CLNS1A NUP98 Coefficient of Variation (CV) (8 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 -log10(P-value) HMBS OR52E2 FNBP4 OR51C1P OR52N1 OR51L1 OSBP Interquartile Range (IQR) (7 Significant Genes) P=0.05 0 5 10 15 20 25 -log10(P-value) HBB OR52J3 10th Percentile (2 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 -log10(P-value) OR51B4 OR56B1 OR51E2 90th Percentile (3 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 -log10(P-value) HBB OR52A1 OR52J3 OR52E1 OR52M1 OR51B4 CCDC153 PGAP2 Skewness (8 Significant Genes) P=0.05 0 2 4 6 8 10 -log10(P-value) HBB OR52J3 OR52M1 OR52E1 OR51B4 Kurtosis (5 Significant Genes) P=0.05 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 -log10(P-value) OR51V1 OR51B4 HBD HBB OR51S1 Mode (5 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 -log10(P-value) OR51V1 OR52J3 OR52E1 HBB OR51L1 OR51E2 OR52L1 BAD Peak Count (8 Significant Genes) P=0.05 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 -log10(P-value) OR51V1 HBB OR51L1 OR52J3 OR52E1 OR52L1 BAD Peak Density (7 Significant Genes) P=0.05 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 -log10(P-value) OR51E2 OR52J3 OR52E1 Peak Mean (3 Significant Genes) P=0.05 0 5 10 15 20 25 30 -log10(P-value) HBB OR52A1 OR51B4 CCKBR ENSG00000254979 OR52J3 HMBS FGF19 OR52E2 OR52E1 Shannon Entropy (10 Significant Genes) P=0.05 Figure S6. The significant genes selected by the 17 distribution descriptor in the gene-level analysis. The results are retrieved from the haplotype 2 attention scores Liu et al. | Supplementary Information | 20 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.05 0.10 0.15 0.20 0.25AUPRC (Log2FC) Window = 3bp Evo2_1b Evo2_40b Evo2_7b Genos-1.2b Genos-v1 Genos-v2 LucaOne LucaVirus Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.05 0.10 0.15 0.20 0.25 0.30AUPRC (Log2FC) Window = 5bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40AUPRC (Log2FC) Window = 10bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.1 0.2 0.3 0.4 0.5AUPRC (Log2FC) Window = 20bp Figure S7. Evaluation of Detection Quality - AUPRC Analysis (Higher is better).Evaluation of ranking performance across different window sizes (3bp, 5bp, 10bp, 20bp). While AUPRC measures the ranking order, the Genos-v2 model (brown) consis- tently demonstrates strong ranking capabilities across most haplotype comparisons. Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 1 2 3 4 5 6SNR (Log2FC) Window = 3bp Evo2_1b Evo2_40b Evo2_7b Genos-1.2b Genos-v1 Genos-v2 LucaOne LucaVirus Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 2 3 4 5 6SNR (Log2FC) Window = 5bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 2 3 4 5SNR (Log2FC) Window = 10bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 2 3 4 5 6SNR (Log2FC) Window = 20bp Figure S8. Evaluation of Detection Quality - Signal-to-Noise Ratio (SNR) Analysis (Higher is better). Assessing the magnitude contrast between variant signals and background noise. A higher SNR confirms that the identified signals have a significant magnitude difference compared to the background, indicating that the model’s attention peaks at risk variants are biologically distinct from the noise floor. Liu et al. | Supplementary Information | 21 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10FRiW (Log2FC) Window = 3bp Evo2_1b Evo2_40b Evo2_7b Genos-1.2b Genos-v1 Genos-v2 LucaOne LucaVirus Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.04 0.06 0.08 0.10 0.12FRiW (Log2FC) Window = 5bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22FRiW (Log2FC) Window = 10bp Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 0.10 0.15 0.20 0.25 0.30 0.35FRiW (Log2FC) Window = 20bp Figure S9. Evaluation of Attention Efficiency - Fraction of Signal in Windows (FRiW) (Higher is better). Quantifying the global attention budget allocation. Models with higher FRiW scores are more efficient, concentrating their attention mass into the relevant variant windows rather than dispersing it across the sequence. Hap1 1v2 Hap2 1v2 Hap1 2v3 Hap2 2v3 Hap1 1v3 Hap2 1v3 50 60 70 80 90 100 110 120WDist (Log2FC) Evo2_1b Evo2_40b Evo2_7b Genos-1.2b Genos-v1 Genos-v2 LucaOne LucaVirus Figure S10. Evaluation of Spatial Precision - Distance-Weighted Mean (Lower is better). A threshold-free metric measuring spatial precision. Lower values indicate that high attention scores are physically closer to the target variants, minimizing spatial deviation. Liu et al. | Supplementary Information | 22 .CC-BY 4.0 International licenseperpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint