{"paper_id":"02479441-3ea2-4aea-80e8-019b5da6d0ab","body_text":"ATLAS: Population-Level Disease Locus Discovery\nvia Differential Attention in Genomic Language\nModels\nYuqi Liu1,2, Kaiwen Deng2, Yuhua Ye3, Jiajie Zhan2, Zilin Wang2, Shicheng Chen2, Xinyue Hu2, An Chang2, Zhaorong\nLi2, Xin Jin2, Shiping Liu2, Kui Chen2, Huijun Shen2, Xianzhi Qi2, Xiangmin Xu3, Haiqiang Zhang2\n1Hangzhou Institute for Advanced Study, UCAS, Hangzhou, China; 2Genos Team; 3Southern Medical University,\nGuangzhou, China\nIdentifying disease-associated genetic variants re-\nmains a key challenge in genomics, especially\nin small cohorts or for rare and complex muta-\ntion types where genome-wide association studies\n(GWAS) often fall short. We introduce ATLAS, a\npopulation-level framework that leverages attention\nsignals from pretrained genomic language mod-\nels (gLMs) to detect disease-associated genes and\nloci directly from raw sequences—without requir-\ning explicit variant calls or supervised training. AT-\nLAS first performs gene-level differential attention\nanalysis to prioritize candidate genes, followed by\nbase-level analysis to localize disease-associated\nregions at single-haplotype resolution. We validate\nATLAS on synthetic andβ-thalassemia datasets,\ndemonstrating robust performance across diverse\nallele frequencies (down to 10%), cohort sizes (be-\nlow 200 individuals per group), and genomic scales.\nCompared to GWAS, ATLAS achieves higher recall\nof known loci and captures haplotype-specific sig-\nnals missed by traditional methods. Cross-model\nbenchmarking further shows that precise localiza-\ntion depends on both model size and pretraining on\ndiverse human genomes. In summary, ATLAS of-\nfers a scalable, sequence-native alternative to tradi-\ntional statistical genetics.\nGenomic Language Models | Disease Locus Discovery |\nPopulation-Level Genomics | Attention-Based Interpreta-\ntion\nCorrespondence:zhanghaiqiang@genomics.cn\nIntroduction\nUnderstanding how genetic variants affect gene activity\nis central to linking genotype to disease. Variants such\nas missense single-nucleotide polymorphisms (SNPs),\ninsertions, and deletions can alter protein structure,\ndisrupt regulatory elements, or perturb transcriptional\nregulation, contributing to Mendelian diseases, inher-\nited metabolic disorders, and complex diseases includ-\ning cancer and neurodevelopmental disorders Cutting\n(2015); Turner and Eichler (2019); Pagel et al. (2019);\nMontella et al. (2025). Accurately identifying such vari-\nants remains a central challenge in human genetics,\nwith broad implications for disease prediction, mecha-\nnism discovery, and precision medicine.\nGenome-wide association studies (GWAS) are the\nmost widely used framework for identifying disease-\nassociated variants, enabling the discovery of thou-\nsands of loci linked to complex traits through large-\nscale genotype–phenotype correlation analyses Risch\nand Merikangas (1996); Klein et al. (2005); Sollis et al.\n(2023). However, GWAS provide limited resolution at\nthe haplotype level and are not optimized for non-SNP\nvariants such as insertions and deletions Tewhey et al.\n(2011); Alkan et al. (2011). Their statistical power also\ndepends strongly on allele frequency and effect size,\noften requiring very large cohorts to detect modest ef-\nfects Visscher et al. (2017). Complementary bioinfor-\nmatics tools predict variant effects using biological fea-\ntures such as sequence conservation and protein prop-\nerties, avoiding large sample size requirements, but are\ntypically restricted to predefined features and individual\nvariants Adzhubei et al. (2013); Vaser et al. (2016).\nRecent advances in genomic large language models\n(gLMs) provide a new paradigm for sequence-based\nvariant analysis. Trained on massive genomic corpora,\ngLMs capture long-range dependencies and achieve\nstate-of-the-art performance in regulatory annotation,\nvariant effect prediction, and functional genomicsDalla-\nTorre et al. (2025); Brixi et al. (2025); Lin et al. (2025).\nUnlike GWAS, gLMs operate directly on raw DNA se-\nquences and naturally accommodate heterogeneous\nvariants, including substitutions and indels. However,\nmost gLM-based applications focus on supervised pre-\ndiction at the individual level Avsec et al. (2021); Bene-\ngas et al. (2023), leaving population-level sequence\ncomparisons—analogous to GWAS but in a learned em-\nbedding space—largely unexplored.\nIn this paper, we propose ATLAS (Attention-based Lo-\ncus Analysis System), an efficient attention-based ex-\nplanation framework built on the genomic language\nmodel Genos Lin et al. (2025). We hypothesize that at-\ntention weights encode sequence-level importance and\nthat functionally relevant loci manifest as statistically sig-\nnificant attention distribution shifts between case and\ncontrol populations. By directly contrasting these at-\ntention patterns, ATLAS enables the identification of\ndisease-associated loci without relying on explicit vari-\nLiu et al. |\n| February 9, 2026 | 1–22\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nant annotations or large sample sizes. Our main contri-\nbutions are as follows.\n•End-to-end disease locus discovery. We pro-\npose ATLAS, a framework that directly analyzes\ngenomic sequences from multiple populations to\nidentify disease-associated loci and genes without\nexplicit variant calling.\n• Robust and flexible interpretation. ATLAS is ca-\npable of multi-scale analysis at both gene and base\nlevels with single-haplotype resolution, demon-\nstrating robustness in small cohorts and high sen-\nsitivity to low-frequency variants.\n• Empirical validation on real-world data. AT -\nLAS not only recovers known disease-associated\nloci reported in the literature and GWAS but also\nidentifies additional informative candidates on β-\nthalassemia datasets.\n• Revisiting the value of human-centric founda-\ntion models. By benchmarking diverse archi-\ntectures, we demonstrate that massive parame-\nter scale in generalist models does not guaran-\ntee performance. Instead, we conclude that ac-\ncurate disease localization critically depends on\nthe synergy between model capacity and extensive\nhuman-centric pretraining.\nRelated Work\nGenomic Language Foundation Models\nThe application of large language models (LLMs)\nto genomics has shifted sequence analysis from\nalignment-based statistics to representation learning\nover raw DNA. These genomic language models\n(gLMs) are trained on large genomic corpora with self-\nsupervised objectives to capture long-range dependen-\ncies and contextual sequence semantics. Representa-\ntive general-purpose gLMs include LucaOne and the\nNucleotide Transformer series, which leverage multi-\nspecies genomic data to learn transferable sequence\nrepresentations Dalla-Torre et al. (2025); He et al.\n(2025). Evo 2 further extends this paradigm by mod-\neling genomic sequences across all domains of life with\nstrong generative capabilityBrixi et al. (2025). However,\nmost general-purpose gLMs rely on reference genomes\nor cross-species consensus signals, limiting their sensi-\ntivity to human population variation and disease-specific\nsequence heterogeneity. Therefore, this work builds\non Genos, a human-centric genomic language founda-\ntion model trained on large-scale human population se-\nquencing data Lin et al. (2025), which has learned rep-\nresentations that are more sensitive to pathogenic vari-\nants and human-specific regulatory patterns.\nThe Attention-based Model Interpretation\nAttention mechanisms are intrinsic to Transformer ar-\nchitectures and provide an explicit, quantitative mea-\nsure of how models weight long-range dependencies\nacross input sequences. In genomics, attention has\nbeen widely used as an interpretability tool to reveal\nbiologically meaningful interactions learned during su-\npervised training. For example, Enformer demonstrated\nthat attention maps capture long-range regulatory in-\nteractions underlying gene expression prediction Avsec\net al. (2021). Beyond supervised tasks, attention has\nalso been applied to unsupervised structural discovery,\nincluding the recovery of transcription factor binding mo-\ntifs and the reconstruction of three-dimensional chro-\nmatin contact maps directly from sequences Tomaz da\nSilva et al. (2025); Boshar et al. (2025). While exist-\ning attention-based analyses predominantly focus on\nwithin-sequence interpretation for individual genomic\nsequences, our framework shifts attention analysis to-\nward population-level comparison.\nVariant Effect Prediction Models\nPrior work has largely focused on supervised variant ef-\nfect prediction, estimating the impact of specific vari-\nant classes. Many approaches excel by specializ-\ning in defined biological mechanisms. For example,\nSpliceAI identifies splice-disrupting variants by model-\ning sequence context but is limited to splicing regula-\ntion Jaganathan et al. (2019). Similarly, AlphaMissense\npredicts missense pathogenicity using protein language\nmodels, yet remains restricted to coding regions Cheng\net al. (2023). Unlike these variant-centric methods that\nevaluate mutations individually, ATLAS offers a model-\nagnostic pipeline capable of detecting diverse events\nwithout task-specific supervision or predefined variant\nclasses.\nStatistical and Bioinformatic Approaches\nGenome-wide association studies (GWAS) consti-\ntute the dominant statistical framework for genotype-\nphenotype analysis, but their power critically depends\non allele frequency, effect size, and large cohort sizes,\nrequiring 104–106 samples to reliably detect variants\nat modest (10%) frequencies Visscher et al. (2017).\nRegion-based aggregation methods such as SKAT par-\ntially alleviate this limitation by combining signals across\npredefined genomic regions or genes, yet still require\nmoderate to large sample sizes for stable inference Wu\net al. (2011); Zhan et al. (2016). In parallel, classical\nbioinformatic tools (e.g., SIFT4G, PolyPhen-2) estimate\nvariant effects using curated annotations and prior bi-\nological knowledge, but are largely restricted to cod-\ning single-nucleotide substitutions and struggle to cap-\nture haplotype context or signals in poorly annotated re-\ngions Adzhubei et al. (2013); Vaser et al. (2016). To-\ngether, these limitations motivate alternative population-\nlevel frameworks that operate directly on sequence rep-\nresentations and are less dependent on explicit variant\nenumeration or extensive prior annotation.\nLiu et al. |\n| 2\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nS tep 1: Input D ata S tep 2: A ttention Extraction S tep 3: G ene-level A nalysis\nS tep 4: Base-level A nalysis & Cluster\nControl\nNg0S equences\nH aplotype1\nH aplotype2\nControl Input\nCase Input\nH aplotype1\nCase S equences\nM utations\nH aplotype2\ngLM s\nm ult-head\nNg0attention\nNg0m atrix\nlast layer\nattn. score\ncolum n sum\nG ene 1\nG ene 2\nG ene N\nCase\nControl\nattn 1\nattn 2\nattn N\nattn 1\nattn 2\nattn N \nD istribution com parison\nA ttn 1\nA ttn 1\nA ttn 2\nA ttn 2\nA ttn N\nA ttn N\nControl\nCase\nelem ent-w ise\nNg0average\nControl Case\np=0.05\nG enes\n-log10（p-value）\nCandidate gene\nCandidate gene:\nD NA sequence input\nS tatistical tests\n* * * \nA ttn-D iff\nlog2FC\nK now n variants\nD iiferential \nNg0A ttention \nNg0A nalysis\ntarget Com m on\nRegions w ith target variants\nCluster the high-confident loci \nlog2FC\nattn. score\nFigure 1. Overview of the ATLAS workflow. Step 1: ATLAS operates on haplotype-resolved genomic sequences derived from case and control cohorts. Step\n2: Each sequence is processed a genomic language model (e.g., Genos). Multi-head attention matrices are extracted from the final layer, averaged element-wise,\nand aggregated via column-wise summation to obtain nucleotide-level importance scores. Step 3: ATLAS prioritizes candidate genes by performing statistical tests\non attention score distributions between cohorts, identifying genes that exhibit significant disease-associated attention fluctuations relative to the background. Step\n4: Within candidate genes, the framework calculates differential attention scores to identify specific loci with significant signal divergence. High-confidence sites are\nfinally clustered to delineate discrete risk-associated genomic regions.\nMethods\nIn this section, we present the ATLAS pipeline and its\nbenchmarking against GWAS (Figure 1). We first de-\nscribe how attention scores are extracted from genomic\nsequences using a pretrained genomic language model\nand how variable-length inputs are handled. We then in-\ntroduce the gene-level differential attention analysis for\nidentifying candidate genes. Next, we dive into these\ncandidates, detecting disease-associated loci at base-\nlevel resolution and clustering them into regions of in-\nterest. Finally, we summarize the GWAS procedures\nand describe how GWAS results are used for compar-\native evaluation. The code of ATLAS is available at\nhttps://github.com/BGI-HangzhouAI/ATLAS.\nRetrieve and Calculate Attention Scores\nWe extract attention scores from the final Transformer\nlayer of each genomic language foundation model, with\nexceptions for specific architectures (e.g., the Evo 2 se-\nries Brixi et al. (2025); see Supplementary Note 3). This\napproach is motivated by evidence that the final layer\ncaptures the most globally contextualized sequence in-\nformation Vig and Belinkov (2019) and serves as the\nprimary interface for downstream biological tasks Marin\net al. (2023). Consequently, we consistently employ\nfinal-layer attention to maximize the capture of high-\nlevel semantic features.\nLet the input sequence consist of L tokens. For each\nattention head h, we extract the query and key projec-\ntions, Q(h) and K(h), and apply Rotary Position Em-\nbeddings (RoPE). We first define the pre-softmax atten-\ntion matrix A(h) incorporating the causal mask Mcausal.\nThe final saliency score sj for the j-th position is then\nderived by averaging the attention probabilities across\nH heads and summing the attention weights received\nfrom all query positions (i.e., column-wise summation):\nA(h) = RoPE(Q(h))RoPE(K(h))⊤\n√\nd\n+M causal\nsj = 1\nH\nL∑\ni=1\nH∑\nh=1\nsoftmax\n(\nA(h)\n)\ni,j\n(1)\nTo efficiently compute attention scores across vary-\ning sequence lengths, we adopt a three-tiered strat-\negy based on sequence lengthL. These thresholds\nare empirically determined to balance scoring latency\nand GPU memory consumptions on an NVIDIA H100\n(80GB) GPU, ensuring numerical stability and signal in-\ntegrity (see detailed benchmarks in Appendix C).\n•Vanilla attention (L≤4,096):For short se-\nquences, we explicitly materialize the full L×L\nattention matrix. Our benchmarks indicate that\nwithin this range, the computational overhead is\nmanageable, allowing full-matrix operations to be\nLiu et al. |\n| 3\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nefficiently handled even on GPUs with more con-\nstrained memory capacities, such as the NVIDIA\nA40 (48GB).\n•FlashAttention-based (4,096< L≤ 131,072):\nFor sequences up to 128k (where 1k = 1, 024),\nwe employ a FlashAttention-style algorithm Dao\net al. (2022). This approach computes exact atten-\ntion scores using block-wise tiling and streaming\nsoftmax, effectively bypassing the O(L2) memory\nbottleneck while maintaining identical precision to\nvanilla attention.\n• Chunked processing (L > 131,072): For even\nlonger sequences exceeding the hardware’s\nsingle-pass capacity (128k), where memory con-\nsumption and computational costs grow quadrati-\ncally, we apply a sliding-window chunking strategy.\nWe empirically select a chunk size of 8,192 with\na 4,096 overlap(Figure S1), as this configuration\ndemonstrates the optimal balance between com-\nputational throughput and signal quality. To miti-\ngate boundary effects and preserve continuity, we\ndiscard the outer half of each overlap region and\nretain only the central region of each chunk for fi-\nnal reconstruction.\nGene-wise Differential Attention Analysis\nWe first identify candidate disease-associated genes\nusing a gene-wise differential attention analysis, which\ncompares attention distributions aggregated over de-\nfined genomic windows between control and disease\ncohorts. Prior work has shown that sequence variants\ncan induce systematic and biologically meaningful per-\nturbations in attention patterns of genomic Transformer\nmodels, with detectable fluctuations in aggregated at-\ntention distributions Consens et al. (2025). Motivated\nby this evidence, we hypothesize that genes involved in\ndisease will exhibit reproducible differences in attention\ndistributions relative to controls.\nFor each gene, we consider the entire gene body. The\nstart and end positions are retrieved directly from the\nEnsembl database. We summarize the per-base atten-\ntion scores across these regions for each sample us-\ning a set of complementary statistics designed to cap-\nture central tendency, extremum, and dispersion (Sup-\nplementary Note 5). For each summary statistic, we\ncompare the distributions between control and disease\ncohorts using a two-sided Mann–Whitney U test. To ac-\ncount for multiple comparisons across the genome, we\napply the Benjamini-Hochberg procedure to control the\nFalse Discovery Rate (FDR). Genes with FDR-adjusted\np-values< 0.05are considered significant.\nAcross our evaluations, Shannon Entropy consistently\nexhibits the strongest discrimination. We attribute this\nto the model’s sensitivity to risk syntax: rather than dis-\npersing attention, disease-associated variants act as\nstrong attractors. This causes the model’s focus to\nsystematically concentrate on specific loci, thereby re-\nducing the distributional entropy compared to controls.\nConsequently, Shannon Entropy is emphasized in sub-\nsequent analyses.\nBase-wise Differential Attention Analysis\nTo localize potentially disease-associated loci within the\ncandidate genes prioritized in Section , we perform a\nbase-level differential attention analysis. Drawing an\nanalogy to differential expression analysis used to iden-\ntify disease signatures Rosati et al. (2024), we hypothe-\nsize that high-risk loci induce systematic fluctuations in\nattention patterns between case and control cohorts.\nFor each genomic position j, we aggregate attention\nscores across valid samples in each cohort. LetSc and\nSd denote the sets of samples where positionj is effec-\ntively present (i.e., non-deleted and non-padding) in the\ncontrol and disease groups, respectively. To rigorously\nhandle Indels, we align comparisons to the reference\ncoordinates: for deletions (e.g., AAAT →A), differen-\ntial attention is quantified at the retained anchor (start)\nposition. Positions falling within the deleted span are\nexcluded from Sg for the affected samples to prevent\nzero-inflation artifacts. We compute the base-wise log2\nfold change (LFCj) as:\nLFCj = log2\n(µd,j +ϵ\nµc,j +ϵ\n)\n,\nwhereµg,j = 1\n|Sg|\n∑\ni∈Sg\nAi,j\n(2)\nwhereA i,j represents the attention score of sample i\nat position j, and ϵis a small pseudo-count to ensure\nnumerical stability.\nThe statistical significance of this difference is assessed\nusing a two-sided Mann–Whitney U test, with p-values\nadjusted via the Benjamini–Hochberg (BH) procedure.\nA position is identified as ahighly differentiated attention\nlocus if it satisfies two criteria: (1) adjusted p-value<\n0.01; and (2) absolute LFCj exceeds a length-adaptive\nquantile threshold (qL). This dynamic thresholding strat-\negy is critical for maintaining a constant false discov-\nery rate across varying genomic contexts, as detailed in\nSection .\nClustering and Delineating Disease-associated Re-\ngions\nWe observe that differential attention signals do not al-\nways strictly co-localize with target variants at single-\nbase resolution; rather, they exhibit a proximal enrich-\nment pattern, where significant attention fluctuations\nspatially cluster around disease-associated loci. Con-\nsequently, we aim to aggregate these discrete high-\nconfidence signals into contiguous candidate risk re-\ngions.\nAdaptive Thresholding. To ensure consistent detec-\ntion sensitivity, the filtering threshold qL introduced in\nLiu et al. |\n| 4\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nSection is formulated as a function of sequence length\nL. We empirically calibrate the baseline threshold on 4\nkb sequences, identifying the 95th percentile (α= 0.05)\nas the optimal cutoff for effective signal isolation. How-\never, applying this fixed threshold to longer sequences\nresults in elevated false discovery rates due to the ex-\npanded search space. To counteract this, we define qL\n(forL in kb) as:\nqL = 1−4α\nL·log10\n(L\n4 + 9\n)\n(3)\nThis formulation is strictly grounded in our experimental\nobservations: the term is designed to satisfy the bound-\nary conditionq L = 1−α= 0.95at the baseline length\n(L= 4kb), while imposing progressively stricter filter-\ning constraints (asymptotically approaching 1) for longer\nsequences to rigorously suppress the accumulation of\nbackground noise.\nDensity-based Clustering.We apply DBSCAN Ester\net al. (1996); Schubert et al. (2017) to the loci satisfy-\ning the qL criterion, calculating clusters along 1D ge-\nnomic coordinates using Euclidean distance. We set\nthe neighborhood radius ϵ= 20 bp, approximating the\ntypical length of transcription factor binding sites or reg-\nulatory motifs, and min_samples = 5 to ensure that de-\ntected clusters represent robust, multi-base signal ac-\ncumulations rather than isolated artifacts.\nBenchmark with GWAS\nTo establish a baseline comparison with traditional sta-\ntistical methods, we performed genome-wide associa-\ntion studies (GWAS) on genotype data from 1,429 β-\nthalassemia carriers and patients (details in Supple-\nmentary Note 2 A).\nWe implemented a stringent quality control (QC)\npipeline using VCFtools Danecek et al. (2011). First,\nmulti-allelic variants were split into biallelic records, and\nchromosome identifiers were standardized. We then\napplied genotype-level filtering to set genotypes with\nsequencing depth (DP) < 4 to missing. Subsequently,\nvariant-level filtering was performed to retain only auto-\nsomal loci with a missingness rate < 15% and a minor\nallele frequency (MAF)≥1%.\nThe post-QC dataset was converted to PLINK binary\nformat using plink2 Chang et al. (2015). We performed\na case–control association analysis using a logistic re-\ngression model to test the additive effect of each vari-\nant on the binary disease phenotype. To control for\nfalse positives, statistical significance was assessed us-\ning the conventional genome-wide significance thresh-\nold (p< 6.3×10−9).\nExperiment Design\nSynthetic Datasets\nTo systematically evaluate ATLAS’s robustness under\ncontrolled conditions—particularly for small cohorts and\nlow allele frequencies—we constructed a series of syn-\nthetic datasets using background noise derived from\nthe realβ-thalassemia cohort (details in Supplementary\nNote 2 A).\nWe sampled four genomic regions of increasing length\n(4 kb, 20 kb, 128 kb, and 384 kb) from the reference\ngenome, explicitly excluding the HBB locus to prevent\ndata leakage. Within each region, synthetic target vari-\nants were inserted as ground truth, simulating increas-\ning complexity: from 8 SNPs in 4 kb regions to a mix-\nture of 20 variants (15 SNPs + 5 Indels) in larger win-\ndows. Genotypes were assigned based on disease sta-\ntus: cases were modeled as homozygous mutants (1|1)\nand controls retained the reference genotype (0|0).\nWe designed two experimental settings to stress-test\nthe model:\n• Allele Frequency (AF) Sensitivity: In the 4 kb re-\ngion, we simulated lower penetrance by randomly\ndownsampling mutant genotypes (1|1→0|0) in\ncase samples to target frequencies of 70%, 50%,\n20%, and 10%, while keeping background sites un-\nchanged.\n• Sample Efficiency: We varied the balanced\ncase–control cohort sizes from 200:200 down to\n10:10, fixing the penetrance of synthetic target vari-\nant at 100%.\nSequence construction followed a strict coordinate\nmapping pipeline to handle Indel-induced shifts (see\nSupplementary Note 2 B).\nThalassemia Cohort and Data Preprocessing\nWe designed this real-world evaluation to assess\nwhether ATLAS can recover known risk loci under real-\nistic population heterogeneity and, crucially, to identify\nplausible disease-associated signals beyond curated\nannotations.\nThe β-thalassemia dataset was derived from a cross-\nsectional whole-genome sequencing study investigating\nthe clinical heterogeneity of hemoglobinopathies com-\nprising 1,429 individuals. For validation, we referenced\nthe IthaGenes database Kountouris et al. (2014), which\ndocuments 512 thalassemia-associated variants in the\nHBB gene. Within our cohort, 31 of these recorded vari-\nants were identified. We selected the 8 most preva-\nlent variants routinely used in clinical diagnosis as the\nground-truth set for our primary performance bench-\nmarks Writing Group For Practice Guidelines For Di-\nagnosis And Treatment Of Genetic Diseases Medical\nGenetics Branch Of Chinese Medical Association et al.\n(2020).\nRaw genotypes were processed to generate model-\nready sequences. First, VCF records were phased\nusing Beagle4 Browning and Browning (2007) to ob-\ntain haplotype-resolved genotypes. Subsequently, we\nreconstructed full haplotype sequences for each indi-\nvidual by applying variants to the GRCh38 reference\nLiu et al. |\n| 5\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\ngenome. Specific rules for handling Indels, coordinate\nmapping, and reverse-complementation for negative-\nstrand genes are detailed in Supplementary Note 2 B.\nGenome-Wide Scanning on Chromosome 11.To eval-\nuate beyond knownHBB mutations, we extended the\nanalysis to all protein-coding genes on chromosome\n11. This genome-wide scan aimed to: (i) identify pre-\nviously uncharacterized loci; and (ii) evaluate method\nspecificity, under the expectation that non–hematology-\nrelated genes should exhibit minimal differential atten-\ntion signals.\nEvaluation Metric\nBase-Resolution Localization Accuracy. To evaluate\nlocalization performance at single-base resolution, we\nmeasure the extent to which attention-derived signals\nconcentrate near known disease-associated variant po-\nsitions. We posit that informative signals should prefer-\nentially localize in the immediate vicinity of causal vari-\nants; distal signals are considered less actionable for\ndownstream validation.\nGiven the extreme class imbalance between causal\nvariants (rare) and background bases (abundant), we\nadopt the Area Under the Precision–Recall Curve\n(AUPRC) as our primary metric. We utilize absolute\nlog2 fold-change (log 2 FC) values as scores, treating\nbases within fixed windows around each variant as pos-\nitive labels. To provide a comprehensive assessment of\nsignal quality beyond ranking, we report three comple-\nmentary metrics:\n• Signal-to-Noise Ratio (SNR): Measuring the\nmagnitude contrast between signal and back-\nground regions.\n• Fraction of Signal in Windows (FRiW):Quantify-\ning the efficiency of attention mass allocation (anal-\nogous to ChIP-seq FRiP scores).\n• Weighted Distance (WDist): Assessing spatial\nprecision without imposing hard window bound-\naries.\nFor cross-scale comparisons (e.g., 4 kb vs. 384 kb se-\nquences), we utilize length-normalized variants of FRiW\nand WDist, alongside the mean percentile rank of true\npositions. Detailed mathematical definitions are pro-\nvided in Supplementary Note 4.\nCluster-Level Recovery. We further evaluate perfor-\nmance at the region level by assessing whether the\ncontiguous clusters identified by our pipeline (Section )\nsuccessfully overlap with known variant loci. For syn-\nthetic datasets, we compute both precision and recall\nto measure the recovery of predefined variants and\nthe suppression of spurious clusters. For real-world\ncohorts (e.g., β-thalassemia), where ground-truth an-\nnotations are inherently incomplete, we prioritize re-\ncall—quantifying the coverage of clinically validated loci.\nClusters lacking overlap with annotated variants are not\nstrictly penalized as false positives, as they may repre-\nsent novel, uncharacterized biological signals.\nTable 1. Biological evidence for top-ranked β-thalassemia associated genes\nidentified by ATLAS.\nGene Biological Relevance\nHBMSβ-thalassemia associated gene inferred by\nGeneCard Stelzer et al. (2016) and DISEASE\ndatabases Grissa et al. (2022).\nOR52A1 Contains the γ-globin enhancer; variants in\nthis region can affect erythropoiesis and mod-\nulate thalassemia phenotypes Himadewi et al.\n(2021).\nHBG1 A key modifier gene encoding γ-globin. Up-\nregulation (↑HbF) ameliorates clinical severity\nin β-thalassemia.\nHBD Encodes δ-globin. Co-inherited variants or al-\ntered HbA2 levels are diagnostic hallmarks for\nβ-thalassemia carriers.\nResults\nIn this section, we present the evaluation of ATLAS in\nthree parts. First, we demonstrate its practical efficacy\nby identifying disease-associated genes and localizing\nfine-grained signals in a real-world β-thalassemia co-\nhort. Second, we validate the method’s robustness un-\nder controlled conditions using synthetic datasets with\nvarying cohort sizes and signal strengths. Finally, we\nbenchmark different foundation models to reveal how\nmodel scale and data diversity impact disease localiza-\ntion performance.\nGene-level differential attention identifies disease-\nassociated genes\nOur gene-level differential attention analysis success-\nfully prioritizes the HBB gene as the top candidate on\nchromosome 11, demonstrating the model’s capacity\nto distinguish disease-associated signals from the ge-\nnomic background. As illustrated in Figure 2, HBB\ndominates the rankings across both haplotypes, achiev-\ning significance levels (−log10padj) that are 6.36×and\n9.60×higher than the second-ranked candidates on\nhaplotype 1 and haplotype 2, respectively. Specifically,\ndisease samples exhibite a sharp reduction in Shan-\nnon entropy within the HBB locus compared to con-\ntrols (padj< 10−32, Figure 3). This significant drop indi-\ncates that the model’s attention—typically dispersed in\ncontrols—becomes systematically focused on specific\ndisease-associated sites in the disease state. This pat-\ntern proves robust across multiple distributional metrics,\nincluding standard deviation and coefficient of variation\n(Figure S5 and S6).\nBeyond the expected HBB signal, the analysis demon-\nstrates high specificity. Among the top candidates,\nonly 15 other genes surpass the significance thresh-\nold (p adj < 0.05) across the entire chromosome. Cru-\nLiu et al. |\n| 6\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\n0 10 20 30 40\n-log10(p-value)\nHBB\nHMBS\nOR52A1\nOR52E2\nOR52E1\nOR52J3\nOR4D9\nHBG1\nOR51C1P\nOR51B4\nHBD\nOR51F1\nNDUFC2\nCCDC88B\nTRIM77\nOR51Q1\nENSG00000284931\nPATL1\nOR51I2\nOR4D11\nShannon Entropy\nHaplotype 1\nAdjusted p=0.05\n0 5 10 15 20 25 30\n-log10(p-value)\nHBB\nOR52A1\nOR51B4\nCCKBR\nENSG00000254979\nOR52J3\nFGF19\nHMBS\nOR52E1\nOR52E2\nPATL1\nOR52L1\nOR52N4\nZFTA\nOR51C1P\nCD5\nCEP295\nDENND5A\nOR2AT4\nOR52M1\nShannon Entropy\nHaplotype 2\nAdjusted p=0.05\nFigure 2.Top-20 genes with differently distributed attentions quantified in en-\ntropy. The bar plots rank the top-20 protein-coding genes on chromosome 11\nby−log10(p)values from Wilcoxon rank-sum tests for Shannon entropy, high-\nlighting HBB as the most significant gene.\ncially, four of these genes—HBG1,HBD,HBMS, and\nOR52A1—are biologically validated as thalassemia\nmodifiers or hemoglobin regulators (Table 1). The\nrecovery of these secondary but functionally relevant\ngenes, amidst a low false-positive background, confirms\nthat differential attention entropy serves as an effective,\nzero-shot filter for isolating disease-relevant loci prior to\nfine-grained mapping.\nCase Control\n7.338\n7.340\n7.342\n7.344\n7.346\n7.348Shannon Entropy\nHBB Haplotype 1\nCase Control\n7.338\n7.340\n7.342\n7.344\n7.346\n7.348\nHBB Haplotype 2\nFigure 3.Comparison of Shannon entropy values of HBB in case and control\ngroups. Both haplotypes show significantly lower entropy values in the case\ngroup compared to the control group.\nSynthetic validation of base-level localization accu-\nracy\nWe evaluate ATLAS on synthetic datasets to quantify\nrobustness under varying conditions, reporting metrics\naveraged across window sizes and haplotypes. The 4\nkb dataset with 100% allele frequency and 1,429 sam-\nples serves as the primary baseline.\nATLAS demonstrates high resilience to data sparsity\nand rare variants (Table 2, Figure S3). Notably, clear\nsignal concentration around synthetic target sites re-\nmained observable even at 10% allele frequency or\nwith as few as 10 individuals per group. Base-level\nmetrics (AUPRC, SNR, FRiW) show minimal degrada-\ntion (<10%) under these challenging conditions, while\ncluster-level precision and recall consistently remained\nabove 0.80. This confirms that ATLAS can reliably pri-\noritize rare risk variants even in small-scale studies.\nTable 2. Base-level localization and cluster-level recovery performance on 4 kb\nsynthetic sequences under varying cohort sizes and allele frequencies.\nCohor\nt Allele AUPRC↑SNR↑FRiW↑Prec.↑Recall↑Size (N) Freq.\nEff\nect of Allele Frequency (Cohort Size = 1,429)\n1,429 10% 0.350 9.430 0.150 0.845 0.688\n1,429 20% 0.369 10.041 0.159 0.875 0.875\n1,429 50% 0.385 10.028 0.160 0.889 1.000\n1,429 70% 0.384 9.731 0.157 0.889 1.000\n1,429 100% 0.382 9.213 0.150 0.845 1.000\nEff\nect of Cohort Size (Allele Frequency = 100%)\n200 100% 0.382 9.193 0.150 0.845 1.000\n100 100% 0.382 9.124 0.149 0.889 1.000\n50 100% 0.382 9.111 0.149 0.889 1.000\n20 100% 0.380 9.084 0.148 0.800 1.000\n10 100% 0.384 9.078 0.148 0.845 1.000\nSignal localization scales effectively across genomic\ncontexts. As shown in Table 3, we extend the\nevaluation to sequences ranging from 4 kb to 384\nkb. Length-normalized metrics demonstrate consis-\ntent performance: FRiW Enrichment scales with se-\nquence length, indicating effective signal isolation\nagainst expanding genomic backgrounds, while Nor-\nmalized Weighted Distance remained consistently low\n(< 0.4). Notably, for the longest sequences (384\nkb), cluster-level recovery achieves near-perfect scores\n(Precision/Recall≈1.0), validating the pipeline’s capa-\nbility to detect sparse signals in large genomic windows\n(Figure S4).\nTable 3. Performance scaling across increasing sequence lengths (4 kb to 384\nkb).\nSeq.\nFRiW Normalized Mean Prec.↑Recall↑Length Enrich. ↑ WDist↓ Rank↑\n4\nkb 8.03 0.373 0.926 0.844 1.000\n20 kb 19.04 0.328 0.908 0.909 1.000\n128 kb 63.41 0.252 0.908 0.950 0.950\n384 kb 129.82 0.325 0.915 1.000 1.000\nBase-level discovery of disease-associated regions\nWe next apply base-level differential attention analysis\nto top candidate genes, where ATLAS demonstrates su-\nperior recall of known disease-associated variants in\nHBBcompared to standard GWAS. We compare the\ndetected attention clusters against both GWAS results\n(p<6.3×10−9) and the 31 clinical variants reported in\nIthaGenes. As illustrated in Figure 4, while GWAS iden-\ntify only a single locus overlapping with known variants,\nATLAS detects 4 distinct clusters on haplotype 1 (cov-\nering 7 reported sites) and 3 clusters on haplotype 2\n(covering 6 reported sites). This confirms that attention-\nderived signals can recover a substantially larger pro-\nportion of clinically reported variants that are statistically\nelusive to standard association mapping.\nExtending the analysis to the other 15 candidate genes,\nATLAS shows strong agreement with standard methods\nwhile offering superior spatial resolution. Our method\nsuccessfully recovers all GWAS-significant loci (e.g., in\nLiu et al. |\n| 7\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nFigure 4.Comparision of the derived cluster, known loci, and GWAS inference on both haplotypes. Here we visualize the four genes that both GWAS and ATLAS\nhave found. The gray regions are the clusters with their cluster score showed as the colored dots. The red lines are the known loci. The orange star is the GWAS\ninferred locus.\nOR52A1,HBD) and identifies latent signals in 6 addi-\ntional genes. Notably, the detected clusters span an\naverage of only 11.62 bp. This fine-grained localization\nsignificantly narrows down the search space for putative\nrisk variants, providing much tighter candidate windows\ncompared to typical LD-based association blocks. Fur-\nthermore, ATLAS leverages haplotype information to re-\nveal phase-specific patterns—such as the asymmetric\nattention clusters inHBD —which are typically obscured\nin population-based GWAS signals.\nImpact of model capacity and pretraining diversity\non loci discovery\nFurthermore, we explore how model architecture and\npretraining data influence the localization of human dis-\nease variants. We benchmark models with varying\nscales and training domains—including the Evo2 series,\nLuca series, and Genos family—on the β-thalassemia\ncohort (Table 4).\nFirst, our observations indicate that model capacity is a\nsignificant factor in signal quality, particularly when the\ndomain aligns. Within the human-centric Genos fam-\nily, scaling up parameters yielded consistently cleaner\nsignals. As shown in Table 4, Genos-v2 substantially\nimproves metrics over Genos-1.2B (AUPRC: 0.173→\n0.265; SNR: 3.24→4.94), suggesting that larger mod-\nels benefit from enhanced semantic denoising capabili-\nties in non-informative regions.\nSecond, the results also suggest that scaling parame-\nters alone may not guarantee performance gains if the\npretraining data is not sufficiently aligned with the target\ntask. This is notable in the comparison with the gener-\nalist Evo2-40B model: despite its substantial parameter\ncount, it does not exhibit a commensurate performance\nadvantage in localizing human pathogenic variants (Avg\nAUPRC≈0.093), performing similarly to smaller base-\nlines. In contrast, Genos-v2, which benefits from exten-\nsive human genome diversity, consistently outperforms\ngeneralist counterparts.\nThe findings imply that for specific human disease\ntasks, achieving reliable localization performance likely\nrequires a synergy between sufficient model capacity\nand diverse human-centric pretraining.\nTable 4. Foundation-model comparison onβ-thalassemia (HBB) under ATLAS.\nModel Pretr\nain Avg. Avg. Avg. Avg.\ndomain AUC↑AUPRC↑SNR↑FRiW↑\nGenos-v2\nHuman (diverse) 0.807 0.265 4.941 0.169\nGenos-v1 Human 0.804 0.258 4.496 0.157\nGenos-1.2b Human 0.777 0.173 3.237 0.121\nEvo2-7b Generalist 0.712 0.147 2.566 0.096\nEvo2-1b Generalist 0.753 0.141 2.782 0.107\nEvo2-40b Generalist 0.709 0.093 2.139 0.082\nLucaOne Generalist 0.687 0.094 2.148 0.081\nLucaVirus Virus 0.695 0.087 1.908 0.074\nDiscussions\nConclusion.We present ATLAS, a framework that es-\ntablishes a new paradigm for population-level discovery\nthrough the lens of genomic language models. By de-\ncoding internal attention patterns, ATLAS bypasses the\nlimitations of traditional frequency-based statistics, en-\nabling the discovery of disease-associated loci directly\nfrom sequence semantics.\nOur results confirm that identifying risk variants does not\nstrictly require massive cohorts or clear statistical sepa-\nration; rather, actionable signals can be extracted from\nthe model’s intrinsic understanding of genomic syntax.\nATLAS thus positions itself not merely as a complement\nto GWAS, but as a critical instrument for illuminating\nthe \"genetic dark matter\"—including rare variants, com-\nplex haplotypes, and non-coding regions—that remains\ninaccessible to conventional methods. Ultimately, this\nwork paves the way for a future where genomic lan-\nguage models become the standard engine for decod-\ning the complex syntax of human disease.\nLiuet al.|\n| 8\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nLimitations and Future Work.First, this study focuses\non binary phenotypes driven by major genes; extending\nATLAS to complex, polygenic traits remains a key fu-\nture direction. Second, our analysis currently relies on\nreconstructed genomes with inherent processing noise.\nAs high-fidelity sequencing data becomes more acces-\nsible, ATLAS is expected to leverage these improved in-\nputs to naturally enhance localization precision without\narchitectural changes. Finally, while ATLAS effectively\nprioritizes candidate loci, establishing definitive causal-\nity warrants further empirical verification. Future work\nwill focus on validating these computational predictions\nthrough large-scale functional assays.\nEthics Statement\nTheβ-thalassemia dataset utilized in our research are\nfrom the study approved by the ethics committee of\nNanfang Hospital, Southern Medical University, and the\nethical committees of each local hospital participating\nin this study (Approval No. NFEC-2019-039). All sub-\njects and/or their guardians provided written informed\nconsent. The data supporting the findings of this study\nwill be made available upon reasonable request to the\ncorresponding author, subject to approval by the original\ndata custodian(s).\nReferences\nAdzhubei, I., Jordan, D. M., and Sunyaev, S. R. (2013). Predicting functional effect of human\nmissense mutations using PolyPhen-2.Curr. Protoc. Hum. Genet., Chapter 7(1):Unit7.20.\nAlkan, C., Coe, B. P ., and Eichler, E. E. (2011). Genome structural variation discovery and\ngenotyping. Nat. Rev. Genet., 12(5):363–376.\nAvsec, , Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R.,\nAssael, Y ., Jumper, J., Kohli, P ., and Kelley, D. R. (2021). Effective gene expression\nprediction from sequence by integrating long-range interactions. Nat. Methods, 18(10):\n1196–1203.\nBenegas, G., Batra, S. S., and Song, Y . S. (2023). DNA language models are power-\nful predictors of genome-wide variant effects. Proc. Natl. Acad. Sci. U. S. A., 120(44):\ne2311219120.\nBoshar, S., Evans, B., Tang, Z., Picard, A., Adel, Y ., Lorbeer, F . K., Rajesh, C., Karch, T.,\nSidbon, S., Emms, D., Mendoza-Revilla, J., Al-Ani, F ., Seitz, E., Schiff, Y ., Bornachot,\nY ., Hernandez, A., Lopez, M., Laterre, A., Beguir, K., Koo, P ., Kuleshov, V., Stark, A.,\nde Almeida, B. P ., and Pierrot, T. (2025). A foundational model for joint sequence-function\nmulti-species modeling at scale for long-range genomic prediction. bioRxiv.\nBrixi, G., Durrant, M. G., Ku, J., Poli, M., Brockman, G., Chang, D., Gonzalez, G. A., King,\nS. H., Li, D. B., Merchant, A. T., Naghipourfar, M., Nguyen, E., Ricci-Tam, C., Romero,\nD. W., Sun, G., Taghibakshi, A., Vorontsov, A., Y ang, B., Deng, M., Gorton, L., Nguyen,\nN., Wang, N. K., Adams, E., Baccus, S. A., Dillmann, S., Ermon, S., Guo, D., Ilango,\nR., Janik, K., Lu, A. X., Mehta, R., Mofrad, M. R. K., Ng, M. Y ., Pannu, J., Re, C.,\nSchmok, J. C., St. John, J., Sullivan, J., Zhu, K., Zynda, G., Balsam, D., Collison, P .,\nCosta, A. B., Hernandez-Boussard, T., Ho, E., Liu, M.-Y ., McGrath, T., Powell, K., Burke,\nD. P ., Goodarzi, H., Hsu, P . D., and Hie, B. (2025). Genome modeling and design across\nall domains of life with evo 2. bioRxiv.\nBrowning, S. R. and Browning, B. L. (2007). Rapid and accurate haplotype phasing and\nmissing-data inference for whole-genome association studies by use of localized haplo-\ntype clustering. Am. J. Hum. Genet., 81(5):1084–1097.\nChang, C. C., Chow, C. C., Tellier, L. C., Vattikuti, S., Purcell, S. M., and Lee, J. J. (2015).\nSecond-generation PLINK: rising to the challenge of larger and richer datasets. Giga-\nscience, 4(1):7.\nCheng, J., Novati, G., Pan, J., Bycroft, C., Žemgulyt ˙e, A., Applebaum, T., Pritzel, A., Wong,\nL. H., Zielinski, M., Sargeant, T., Schneider, R. G., Senior, A. W., Jumper, J., Hassabis, D.,\nKohli, P ., and Avsec, (2023). Accurate proteome-wide missense variant effect prediction\nwith AlphaMissense. Science, 381(6664):eadg7492.\nConsens, M. E., Diaz-Navarro, A., Chu, V., Stein, L., He, H. H., Moses, A., and Wang, B.\n(2025). Interpreting attention mechanisms in genomic transformer models: a framework\nfor biological insights. bioRxiv, pages 2025–06.\nCutting, G. R. (2015). Cystic fibrosis genetics: from molecular understanding to clinical\napplication. Nat. Rev. Genet., 16(1):45–56.\nDalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski,\nA. H., Oteri, F ., Dallago, C., Trop, E., de Almeida, B. P ., Sirelkhatim, H., Richard, G.,\nSkwark, M., Beguir, K., Lopez, M., and Pierrot, T. (2025). Nucleotide transformer: build-\ning and evaluating robust foundation models for human genomics. Nat. Methods, 22(2):\n287–297.\nDanecek, P ., Auton, A., Abecasis, G., Albers, C. A., Banks, E., DePristo, M. A., Handsaker,\nR. E., Lunter, G., Marth, G. T., Sherry, S. T., McVean, G., Durbin, R., and 1000 Genomes\nProject Analysis Group. (2011). The variant call format and VCFtools. Bioinformatics, 27\n(15):2156–2158.\nDao, T., Fu, D. Y ., Ermon, S., Rudra, A., and Ré, C. (2022). FlashAttention: Fast and\nmemory-efficient exact attention with IO-awareness. arXiv [cs.LG].\nEster, M., Kriegel, H., Sander, J., and Xu, X. (1996). A density-based algorithm for discover-\ning clusters in large spatial databases with noise. KDD, pages 226–231.\nGrissa, D., Junge, A., Oprea, T. I., and Jensen, L. J. (2022). Diseases 2.0: a weekly updated\ndatabase of disease-gene associations from text mining and data integration. Database\n(Oxford), 2022(baac019).\nHe, Y ., Fang, P ., Shan, Y ., Pan, Y ., Wei, Y ., Chen, Y ., Chen, Y ., Liu, Y ., Zeng, Z., Zhou, Z., Zhu,\nF ., Holmes, E. C., Y e, J., Li, J., Shu, Y ., Shi, M., and Li, Z. (2025). Generalized biological\nfoundation model with unified nucleic acid and protein language. Nat. Mach. Intell., 7(6):\n942–953.\nHimadewi, P ., Wang, X. Q. D., Feng, F ., Gore, H., Liu, Y ., Yu, L., Kurita, R., Nakamura, Y .,\nPfeifer, G. P ., Liu, J., and Zhang, X. (2021). 3’HS1 CTCF binding site in humanβ-globin\nlocus regulates fetal hemoglobin expression. Elife, 10(e70557).\nJaganathan, K., Kyriazopoulou Panagiotopoulou, S., McRae, J. F ., Darbandi, S. F ., Knowles,\nD., Li, Y . I., Kosmicki, J. A., Arbelaez, J., Cui, W., Schwartz, G. B., Chow, E. D., Kanterakis,\nE., Gao, H., Kia, A., Batzoglou, S., Sanders, S. J., and Farh, K. K.-H. (2019). Predicting\nsplicing from primary sequence with deep learning. Cell, 176(3):535–548.e24.\nKlein, R. J., Zeiss, C., Chew, E. Y ., Tsai, J.-Y ., Sackler, R. S., Haynes, C., Henning, A. K.,\nSanGiovanni, J. P ., Mane, S. M., Mayne, S. T., Bracken, M. B., Ferris, F . L., Ott, J.,\nBarnstable, C., and Hoh, J. (2005). Complement factor H polymorphism in age-related\nmacular degeneration. Science, 308(5720):385–389.\nKountouris, P ., Lederer, C. W., Fanis, P ., Feleki, X., Old, J., and Kleanthous, M. (2014).\nIthaGenes: an interactive database for haemoglobin variations and epidemiology. PLoS\nOne, 9(7):e103020.\nLin, A., Xie, B., Y e, C., Wang, C., Chen, D., Wang, E., Lu, F ., Xue, G., Zhang, H., Zhan, J.,\nZhang, J., Pang, J., Liang, J., Lin, J., Ma, J., Hu, J., Ma, J., Dong, J., Li, J., Liu, J., Chen,\nJ., Li, J., Ding, K., Deng, K., Chen, K., Wang, L., Liu, L., Guo, L., Xiong, L., Y ang, L.,\nCheng, M., Chen, N., Chen, R., Sun, S., Li, S., Chen, S., Liu, S., Xie, S., Liu, S., Zhou,\nT., Tang, W., Zhang, W., Jiang, X., Qi, X., Jin, X., Tan, X., Hu, X., Xu, X., Feng, X., Lu,\nY ., Gao, Y ., Shang, Y ., He, Y ., Yuan, Y ., Wang, Y ., Liu, Y ., Xiao, Z., Meng, Z., Li, Z., Zhao,\nZ., Y ang, Z., and Wang, Z. (2025). Genos: A human-centric genomic foundation model.\nGigascience, (giaf132).\nMarin, F . I., Teufel, F ., Horlacher, M., Madsen, D., Pultz, D., Winther, O., and Boomsma, W.\n(2023). BEND: Benchmarking DNA language models on biologically meaningful tasks.\narXiv [q-bio.GN].\nMontella, A., Tirelli, M., Lasorsa, V. A., Aievola, V., Cerbone, V., Manganiello, R., Iolascon,\nA., and Capasso, M. (2025). Regulatory non-coding somatic mutations as drivers of\nneuroblastoma. Br. J. Cancer, 132(5):469–480.\nPagel, K. A., Antaki, D., Lian, A., Mort, M., Cooper, D. N., Sebat, J., Iakoucheva, L. M.,\nMooney, S. D., and Radivojac, P . (2019). Pathogenicity and functional impact of non-\nframeshifting insertion/deletion variation in the human genome. PLoS Comput. Biol., 15\n(6):e1007112.\nRisch, N. and Merikangas, K. (1996). The future of genetic studies of complex human\ndiseases. Science, 273(5281):1516–1517.\nRosati, D., Palmieri, M., Brunelli, G., Morrione, A., Iannelli, F ., Frullanti, E., and Giordano,\nA. (2024). Differential gene expression analysis pipelines and bioinformatic tools for\nthe identification of specific biomarkers: A review. Comput. Struct. Biotechnol. J., 23:\n1154–1168.\nSchubert, E., Sander, J., Ester, M., Kriegel, H. P ., and Xu, X. (2017). DBSCAN revisited,\nrevisited: Why and how you should (still) use DBSCAN. ACM Trans. Database Syst., 42\n(3):1–21.\nSollis, E., Mosaku, A., Abid, A., Buniello, A., Cerezo, M., Gil, L., Groza, T., Güne¸ s, O., Hall,\nP ., Hayhurst, J., Ibrahim, A., Ji, Y ., John, S., Lewis, E., MacArthur, J. A. L., McMahon, A.,\nOsumi-Sutherland, D., Panoutsopoulou, K., Pendlington, Z., Ramachandran, S., Stefanc-\nsik, R., Stewart, J., Whetzel, P ., Wilson, R., Hindorff, L., Cunningham, F ., Lambert, S. A.,\nInouye, M., Parkinson, H., and Harris, L. W. (2023). The NHGRI-EBI GWAS catalog:\nknowledgebase and deposition resource. Nucleic Acids Res., 51(D1):D977–D985.\nStelzer, G., Rosen, N., Plaschkes, I., Zimmerman, S., Twik, M., Fishilevich, S., Stein, T. I.,\nNudel, R., Lieder, I., Mazor, Y ., Kaplan, S., Dahary, D., Warshawsky, D., Guan-Golan, Y .,\nKohn, A., Rappaport, N., Safran, M., and Lancet, D. (2016). The GeneCards suite: From\ngene data mining to disease genome sequence analyses. Curr. Protoc. Bioinformatics,\n54(1):1.30.1–1.30.33.\nTewhey, R., Bansal, V., Torkamani, A., Topol, E. J., and Schork, N. J. (2011). The importance\nof phase information for human genomics. Nat. Rev. Genet., 12(3):215–223.\nTomaz da Silva, P ., Karollus, A., Hingerl, J., Galindez, G. S. T., Wagner, N., Hernandez-Alias,\nX., Incarnato, D., and Gagneur, J. (2025). Nucleotide dependency analysis of genomic\nlanguage models detects functional elements. Nat. Genet., 57(10):2589–2602.\nTurner, T. N. and Eichler, E. E. (2019). The role of DE novo noncoding regulatory mutations\nin neurodevelopmental disorders. Trends Neurosci., 42(2):115–127.\nVaser, R., Adusumalli, S., Leng, S. N., Sikic, M., and Ng, P . C. (2016). SIFT missense\npredictions for genomes. Nat. Protoc., 11(1):1–9.\nVig, J. and Belinkov, Y . (2019). Analyzing the structure of attention in a transformer language\nmodel. arXiv [cs.CL].\nVisscher, P . M., Wray, N. R., Zhang, Q., Sklar, P ., McCarthy, M. I., Brown, M. A., and Y ang,\nJ. (2017). 10 years of GWAS discovery: Biology, function, and translation. Am. J. Hum.\nGenet., 101(1):5–22.\nWriting Group For Practice Guidelines For Diagnosis And Treatment Of Genetic Diseases\nMedical Genetics Branch Of Chinese Medical Association, Shang, X., Wu, X., Zhang, X.,\nFeng, X., and Xu, X. (2020). Clinical practice guidelines for beta-thalassemia. Zhonghua\nYi Xue Yi Chuan Xue Za Zhi, 37(3):243–251.\nLiu et al. |\n| 9\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nWu, M. C., Lee, S., Cai, T., Li, Y ., Boehnke, M., and Lin, X. (2011). Rare-variant association\ntesting for sequencing data with the sequence kernel association test. Am. J. Hum.\nGenet., 89(1):82–93.\nZhan, X., Hu, Y ., Li, B., Abecasis, G. R., and Liu, D. J. (2016). RVTESTS: an efficient and\ncomprehensive tool for rare variant association analysis using sequence data. Bioinfor-\nmatics, 32(9):1423–1426.\nLiu et al. |\n | 10\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nSupplementary Note 1: Implementation Details & Reproducibility\nTo ensure the reproducibility of our experiments and the scalability of our approach to long-context genomic se-\nquences, we provide detailed specifications of our computational environment, attention extraction algorithms, and\nhyperparameter optimization strategies.\nA. Computational Environment\nAll models were trained and evaluated on a high-performance computing node optimized for large-scale deep learn-\ning.\n•Hardware Infrastructure: A single node equipped with 4×NVIDIA A40 GPUs (48GB VRAM per GPU). This\nsetup utilizes the Ampere architecture to support efficient bfloat16 mixed-precision training.\n• Software Stack:\n– Framework: PyTorch2.5.1+cu124 with Python 3.12.2.\n– Acceleration: FlashAttention-2 (v2.8.3) was employed to optimize the memory hierarchy (HBM vs.\nSRAM), significantly reducing the IO overhead for attention matrix computation.\n– CUDA Toolkit: Version12.4.\nB. Attention Score Extraction & Alignment\nUnlike standard NLP tasks where token indices align linearly with positions, genomic analysis requires precise\nmapping between attention weights and biological coordinates, especially in the presence of Indels.\nB.1. FlashAttention-based Extraction Logic. We extract raw per-token attention importance scores directly from the\nTransformer layers. To handle the quadratic complexity of attention in long sequences, we implement a block-wise\nextraction algorithm based on FlashAttention logic. The detailed procedure is formally described in Algorithm 1.\nAlgorithm 1FlashAttention-based Per-token Attention Extraction\n1:Input:Sequence S, modelfθ, block sizeBr, causal flagC\n2: Output: Attention importance vector v∈RL\n3: Q∈RB×Hq×L×D, K∈RB×Hkv×L×D←Hook(fθ,S)\n4: ifHkv 128k), we performed an ablation study to\nselect the optimal window sizeC and overlapO. We measured the Area Under the Precision-Recall Curve (AUPRC)\nas a proxy for the model’s Signal-to-Noise Ratio (SNR) in detecting regulatory variants.\nAs shown in Figure S1, we evaluated chunk sizes ranging from 4k to 32k. The configuration of C = 8,192 (Chunk\nSize) and O = 4,096 (Overlap) yielded the highest AUPRC across multiple sequence lengths (green line). While\nlarger chunks (32k) theoretically offer more context, our analysis suggests they introduce excessive background\nnoise that dilutes the local signal, in addition to higher computational overhead (see Figure S2). Conversely, smaller\nchunks (4k) fracture long-range dependencies essential for distal regulation. Thus, the 8k/4k setup provides the\noptimal trade-off between signal fidelity and computational cost.\nSupplementary Note 2: Data & Experiments Details\nA. Real Dataset Characteristics\nA.1. Sample Classification and Ground Truth. We designed this real-world evaluation to assess whether ATLAS can\nrecover known related loci under realistic population heterogeneity and identify plausible disease-associated signals\nbeyond curated annotations.\nTheβ-thalassemia dataset was retrieved from a cross-sectional whole-genome sequencing study investigating the\nclinical heterogeneity of hemoglobinopathies. The cohort comprises 1,429 individuals, classified into three groups\nbased on clinical severity:\n• Carriers (N = 409): Individuals who generally exhibit no overt clinical symptoms, although mild anemia and\nmicrocytosis may be observed in hematological tests.\nLiu et al. | Supplementary Information | 12\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nFigure S1. Ablation Study - Performance Analysis (AUPRC).Comparison of variant detection accuracy across varying chunk\nsizes and overlaps. The 8k chunk size with 4k overlap (green line) consistently shows the optimal trade-off between signal\ndetection and context capture.\nFigure S2. Ablation Study - Computational Cost Analysis. Total processing time (hours) and chunk generation overhead.\nWhile larger chunks (32k) reduce total time, they degrade detection performance (as shown in Figure.S1). The 8k size maintains\na reasonable computational cost.\n• Mild Cases (Thalassemia Intermedia, TI, N = 245): Classified as non-transfusion dependent thalassemia\n(NTDT), characterized by low or intermittent transfusion dependence and relatively milder clinical manifesta-\ntions.\n• Severe Cases (Thalassemia Major, TM, N = 775): Classified as transfusion-dependent thalassemia (TDT),\nLiu et al. | Supplementary Information | 13\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\ncharacterized by high transfusion dependence and severe disease progression.\nThe clinical classification of these samples serves as the ground truth labels for all subsequent analyses.\nA.2. Genomic Scope.We analyzed all protein-coding genes located on chromosome 11 with lengths shorter than\n512 kb. This comprises a total of 1,031 genes. The length distribution is as follows:\n•< 64 kb: 1,049 genes (Note: strictly adhering to the <512kb filter).\n• 64 kb−128 kb: 145 genes.\n• 128 kb−256 kb: 72 genes.\n• 256 kb−512 kb: 35 genes.\nB. Data Processing Pipeline\nB.1. Phasing and Haplotype Construction. Haplotype phasing was performed on the original VCF files using Beagle\nv4. Default parameters were applied with genotype probability output enabled (gp=true) and a sliding window size\nofwindow=10.0. This procedure phased unphased genotypes into two haplotypes (hap1andhap2) for each sample.\nB.2. Sequence Construction and Coordinate Mapping (VCF2CSV). We constructed haplotype-specific sequences by\nmapping VCF variants to the GRCh38 reference genome. For each sample, sequences were processed from the 5’\nend to the 3’ end to handle coordinate shifts dynamically. The mapping rules are defined as follows:\n1. No Variant: The reference base is retained, and its absolute genomic position is recorded.\n2. SNPs: The reference base is replaced by the alternative allele, recording the original reference position.\n3. Deletions: Bases are excluded from the sequence, and their positions are not recorded (only remaining bases\nretain position tags).\n4. Insertions: Inserted bases are included sequentially. Crucially, the position of the preceding reference base is\nrepeated for each inserted base to maintain alignment with the reference coordinate system.\nStrand Handling: For genes located on the negative strand, the generated sequences were reverse-complemented,\nand the corresponding position arrays were reversed to maintain alignment with the reference genome coordinate\norder.\nC. Synthetic Dataset Generation\nWe provide the detailed statistics of the synthetic datasets constructed for robustness evaluation in Table 7. The\nsynthetic generation process involved injecting variants into intergenic regions derived from chromosome 11.\nTable 7. Statistics of the Synthetic Datasets across different sequence lengths.\nSeq Length Carriers Mild Severe Target Variants\n(SNP/Indel)\nTarget\nProp.\n4 kb 409 245 775 8/0 100%\n20 kb 409 245 775 8/2 100%\n128 kb 409 245 775 15/5 100%\n384 kb 409 245 775 15/5 100%\nD. Baseline Settings\nD.1. Foundation Model.We utilize Genos-10b-v1 as the foundation model baseline. The input sequences are padded\nor truncated to match the model’s maximum context length where necessary.\nD.2. GWAS Configuration. Genome-Wide Association Studies (GWAS) are conducted using PLINK2. We apply a\nunified threshold strategy for fair comparison.\n• Quality Control (QC): Variants with missingness> 1% (–geno 0.01) and Minor Allele Frequency< 1% (–maf\n0.01) were removed.\n• Locus Identification: A GWAS locus is counted as co-identified if its position falls within an attention-derived\ncluster interval defined as:\nLiu et al. | Supplementary Information | 14\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nListing 1.GWAS Pipeline Commands (PLINK2)\n# 1. Quality Control and VCF to BED c onv e r si on\nplink2 -- vcf / path / to / dataset / a u t o s o m e s _ s p l i t . vcf . gz \\\n-- make - bed \\\n-- out / path / to / output / t h a l _ 1 4 2 9 _ a u t o \\\n-- chr 1 -22 \\\n-- geno 0.01 \\\n-- maf 0.01\n# 2. A s s o c i a t i o n Analysis ( GLM )\n# Note : - -1 ensures output uses 1 - based c o o r d i n a t e s\nplink2 -- bfile / path / to / output / t h a l _ 1 4 2 9 _ a u t o \\\n-- glm omit - ref \\\n-- pheno / path / to / phe n o t y pe s / p h e n o _ b i n a r y . txt \\\n-- covar / path / to / cov a r i a te s / c o v a r _ f i n a l . txt \\\n-- out / path / to / results / g w a s _ r e s u l t s \\\n- -1\n–Forward strand:[start−10,end]\n–Reverse strand: [start,end + 10]\nThe specific PLINK2 commands used for Quality Control (QC) and Association Analysis are listed below:\nSupplementary Note 3: Baseline Model Configurations\nTo benchmark our approach, we compare it against state-of-the-art genomic language models in different pre-train\nscopes to verify the influences of model sizes and training data on downstream tasks. Table 8 summarizes the\narchitectures and specifications. Notably, all selected models utilize a single-nucleotide tokenizer, ensuring a fair\ncomparison at the base resolution level. Regarding the Evo 2 series, given its hybrid StripedHyena architecture\nwhere attention heads are interspersed with convolution operators, we identified the best-performing layers reported\nin the original study and selected the nearest attention layer immediately preceding them for analysis.\nTable 8. Summary of Baseline Models. The \"Extraction Layer\" column indicates which layer’s attention weights were used for\nanalysis.\nModel\nName Architecture Pre-train Scope Data Volume Context Length Param Size Extraction Layer\nEv\no2-1B-Base StripedHyena 2 Multispecies 1T bp 8K bp 1.1B Layer 10\nEvo2-7B-Base StripedHyena 2 Multispecies 2.1T bp 8K bp 6.5B Layer 24\nEvo2-40B StripedHyena 2 Multispecies 9.3T bp 1M bp 40B Layer 17\nGenos-1.2B\nMoE (Mixture of Experts) Human 1600B tokens 1M 1.2B Last Layer\nGenos-10B MoE (Mixture of Experts) Human 2200B tokens 1M 10B Last Layer\nGenos-10B-v2 MoE (Mixture of Experts) Human 6286B tokens 1M 10B Last Layer\nLucaOne\nTransformer Multispecies 36.95B bp 1k 1.8B Last Layer\nLucaVirus Transformer Viral sequences 50B bp 3k 1B Last Layer\nSupplementary Note 4: Evaluation Metrics for Signals\nA. Overview\nTo evaluate the spatial concentration of differential signals around true variant positions, we develop a set of evalu-\nation metrics. These metrics quantify how well the predicted signals localize to regions surrounding known variants.\nTo enable fair comparison across sequences of different lengths (e.g., 4 kb vs. 384 kb), we additionally introduce\nlength-normalized variants of key metrics.\nB. Notation\nLets= (s 1,s 2,...,sN)denote the signal scores (absolute log 2 fold-change of attention) atNgenomic positions,\nwherepi denotes the genomic coordinate of positioni. Given a set of K true variant positionsV ={v1,v 2,...,vK},\nLiu et al. | Supplementary Information | 15\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nwe define a binary labelyi∈{0,1}indicating whether positioni falls within a window around any variant:\nyi =\n{\n1 if∃v∈V:v−wL≤pi≤v +wR\n0 otherwise (S1)\nwherewL andwR are the left and right window boundaries, respectively (tested at 3 bp, 5 bp, 10 bp, and 20 bp). We\nfurther defineT ={i:yi = 1}as the set of positions within true variant windows (positives),F ={i:yi = 0}as the\nset of background positions (negatives), with nT =|T|andnF =|F|denoting their respective counts. For spatial\nmetrics, we definedi = minv∈V|pi−v|as the distance from positioni to the nearest variant.\nC. Primary Metric: AUPRC\nWe adopt the Area Under the Precision–Recall Curve (AUPRC) as the primary metric to quantify ranking per-\nformance under severe class imbalance. AUPRC summarizes the precision-recall trade-off across all classification\nthresholds:\nAUPRC =\n∑\nk\n(Rk−Rk−1)·Pk (S2)\nwherePk andRk are the precision and recall at thek-th threshold. AUPRC is particularly informative when positive\ncases are rare. The baseline (random classifier) AUPRC equals the proportion of positives (n T/N); thus, AUPRC\nvalues should be interpreted relative to this baseline when comparing across sequences of different lengths. As\nshown in Figure S7, we evaluated performance across varying window sizes to ensure robustness.\nD. Complementary Metrics\nTo overcome the limitations of threshold-based metrics, we introduce complementary indicators measuring magni-\ntude contrast, signal efficiency, and spatial precision.\nD.1. Signal-to-Noise Ratio (SNR). Categorized as Magnitude Contrast, SNR measures how biologically distinct the\nvariant signal is from the background noise floor:\nSNR = ¯sT\n¯sF\n= nF\nnT\n·\n∑\ni∈Tsi∑\ni∈Fsi\n(S3)\nwhere¯sT and¯sF denote the mean signal in true variant windows and background regions, respectively. SNR > 1\nindicates that signals are, on average, stronger near variants than in background regions; higher values indicate\nbetter signal specificity. Figure S8 illustrates the SNR distribution.\nD.2. Fraction of Signal in Windows (FRiW). Categorized as Signal Efficiency, this metric is analogous to the FRiP\nscore in ChIP-seq. It quantifies the proportion of total signal that falls within true variant windows:\nFRiW =\n∑\ni∈Tsi\n∑N\ni=1si\n(S4)\nA low FRiW implies that despite a potentially high AUPRC, the majority of the model’s attention mass is allocated to\nregions outside variant windows (see Figure S9).\nD.3. Signal-Weighted Mean Distance. Categorized as Spatial Precision (Threshold-Free), this metric measures the\naverage distance of signal from the nearest variant, weighted by signal intensity, removing the need for hard window\nboundaries:\nWeighted Distance =\n∑N\ni=1si·di\n∑N\ni=1si\n(S5)\nAlower score is better, indicating that the attention mass is concentrated physically closer to the causal variants\n(see Figure S10).\nE. Length-normalized Metrics\nTo enable fair comparison across sequences of vastly different lengths, we introduce length-normalized variants\nof the above metrics. These normalized metrics account for the expected baseline values under uniform signal\ndistribution, making them suitable for cross-scale comparisons.\nLiu et al. | Supplementary Information | 16\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nE.1. FRiW Enrichment.To account for varying sequence lengths and window sizes, we normalize FRiW by the ex-\npected fraction under a uniform signal distribution:\nFRiWexpected = nT\nN (S6)\nFRiW Enrichment= FRiW\nFRiWexpected\n= N\nnT\n·\n∑\ni∈Tsi\n∑N\ni=1si\n(S7)\nThis fold-enrichment metric is length-independent. A value of 1 implies a signal distribution equivalent to random\nexpectation (uniform background); values > 1 indicate a significant enrichment of attention mass within variant\nwindows.\nE.2. Normalized Weighted Distance. To enable comparison across sequences of different lengths, we normalize the\nweighted distance by the characteristic length scale of the sequence. Under a uniform distribution assumption with\nK variants, the expected distance to the nearest neighbor scales linearly withL/K. We therefore define:\nNormalized Weighted Distance = Weighted Distance\nL/(2K) (S8)\nThis normalization factorL/(2K)serves as a first-order approximation of the expected random distance, facilitating\nfair comparison of spatial accuracy across varying genomic scales.\nE.3. Mean Percentile Rank of True Positions. This rank-based metric evaluates where true variant positions fall in the\nranked signal distribution:\nMean RankT = 1\nnT\n∑\ni∈T\nR(si)\nN−1 (S9)\nwhereR(s i)is the rank of si among all scores (0 for lowest, N−1 for highest). This metric ranges from 0 to 1,\nwhere 0.5 indicates performance equivalent to random ranking, and values approaching 1 indicate that positions\nnear variants consistently have high signal scores. As a rank-based metric, Mean Percentile Rank is completely\nlength-independent.\nSupplementary Note 5: Gene-Level Statistical Descriptors\nThe gene-level analysis aims to identify distribution differences of attention scores across entire genes. For a gene\nwithM positions and attention scores s = (s1,s 2,...,sM), we compute 17 descriptors categorized into four groups.\nIn our experiments, Max, Std (σ), Top5%Mean, CV, Median, IQR, and Entropy provide the most significant sepa-\nration between the informative (HBB) and control genes, suggesting that both the magnitude of extreme values and\nthe overall distribution shape are informative for distinguishing regulatory patterns.\nA. Location/Scale (10 metrics)\n• Mean: The arithmetic mean of all scores:\nµ= 1\nM\n∑M\nj=1\nsj (S10)\n•Median:The middle value when scores are sorted, i.e., Q0.50.\n• Top5% Mean: Mean of the top 5% highest scores:\nTop5% Mean= 1\n|H|\n∑\nj∈H\nsj,whereH={j: sj≥Q0.95} (S11)\n• Low5% Mean: Mean of the bottom 5% lowest scores:\nLow5% Mean = 1\n|L|\n∑\nj∈L\nsj,whereL= {j:sj≤Q0.05} (S12)\n• Max: The maximum score across all positions:\nMax = max\nj∈{1,...,M}\nsj (S13)\nLiu et al. | Supplementary Information | 17\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\n•Standard Deviation:Measures the spread of scores around the mean:\nσ=\n√\n1\nM−1\n∑M\nj=1\n(sj−µ)2 (S14)\n•Coefficient of Variation (CV):Scale-normalized dispersion:\nCV= σ\nµ (S15)\n•Interquartile Range (IQR): The range between the 25th and 75th percentiles:\nIQR =Q0.75−Q0.25 (S16)\n• 10th Percentile (Q0.10) and 90th Percentile (Q0.90): Values below which 10% and 90% of scores fall, respec-\ntively.\nB. Distribution Shape (3 metrics)\n• Skewness: The standardized 3rd central moment, measuring distribution asymmetry:\nSkewness = 1\nM\n∑M\nj=1\n(sj−µ\nσ\n)3\n(S17)\nPositive skewness indicates a right-tailed distribution; negative skewness indicates a left-tailed distribution.\n•Kurtosis:The standardized 4th central moment, measuring tail heaviness:\nKurtosis = 1\nM\n∑M\nj=1\n(sj−µ\nσ\n)4\n−3(S18)\nValues> 0 (leptokurtic) indicate heavy tails; values< 0 (platykurtic) indicate light tails.\n• Mode: The most frequent value, computes via a fixed-bin histogram as the center of the bin with maximum\ncount.\nC. Peak Structure (3 metrics)\nTo capture local regulatory motifs and identify regions of concentrated attention:\n• Peak Count: Number of local maxima identified, where positionj is a local maximum ifsj>sj−1andsj>sj+1.\n• Peak Density: PeakCount normalized by gene length:\nPeak Density = PeakCount\nM (S19)\n•Peak Mean:Mean attention score at peak summits:\nPeak Mean = 1\n|P|\n∑\nj∈P\nsj (S20)\nwherePis the set of positions identified as local maxima.\nD. Information (1 metric)\n•Shannon Entropy: Measures the sparsity or concentration of the attention distribution. After normalizing scores\nto a probability distributionpj =sj/∑M\nk=1sk, the entropy is computed as:\nH(s) =−\n∑M\nj=1\npj log(pj +ϵ) (S21)\nwhereϵis a small constant for numerical stability. Lower entropy indicates more concentrated (sparse) attention;\nhigher entropy indicates more uniform distribution.\nLiu et al. | Supplementary Information | 18\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nFigure S3. ATLAS results under different allele frequencies and cohort sizes. Red vertical lines are the positions of the\nsynthetic variants. Dots are the bases with significant attention differences. The gray areas are the clusters.\nFigure S4. ATLAS results under different sequence lengths Notations are the same as Figure S3. The gray areas may not\nbe clearly visible in long sequences.\nLiu et al. | Supplementary Information | 19\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\n0 1 2 3 4 5 6 7 8\n-log10(P-value)\nOR51V1\nHMBS\nOR51M1\nCLNS1A\nUBQLN3\nLDHA\nNDUFC2\nHBG1\nPATL1\nOR52J3\nNUP98\nNDUFC2-KCTD14\nOR51F1\nHBB\nMean\n(14 Significant Genes)\nP=0.05\n0 2 4 6 8 10\n-log10(P-value)\nOR51B4\nENSG00000284931\nUBQLN3\nHBD\nOR51Q1\nOR51C1P\nOR56A4\nSCGB2A2\nOR52I2\nOR52J3\nOR51E2\nMedian\n(11 Significant Genes)\nP=0.05\n0 1 2 3 4 5\n-log10(P-value)\nHMBS\nOR52E2\nHBG1\nOR51C1P\nOR52J3\nUBQLN3\nOR51B4\nTop 5% Mean\n(7 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6 7 8\n-log10(P-value)\nHBB\nOR51B4\nOR51I2\nUBQLN3\nOR51E2\nOR51B6\nOR4D9\nSERPING1\nENSG00000284931\nUBQLNL\nLow 5% Mean\n(10 Significant Genes)\nP=0.05\n0 2 4 6 8 10\n-log10(P-value)\nHBB\nCLNS1A\nHBD\nHBG1\nOR52E2\nHMBS\nUBQLN3\nSCGB2A2\nOR51F1\nELP4\nOR51B4\nMax\n(11 Significant Genes)\nP=0.05\n0 2 4 6 8\n-log10(P-value)\nHMBS\nOR51V1\nHBG1\nHBD\nOR52E2\nOR52J3\nUBQLN3\nCLNS1A\nOR51B4\nOR52E1\nOR51C1P\nOR51Q1\nStandard Deviation\n(12 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6 7 8\n-log10(P-value)\nHMBS\nOR51V1\nHBD\nHBG1\nOR52E2\nOR52J3\nUBQLN3\nCLNS1A\nOR51B4\nUBQLNL\nNDUFC2\nOR52E1\nPATL1\nOR51Q1\nCoefficient of Variation (CV)\n(14 Significant Genes)\nP=0.05\n0 1 2 3 4 5\n-log10(P-value)\nOR52E2\nHBG1\nOR51B4\nOR51B6\nHMBS\nHBD\nLRP4\nSCGB2A2\nNDUFC2\nUBQLN3\nOR52E5\nHBB\nB3GAT3\nSERPING1\nOR51V1\nInterquartile Range (IQR)\n(15 Significant Genes)\nP=0.05\n0 5 10 15 20 25\n-log10(P-value)\nHBB\nOR51B4\nHBD\nOR52J3\nENSG00000284931\n10th Percentile\n(5 Significant Genes)\nP=0.05\n0 1 2 3 4\n-log10(P-value)\nNUP98\nHBD\nENSG00000284931\nHBG1\nOR51I2\nOR51B6\nHMBS\nOR52E2\nOR52E1\nLDHA\nOR51B4\nOR51Q1\n90th Percentile\n(12 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6 7 8\n-log10(P-value)\nOR51B4\nHBB\nOR52E1\nOR52J3\nOR51F1\nHBG1\nOR51C1P\nOR52E2\nSkewness\n(8 Significant Genes)\nP=0.05\n0 2 4 6 8 10 12\n-log10(P-value)\nHBB\nOR51B4\nOR52E1\nOR52E2\nOR52J3\nFGF19\nHBG1\nOR51C1P\nOR51F1\nHMBS\nOR52I2\nKurtosis\n(11 Significant Genes)\nP=0.05\n0 2 4 6 8 10\n-log10(P-value)\nHBD\nHBB\nOR51V1\nHBG1\nENSG00000284931\nOR51B4\nHARBI1\nOR51F1\nMode\n(8 Significant Genes)\nP=0.05\n0 2 4 6 8\n-log10(P-value)\nOR51V1\nENSG00000284931\nOR56A4\nOR52E1\nUBQLN3\nOR51E2\nOR52J3\nHBB\nPGA3\nCWF19L2\nOR51I1\nOR51I2\nSCGB1A1\nCCDC153\nHMBS\nPeak Count\n(15 Significant Genes)\nP=0.05\n0 2 4 6 8\n-log10(P-value)\nOR51V1\nENSG00000284931\nHBG1\nOR56A4\nOR52E1\nHBB\nUBQLN3\nOR51E2\nOR52J3\nCWF19L2\nHMBS\nPGA3\nOR51I1\nCCDC153\nSCGB1A1\nOR51I2\nCCDC88B\nPeak Density\n(17 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6 7\n-log10(P-value)\nENSG00000284931\nHMBS\nOR52E1\nHBG1\nOR51C1P\nOR56A4\nOR51E2\nUBQLN3\nOR52J3\nOR52A1\nPeak Mean\n(10 Significant Genes)\nP=0.05\n0 5 10 15 20 25 30 35 40\n-log10(P-value)\nHBB\nHMBS\nOR52A1\nOR52E2\nOR52E1\nOR52J3\nHBG1\nOR4D9\nOR51B4\nOR51C1P\nHBD\nOR51F1\nNDUFC2\nShannon Entropy\n(13 Significant Genes)\nP=0.05\nFigure S5. The significant genes selected by the 17 distribution descriptor in the gene-level analysis. The results are\nretrieved from the haplotype 1 attention scores\n0 1 2 3 4 5 6\n-log10(P-value)\nCLNS1A\nHMBS\nOR51V1\nMean\n(3 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0\n-log10(P-value)\nOR51E2\nOR51L1\nOR52J3\nOR51C1P\nOR52M1\nMedian\n(5 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5\n-log10(P-value)\nOR51B4\nDENND5A\nHMBS\nOR52J3\nTop 5% Mean\n(4 Significant Genes)\nP=0.05\n0 1 2 3 4 5\n-log10(P-value)\nHBB\nOR51E2\nLow 5% Mean\n(2 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6 7 8\n-log10(P-value)\nHBB\nCLNS1A\nHMBS\nOR51B4\nMax\n(4 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6\n-log10(P-value)\nHMBS\nOR51V1\nOR51B4\nCCKBR\nOR52J3\nFGF19\nCCDC153\nCLNS1A\nOR52E1\nStandard Deviation\n(9 Significant Genes)\nP=0.05\n0 1 2 3 4 5 6\n-log10(P-value)\nHMBS\nOR51V1\nOR51B4\nOR52J3\nCCKBR\nCCDC153\nCLNS1A\nNUP98\nCoefficient of Variation (CV)\n(8 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5\n-log10(P-value)\nHMBS\nOR52E2\nFNBP4\nOR51C1P\nOR52N1\nOR51L1\nOSBP\nInterquartile Range (IQR)\n(7 Significant Genes)\nP=0.05\n0 5 10 15 20 25\n-log10(P-value)\nHBB\nOR52J3\n10th Percentile\n(2 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0\n-log10(P-value)\nOR51B4\nOR56B1\nOR51E2\n90th Percentile\n(3 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0\n-log10(P-value)\nHBB\nOR52A1\nOR52J3\nOR52E1\nOR52M1\nOR51B4\nCCDC153\nPGAP2\nSkewness\n(8 Significant Genes)\nP=0.05\n0 2 4 6 8 10\n-log10(P-value)\nHBB\nOR52J3\nOR52M1\nOR52E1\nOR51B4\nKurtosis\n(5 Significant Genes)\nP=0.05\n0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00\n-log10(P-value)\nOR51V1\nOR51B4\nHBD\nHBB\nOR51S1\nMode\n(5 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5\n-log10(P-value)\nOR51V1\nOR52J3\nOR52E1\nHBB\nOR51L1\nOR51E2\nOR52L1\nBAD\nPeak Count\n(8 Significant Genes)\nP=0.05\n0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5\n-log10(P-value)\nOR51V1\nHBB\nOR51L1\nOR52J3\nOR52E1\nOR52L1\nBAD\nPeak Density\n(7 Significant Genes)\nP=0.05\n0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00\n-log10(P-value)\nOR51E2\nOR52J3\nOR52E1\nPeak Mean\n(3 Significant Genes)\nP=0.05\n0 5 10 15 20 25 30\n-log10(P-value)\nHBB\nOR52A1\nOR51B4\nCCKBR\nENSG00000254979\nOR52J3\nHMBS\nFGF19\nOR52E2\nOR52E1\nShannon Entropy\n(10 Significant Genes)\nP=0.05\nFigure S6. The significant genes selected by the 17 distribution descriptor in the gene-level analysis. The results are\nretrieved from the haplotype 2 attention scores\nLiu et al. | Supplementary Information | 20\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.05\n0.10\n0.15\n0.20\n0.25AUPRC (Log2FC)\nWindow = 3bp\nEvo2_1b\nEvo2_40b\nEvo2_7b\nGenos-1.2b\nGenos-v1\nGenos-v2\nLucaOne\nLucaVirus\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.05\n0.10\n0.15\n0.20\n0.25\n0.30AUPRC (Log2FC)\nWindow = 5bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.05\n0.10\n0.15\n0.20\n0.25\n0.30\n0.35\n0.40AUPRC (Log2FC)\nWindow = 10bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.1\n0.2\n0.3\n0.4\n0.5AUPRC (Log2FC)\nWindow = 20bp\nFigure S7. Evaluation of Detection Quality - AUPRC Analysis (Higher is better).Evaluation of ranking performance across\ndifferent window sizes (3bp, 5bp, 10bp, 20bp). While AUPRC measures the ranking order, the Genos-v2 model (brown) consis-\ntently demonstrates strong ranking capabilities across most haplotype comparisons.\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n1\n2\n3\n4\n5\n6SNR (Log2FC)\nWindow = 3bp\nEvo2_1b\nEvo2_40b\nEvo2_7b\nGenos-1.2b\nGenos-v1\nGenos-v2\nLucaOne\nLucaVirus\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n2\n3\n4\n5\n6SNR (Log2FC)\nWindow = 5bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n2\n3\n4\n5SNR (Log2FC)\nWindow = 10bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n2\n3\n4\n5\n6SNR (Log2FC)\nWindow = 20bp\nFigure S8. Evaluation of Detection Quality - Signal-to-Noise Ratio (SNR) Analysis (Higher is better). Assessing the\nmagnitude contrast between variant signals and background noise. A higher SNR confirms that the identified signals have a\nsignificant magnitude difference compared to the background, indicating that the model’s attention peaks at risk variants are\nbiologically distinct from the noise floor.\nLiu et al. | Supplementary Information | 21\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint \n\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.02\n0.03\n0.04\n0.05\n0.06\n0.07\n0.08\n0.09\n0.10FRiW (Log2FC)\nWindow = 3bp\nEvo2_1b\nEvo2_40b\nEvo2_7b\nGenos-1.2b\nGenos-v1\nGenos-v2\nLucaOne\nLucaVirus\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.04\n0.06\n0.08\n0.10\n0.12FRiW (Log2FC)\nWindow = 5bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.06\n0.08\n0.10\n0.12\n0.14\n0.16\n0.18\n0.20\n0.22FRiW (Log2FC)\nWindow = 10bp\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n0.10\n0.15\n0.20\n0.25\n0.30\n0.35FRiW (Log2FC)\nWindow = 20bp\nFigure S9. Evaluation of Attention Efficiency - Fraction of Signal in Windows (FRiW) (Higher is better). Quantifying the\nglobal attention budget allocation. Models with higher FRiW scores are more efficient, concentrating their attention mass into the\nrelevant variant windows rather than dispersing it across the sequence.\nHap1\n1v2\nHap2\n1v2\nHap1\n2v3\nHap2\n2v3\nHap1\n1v3\nHap2\n1v3\n50\n60\n70\n80\n90\n100\n110\n120WDist (Log2FC)\nEvo2_1b\nEvo2_40b\nEvo2_7b\nGenos-1.2b\nGenos-v1\nGenos-v2\nLucaOne\nLucaVirus\nFigure S10. Evaluation of Spatial Precision - Distance-Weighted Mean (Lower is better). A threshold-free metric measuring\nspatial precision. Lower values indicate that high attention scores are physically closer to the target variants, minimizing spatial\ndeviation.\nLiu et al. | Supplementary Information | 22\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted February 10, 2026. ; https://doi.org/10.64898/2026.02.09.704696doi: bioRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}