Introducing STRAND: A Foundational Sequence Transformer for Range Adaptive Nucleotide Decoding

Introducing STRAND: A Foundational Sequence Transformer for Range Adaptive Nucleotide Decoding | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Introducing STRAND: A Foundational Sequence Transformer for Range Adaptive Nucleotide Decoding Shant Ayanian, Panos Korfiatis, Collin Osborne, Carl Molnar, Marc Blasi, and 14 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6115078/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The advent of high-throughput sequencing has led to an exponential increase in genomic data, highlighting the need for efficient and accurate models to analyze and interpret this information. Here, we introduce a novel, exomic foundational model that leverages a combination of human reference genome and multispecies data to improve variant detection and interpretation. Our model utilizes a short- range transformer architecture and is trained on a large dataset of human exomic sequences derived from the Tapestry study. Through a series of ablation studies and scaling experiments, we demonstrate the effectiveness of our model in pre- dicting next token accuracy and identifying clinically pathogenic variants. We also show that our model outperforms existing models in a range of downstream tasks, including variant effect prediction and disease state identification. In fact, our largest STRAND variant (1B parameters) surpassed previous benchmarks, demonstrating a mean accuarcy of 0.880 (8.2% improvement over the original NT and 7% improvement over NT-v2). Furthermore, we construct a unique exomic ClinVar dataset to evaluate the model’s performance on pathogenicity and disease states. Our results highlight the potential of this model to improve our understanding of the human exome and its role in disease. The model and its applications have significant implications for genomic based diagnosis and personalized medicine including tailored therapeutic development. Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Genetics/Genomics/Medical genomics Transformer Architecture Scaling Experiments Multispecies Data Precision Medicine Disease Diagnosis Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6115078","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":430071647,"identity":"31fea4a3-1157-401b-85f1-c8154b302bfc","order_by":0,"name":"Shant Ayanian","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/ElEQVRIiWNgGAWjYLCCBBAhASIqoCI8RGs5cIZYLQwwLQfbiNDCPyP52YMHNXcY5Gc3P3v8cd42eYPjDYwP3rbh1iJxI83cIOHYMwaDO8fMDQ5uu2244cwBZsO5eLQw3Egwk0hgO8xgIAFkALUwbriRwCbNi0eL/I30bxIJ/w4zyM8AMg7OuW0P1ML+G58Wgxs5ZhKJbYeB1gEZBxtuJ4JsYcanxfDMmzKJxL7DPEC9ZRJnjt1OnnnmYLPknHO4tcgdT98m+ePbYTmgw7ZJVNTctu073nzww5syPN4XSABTiIhQOMDYgEc9EPAfQBOQJ6BhFIyCUTAKRh4AADsWXFjhBjsjAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0001-9319-9001","institution":"Mayo Clinic","correspondingAuthor":true,"prefix":"","firstName":"Shant","middleName":"","lastName":"Ayanian","suffix":""},{"id":430071649,"identity":"f63ef21b-7b8f-47b2-a6e9-3d6fbdf61173","order_by":1,"name":"Panos Korfiatis","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Panos","middleName":"","lastName":"Korfiatis","suffix":""},{"id":430071650,"identity":"3043e09a-b55e-475c-a007-f49e9106bde9","order_by":2,"name":"Collin Osborne","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Collin","middleName":"","lastName":"Osborne","suffix":""},{"id":430071651,"identity":"3d7ce11a-6a36-49de-87de-271cf54bab3e","order_by":3,"name":"Carl Molnar","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Carl","middleName":"","lastName":"Molnar","suffix":""},{"id":430071652,"identity":"7c03ea6a-c0d7-4ad0-8ff8-f23a31ae1d34","order_by":4,"name":"Marc Blasi","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Marc","middleName":"","lastName":"Blasi","suffix":""},{"id":430071653,"identity":"97275df1-8dba-40d8-b981-f9cc7c3afff6","order_by":5,"name":"Xoab Perez","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Xoab","middleName":"","lastName":"Perez","suffix":""},{"id":430071654,"identity":"9b2012ca-aeec-450f-a8a6-b13502fa6d6f","order_by":6,"name":"Clark XU","email":"","orcid":"https://orcid.org/0000-0002-2384-4244","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Clark","middleName":"","lastName":"XU","suffix":""},{"id":430071655,"identity":"d023978f-4a75-4501-bef1-114f25b069cb","order_by":7,"name":"Pravat Das","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Pravat","middleName":"","lastName":"Das","suffix":""},{"id":430071656,"identity":"3cca8001-39c4-420d-b1a4-e574fec1e952","order_by":8,"name":"Matthew Callstrom","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Callstrom","suffix":""},{"id":430071657,"identity":"71488ddb-584e-4375-87ce-276d32dc6922","order_by":9,"name":"Vijay Shah","email":"","orcid":"https://orcid.org/0000-0001-7620-573X","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Vijay","middleName":"","lastName":"Shah","suffix":""},{"id":430071658,"identity":"661b0095-8210-4aa7-830c-b48ab270b534","order_by":10,"name":"Alexander Ryu","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Alexander","middleName":"","lastName":"Ryu","suffix":""},{"id":430071659,"identity":"be9510f4-2d6c-43cd-9a9d-1ec4297686f4","order_by":11,"name":"Konstantinos Lazaridis","email":"","orcid":"https://orcid.org/0000-0002-0437-681X","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Konstantinos","middleName":"","lastName":"Lazaridis","suffix":""},{"id":430071660,"identity":"1a65a56a-486f-4f68-9995-711ac4027eb7","order_by":12,"name":"Ganesh Venkatesh","email":"","orcid":"","institution":"Cerebras Systems","correspondingAuthor":false,"prefix":"","firstName":"Ganesh","middleName":"","lastName":"Venkatesh","suffix":""},{"id":430071661,"identity":"ce22faf2-c02a-4a0d-87cf-8cc335549338","order_by":13,"name":"Vinay Pondenkandath","email":"","orcid":"","institution":"Cerebras Systems","correspondingAuthor":false,"prefix":"","firstName":"Vinay","middleName":"","lastName":"Pondenkandath","suffix":""},{"id":430071662,"identity":"a46b53b3-3817-4dfd-844a-8e5f0c9ea96d","order_by":14,"name":"Natalia Vassilieva","email":"","orcid":"https://orcid.org/0009-0003-1038-1572","institution":"Cerebras Systems","correspondingAuthor":false,"prefix":"","firstName":"Natalia","middleName":"","lastName":"Vassilieva","suffix":""},{"id":430071663,"identity":"ce29798c-3638-4583-a782-f5c25ac5d8c1","order_by":15,"name":"Bhargav Kanakiya","email":"","orcid":"","institution":"Cerebras Systems","correspondingAuthor":false,"prefix":"","firstName":"Bhargav","middleName":"","lastName":"Kanakiya","suffix":""},{"id":430071664,"identity":"1d8fc2e3-d14a-40e4-8482-76d86cfc2f0d","order_by":16,"name":"May Levin","email":"","orcid":"","institution":"Cerebras Systems","correspondingAuthor":false,"prefix":"","firstName":"May","middleName":"","lastName":"Levin","suffix":""},{"id":430071665,"identity":"fb0974d5-f7ee-46e5-9d6a-6a851d924009","order_by":17,"name":"Matthew Redlon","email":"","orcid":"https://orcid.org/0009-0004-3445-2053","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Redlon","suffix":""},{"id":430071648,"identity":"83bac840-db70-44e0-a5af-91d60d9b0a6e","order_by":18,"name":"Elena Myasoedova","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Elena","middleName":"","lastName":"Myasoedova","suffix":""}],"badges":[],"createdAt":"2025-02-26 17:01:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6115078/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6115078/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":79794410,"identity":"fe6995b7-4551-4c1b-8d54-7257fde2ac61","added_by":"auto","created_at":"2025-04-02 21:25:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1071377,"visible":true,"origin":"","legend":"Article File","description":"","filename":"STRANDPaper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6115078/v1_covered_611dd0a3-3d2a-408b-8864-40e6809b139b.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Introducing STRAND: A Foundational Sequence Transformer for Range Adaptive Nucleotide Decoding","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Transformer Architecture, Scaling Experiments, Multispecies Data, Precision Medicine, Disease Diagnosis","lastPublishedDoi":"10.21203/rs.3.rs-6115078/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6115078/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The advent of high-throughput sequencing has led to an exponential increase in\r\ngenomic data, highlighting the need for efficient and accurate models to analyze\r\nand interpret this information. Here, we introduce a novel, exomic foundational\r\nmodel that leverages a combination of human reference genome and multispecies\r\ndata to improve variant detection and interpretation. Our model utilizes a short-\r\nrange transformer architecture and is trained on a large dataset of human exomic\r\nsequences derived from the Tapestry study. Through a series of ablation studies\r\nand scaling experiments, we demonstrate the effectiveness of our model in pre-\r\ndicting next token accuracy and identifying clinically pathogenic variants. We\r\nalso show that our model outperforms existing models in a range of downstream\r\ntasks, including variant effect prediction and disease state identification. In fact,\r\nour largest STRAND variant (1B parameters) surpassed previous benchmarks,\r\ndemonstrating a mean accuarcy of 0.880 (8.2% improvement over the original\r\nNT and 7% improvement over NT-v2). Furthermore, we construct a unique\r\nexomic ClinVar dataset to evaluate the model’s performance on pathogenicity\r\n and disease states. Our results highlight the potential of this model to improve\r\nour understanding of the human exome and its role in disease. The model and\r\nits applications have significant implications for genomic based diagnosis and\r\npersonalized medicine including tailored therapeutic development.","manuscriptTitle":"Introducing STRAND: A Foundational Sequence Transformer for Range Adaptive Nucleotide Decoding","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-26 11:29:45","doi":"10.21203/rs.3.rs-6115078/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8eb82ef6-0f82-48f9-bc7f-06d3c7437519","owner":[],"postedDate":"March 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":45805133,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":45805134,"name":"Biological sciences/Genetics/Genomics/Medical genomics"}],"tags":[],"updatedAt":"2025-04-02T21:17:41+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-26 11:29:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6115078","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6115078","identity":"rs-6115078","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}