A Large-Scale Foundation Model for RNA Enables Diverse Function and Structure Prediction

A Large-Scale Foundation Model for RNA Enables Diverse Function and Structure Prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A Large-Scale Foundation Model for RNA Enables Diverse Function and Structure Prediction Eric Xing, Shuxian Zou, Tianhua Tao, Sazan Mahbub, Caleb Ellington, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6445344/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Accurately predicting RNA structures and functions from nucleotide sequences, or conversely, designing sequences to meet structural and functional requirements, remains a fundamental challenge in RNA biology, largely due to limited annotated data and the poor efficiency of \textit{ab initio} modeling approaches. Here, we introduce AIDO.RNA, a large-scale RNA foundation model that leverages self-supervised pre-training to learn general and effective RNA representations, which can be transferred to tackle a wide range of RNA prediction and design tasks. AIDO.RNA is a 1.6-billion-parameter transformer-based language model, pre-trained on 42 million non-coding RNA (ncRNA) sequences at single-nucleotide resolution. It can be adapted to achieve state-of-the-art performance on 26 out of 28 diverse tasks, including RNA structure and function prediction, mRNA expression modeling, multi-modal RNA isoform expression prediction, and RNA inverse folding, demonstrating its effectiveness and versatility across the board. We find that beyond excelling in ncRNA-related tasks that directly reside in the pre-training data space, AIDO.RNA can be efficiently adapted to new domains with continued domain-specific pre-training to generalize toward untranslated regions and coding regions of mRNA, suggesting a promising pathway to continue to level up biological foundation models in general. We make AIDO.RNA open source and release the utility of the model in AIDO.ModelGenerator, a Python package enabling easy reproduction, application, and extension of our results. Biological sciences/Computational biology and bioinformatics/Computational models Biological sciences/Molecular biology/Transcriptomics Biological sciences/Computational biology and bioinformatics/Machine learning Full Text Additional Declarations There is NO Competing Interest. Supplementary Files SupplementaryInfomationAIDORNA.pdf Supplementary Information Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6445344","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":452774527,"identity":"338d06c3-6816-4060-b176-51c18ff900be","order_by":0,"name":"Eric Xing","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2ElEQVRIiWNgGAWjYFACxgYgIcHAwA6iDSxI0cJzAKRFghTbJBIYIHoJAX7pw40fGNss8uQjn1/d8KNAgoG/vTsBrxbJvsRmCYYzEsWGt3PKbvYAHSZx5uwGvFoMzjC2MTBUSCRunJ2TdoMHqMVAIhe/FnuwFgOglpln0m7+IUaLAQ/UlvkS7MduE2WLxBnGZomEMxKJG3hy2G7LGEjwEPQLfw/7ww8f2+oS57cff3bzzR8bOf72XvxawCAB5MIDPAYgNg9h5TAg38D+gHjVo2AUjIJRMKIAAF1/QvvSBvqXAAAAAElFTkSuQmCC","orcid":"","institution":"GenBio AI; Mohamed bin Zayed University of Artificial Intelligence; Carnegie Mellon University","correspondingAuthor":true,"prefix":"","firstName":"Eric","middleName":"","lastName":"Xing","suffix":""},{"id":452774528,"identity":"ee96fa08-d661-4b3f-8c28-7ae289b1cf8a","order_by":1,"name":"Shuxian Zou","email":"","orcid":"","institution":"GenBio AI; Mohamed bin Zayed University of Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Shuxian","middleName":"","lastName":"Zou","suffix":""},{"id":452774529,"identity":"89f64e83-1630-4f7d-8138-4c93b501639c","order_by":2,"name":"Tianhua Tao","email":"","orcid":"","institution":"GenBio AI; University of Washington","correspondingAuthor":false,"prefix":"","firstName":"Tianhua","middleName":"","lastName":"Tao","suffix":""},{"id":452774530,"identity":"83a625d4-0e6b-49ec-931c-cfd7e8363503","order_by":3,"name":"Sazan Mahbub","email":"","orcid":"","institution":"GenBio AI; Carnegie Mellon University","correspondingAuthor":false,"prefix":"","firstName":"Sazan","middleName":"","lastName":"Mahbub","suffix":""},{"id":452774531,"identity":"034200f9-cc08-4c20-9659-c065f69b35bd","order_by":4,"name":"Caleb Ellington","email":"","orcid":"","institution":"GenBio AI","correspondingAuthor":false,"prefix":"","firstName":"Caleb","middleName":"","lastName":"Ellington","suffix":""},{"id":452774532,"identity":"38849fa8-6f36-448e-be8d-3a5f5089ef5d","order_by":5,"name":"Robin Algayres","email":"","orcid":"","institution":"GenBio AI","correspondingAuthor":false,"prefix":"","firstName":"Robin","middleName":"","lastName":"Algayres","suffix":""},{"id":452774533,"identity":"d7f9c799-9313-4108-ae8b-bd9f45f519b0","order_by":6,"name":"Dian Li","email":"","orcid":"","institution":"GenBio AI","correspondingAuthor":false,"prefix":"","firstName":"Dian","middleName":"","lastName":"Li","suffix":""},{"id":452774534,"identity":"323b47a3-c74a-4f5f-87cc-b15f01457c58","order_by":7,"name":"Yonghao Zhuang","email":"","orcid":"","institution":"GenBio AI; Carnegie Mellon University","correspondingAuthor":false,"prefix":"","firstName":"Yonghao","middleName":"","lastName":"Zhuang","suffix":""},{"id":452774535,"identity":"4f5cbfd8-7da3-4d38-a1f1-020ed6bcb59b","order_by":8,"name":"Hongyi Wang","email":"","orcid":"","institution":"GenBio AI","correspondingAuthor":false,"prefix":"","firstName":"Hongyi","middleName":"","lastName":"Wang","suffix":""},{"id":452774536,"identity":"2f3a19e4-f792-48b8-8547-dbc9bf553c24","order_by":9,"name":"Le Song","email":"","orcid":"","institution":"GenBio AI; Mohamed bin Zayed University of Artificial Intelligence","correspondingAuthor":false,"prefix":"","firstName":"Le","middleName":"","lastName":"Song","suffix":""}],"badges":[],"createdAt":"2025-04-14 11:06:56","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6445344/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6445344/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":86223211,"identity":"0d0e9f1c-ba26-46e0-92d9-77833c534214","added_by":"auto","created_at":"2025-07-08 07:29:37","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3163340,"visible":true,"origin":"","legend":"Article File","description":"","filename":"MainContentAIDORNA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6445344/v1_covered_6d095c3b-0d7c-4974-ae73-379cdae51aea.pdf"},{"id":82177136,"identity":"e040a827-9d16-4d46-b262-8415b9c4fa2c","added_by":"auto","created_at":"2025-05-07 11:18:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":104980,"visible":true,"origin":"","legend":"Supplementary Information","description":"","filename":"SupplementaryInfomationAIDORNA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6445344/v1/76a536efb34f000bb5ce7233.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"A Large-Scale Foundation Model for RNA Enables Diverse Function and Structure Prediction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6445344/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6445344/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Accurately predicting RNA structures and functions from nucleotide sequences, or conversely, designing sequences to meet structural and functional requirements, remains a fundamental challenge in RNA biology, largely due to limited annotated data and the poor efficiency of \\textit{ab initio} modeling approaches. Here, we introduce AIDO.RNA, a large-scale RNA foundation model that leverages self-supervised pre-training to learn general and effective RNA representations, which can be transferred to tackle a wide range of RNA prediction and design tasks. AIDO.RNA is a 1.6-billion-parameter transformer-based language model, pre-trained on 42 million non-coding RNA (ncRNA) sequences at single-nucleotide resolution. It can be adapted to achieve state-of-the-art performance on 26 out of 28 diverse tasks, including RNA structure and function prediction, mRNA expression modeling, multi-modal RNA isoform expression prediction, and RNA inverse folding, demonstrating its effectiveness and versatility across the board. We find that beyond excelling in ncRNA-related tasks that directly reside in the pre-training data space, AIDO.RNA can be efficiently adapted to new domains with continued domain-specific pre-training to generalize toward untranslated regions and coding regions of mRNA, suggesting a promising pathway to continue to level up biological foundation models in general. We make AIDO.RNA open source and release the utility of the model in AIDO.ModelGenerator, a Python package enabling easy reproduction, application, and extension of our results.","manuscriptTitle":"A Large-Scale Foundation Model for RNA Enables Diverse Function and Structure Prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-07 11:18:48","doi":"10.21203/rs.3.rs-6445344/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-methods","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"nmeth","sideBox":"Learn more about [Nature Methods](http://www.nature.com/nmeth)","snPcode":"","submissionUrl":"","title":"Nature Methods","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Research","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"0b697b76-d607-4873-bba7-26d790cde361","owner":[],"postedDate":"May 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":48152242,"name":"Biological sciences/Computational biology and bioinformatics/Computational models"},{"id":48152243,"name":"Biological sciences/Molecular biology/Transcriptomics"},{"id":48152244,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"}],"tags":[],"updatedAt":"2026-03-03T04:10:18+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-07 11:18:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6445344","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6445344","identity":"rs-6445344","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}