Meta-learning on property matrices and LLM embeddings enables state-of-the-art prediction of gene knockdown by modified siRNAs

Meta-learning on property matrices and LLM embeddings enables state-of-the-art prediction of gene knockdown by modified siRNAs | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Meta-learning on property matrices and LLM embeddings enables state-of-the-art prediction of gene knockdown by modified siRNAs Ivan Golovkin, Denis Shatkovskii, Nikita Serov This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7336200/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Six small interference RNAs (siRNAs) have been approved as therapeutics making them promising nanosystems due to selective gene knockdown activity. siRNA design is complex due to various factors, where the chemical modifications are crucial to improve its half-life and stability. Machine learning (ML) enabled more efficient analysis of siRNA data, moreover predicting efficacy and off-target effects. This work proposes a novel pipeline for predicting gene knockdown activity of chemically modified siRNAs across the whole range of activities leveraging both descriptors of siRNA chemical composition-aware property matrices and large language model (LLM) embeddings for target gene encoding. Several general-purpose and domain-specific fine tuned LLMs were benchmarked on the target task, where the Mistral 7B general-purpose model outperformed even the models pre-trained on genomic data. Proposed model based on meta-learning mechanism successfully mitigates data imbalance towards moderate-to-high active constructs and achieves state-of-the-art (SOTA) quality with R2 = 0.84 and a RMSE = 12.27% on unseen data, where the probabilistic outputs of classifiers trained with F-scores up to 0.92 were used as additional descriptors. By filling the gap in the field of advanced chemical composition-aware siRNA design, our model aims to improve the efficacy of developed siRNA-based therapies. Full Text Additional Declarations No competing interests reported. Supplementary Files SI.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7336200","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":509259307,"identity":"4f3f9cfe-a61d-4872-be24-07c5d7829c89","order_by":0,"name":"Ivan Golovkin","email":"","orcid":"","institution":"ITMO University","correspondingAuthor":false,"prefix":"","firstName":"Ivan","middleName":"","lastName":"Golovkin","suffix":""},{"id":509259308,"identity":"50a851bc-ba01-44a7-bd41-caf3f06940d7","order_by":1,"name":"Denis Shatkovskii","email":"","orcid":"","institution":"ITMO University","correspondingAuthor":false,"prefix":"","firstName":"Denis","middleName":"","lastName":"Shatkovskii","suffix":""},{"id":509259310,"identity":"5a139496-8ed9-48f9-9368-eca449e8ac0a","order_by":2,"name":"Nikita Serov","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxUlEQVRIiWNgGAWjYNCCCiidQLyWMyCCGaiFaD2MbVAtRFkj39778DHvvDp5/mnnDzA8/LFNjkH6jAFeLQZnjhsb8247bDjjdjLIYbeNGfhyCGiRSGOTnLntQALD7WT2H0AtiQ08PPi1yM9/BtQypy5BHmoLYS0MN9jYJD42MCcYEK3F4Ewas8GHY4cNN95ONmBISLttzMbDVoDfYe3HGB8k1NTJy91OfMD4w+a2HD8P8wb8DsMAbCSqHwWjYBSMglGABQAAORo+qSgd/bAAAAAASUVORK5CYII=","orcid":"","institution":"ITMO University","correspondingAuthor":true,"prefix":"","firstName":"Nikita","middleName":"","lastName":"Serov","suffix":""}],"badges":[],"createdAt":"2025-08-10 00:38:05","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7336200/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7336200/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93394137,"identity":"d4c4dde5-58bc-46fb-a92d-47239f2290db","added_by":"auto","created_at":"2025-10-13 11:17:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1048165,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7336200/v1_covered_d7835aef-6e45-438c-853a-c068c3dcd8e2.pdf"},{"id":90843938,"identity":"4b2025ea-6ba2-441f-b348-a5178a10cfe6","added_by":"auto","created_at":"2025-09-08 21:54:26","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":286136,"visible":true,"origin":"","legend":"","description":"","filename":"SI.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7336200/v1/975baa18f0e8954ee7e63a12.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Meta-learning on property matrices and LLM embeddings enables state-of-the-art prediction of gene knockdown by modified siRNAs","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7336200/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7336200/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Six small interference RNAs (siRNAs) have been approved as therapeutics making them promising nanosystems due to selective gene knockdown activity. siRNA design is complex due to various factors, where the chemical modifications are crucial to improve its half-life and stability. Machine learning (ML) enabled more efficient analysis of siRNA data, moreover predicting efficacy and off-target effects. This work proposes a novel pipeline for predicting gene knockdown activity of chemically modified siRNAs across the whole range of activities leveraging both descriptors of siRNA chemical composition-aware property matrices and large language model (LLM) embeddings for target gene encoding. Several general-purpose and domain-specific fine tuned LLMs were benchmarked on the target task, where the Mistral 7B general-purpose model outperformed even the models pre-trained on genomic data. Proposed model based on meta-learning mechanism successfully mitigates data imbalance towards moderate-to-high active constructs and achieves state-of-the-art (SOTA) quality with R2 = 0.84 and a RMSE = 12.27% on unseen data, where the probabilistic outputs of classifiers trained with F-scores up to 0.92 were used as additional descriptors. By filling the gap in the field of advanced chemical composition-aware siRNA design, our model aims to improve the efficacy of developed siRNA-based therapies.","manuscriptTitle":"Meta-learning on property matrices and LLM embeddings enables state-of-the-art prediction of gene knockdown by modified siRNAs","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-08 21:54:21","doi":"10.21203/rs.3.rs-7336200/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ac0c2285-38a0-4976-8a68-8ba99fc51579","owner":[],"postedDate":"September 8th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-10-13T11:08:50+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-08 21:54:21","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7336200","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7336200","identity":"rs-7336200","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}