Comprehensive benchmarking of RNA velocity methods across single-cell datasets

Comprehensive benchmarking of RNA velocity methods across single-cell datasets | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Comprehensive benchmarking of RNA velocity methods across single-cell datasets Jin Liu, Yida Wu, Chuihan Kong, Xu Liao, Zhixiang Lin, Xiaobo Sun This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8708834/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background : RNA velocity provides a powerful framework for inferring cellular dynamics from single-cell RNA sequencing data. The rapid proliferation of computational methods within this field has prompted a need for systematic evaluation. However, existing comparisons often suffer from limited scope or incomplete task design, leaving users without clear guidance. Consequently, there is a lack of a comprehensive and standardized benchmark that evaluates methods across diverse biological and technical scenarios using appropriate, context-specific metrics. Results : In this study, we present a comprehensive benchmark of 19 computational RNA velocity tools covering 30 distinct methods. We systematically evaluate 25 splicing dynamics--based methods across eight evaluation tasks, designating directional consistency, temporal precision, negative control robustness, and sequencing depth stability as core tasks, while assessing five multimodal-enhanced methods specifically on the multimodal integration task. These assessments utilize 30 datasets spanning 22 real-world and eight simulated scenarios. Our results reveal a clear trade-off between directional consistency and negative control robustness, distinct group-wise behaviors across temporal modeling strategies, and variability driven by sequencing depth and quantification choices. This study also identifies several methodological gaps, including the need for improved modeling of gene dependence, more accurate temporal inference strategies, and better-designed multimodal architectures. Conclusions : This benchmark establishes a unified framework for evaluating RNA velocity methods. Crucially, we provide task-aware guidance to facilitate method selection based on specific biological contexts and technical constraints, rather than relying on a single overall ranking. Benchmarking RNA velocity Single-cell RNA sequencing Splicing dynamics Multimodal Full Text Additional Declarations No competing interests reported. Supplementary Files Additionalfile1SupplementaryNotes.pdf Additionalfile2SupplementaryFigures.pdf Additionalfile3SupplementaryTables.xlsx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 31 Mar, 2026 Reviews received at journal 29 Mar, 2026 Reviews received at journal 09 Mar, 2026 Reviewers agreed at journal 19 Feb, 2026 Reviewers agreed at journal 09 Feb, 2026 Reviewers invited by journal 29 Jan, 2026 Editor assigned by journal 28 Jan, 2026 Submission checks completed at journal 28 Jan, 2026 First submitted to journal 27 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8708834","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":582664659,"identity":"4e12db78-9753-47e0-a487-adb10e602148","order_by":0,"name":"Jin Liu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA3klEQVRIiWNgGAWjYBACPmYGBoMEBgsGA/YGBmaGA0ChAwS0sEG0SDAY8BwgVguEAmqRSCBWCzuPQcHDHRLy5pKPn0kXnGGQ47uRwPi5AK/DeAwMEs9IGO6cnWYmPeMGg7HkjQRm6RkEtbRJMG64ncMmzfOBIXHDjQSgIBFa7DfcPAPWUk+0FqDhPEAtNxgSDAhrYSsAaUne2ZNmbM0D9NTMMw+bpfFp4ec/vM3wZ5uN7Xb2ww9v8xyzkec7nnzwMz4tIIsMkDgSQMzYgF8DAwPzA0IqRsEoGAWjYIQDAI1qQtDoFN5lAAAAAElFTkSuQmCC","orcid":"","institution":"Chinese University of Hong Kong, Shenzhen","correspondingAuthor":true,"prefix":"","firstName":"Jin","middleName":"","lastName":"Liu","suffix":""},{"id":582664662,"identity":"121f9c65-cfa1-47b4-b8f5-72b68d1515bd","order_by":1,"name":"Yida Wu","email":"","orcid":"","institution":"Chinese University of Hong Kong, Shenzhen","correspondingAuthor":false,"prefix":"","firstName":"Yida","middleName":"","lastName":"Wu","suffix":""},{"id":582664663,"identity":"e26746cd-6dbf-4aed-8a56-29dcb8c94f34","order_by":2,"name":"Chuihan Kong","email":"","orcid":"","institution":"Chinese University of Hong Kong, Shenzhen","correspondingAuthor":false,"prefix":"","firstName":"Chuihan","middleName":"","lastName":"Kong","suffix":""},{"id":582664664,"identity":"433eb508-a1db-49dc-9809-99c8a4c695d1","order_by":3,"name":"Xu Liao","email":"","orcid":"","institution":"Chinese University of Hong Kong, Shenzhen","correspondingAuthor":false,"prefix":"","firstName":"Xu","middleName":"","lastName":"Liao","suffix":""},{"id":582664665,"identity":"902202d9-05e8-4274-92b3-8ab31b416a68","order_by":4,"name":"Zhixiang Lin","email":"","orcid":"","institution":"Chinese University of Hong Kong","correspondingAuthor":false,"prefix":"","firstName":"Zhixiang","middleName":"","lastName":"Lin","suffix":""},{"id":582664667,"identity":"dd39d930-1106-4d4d-997e-b4da7bba4597","order_by":5,"name":"Xiaobo Sun","email":"","orcid":"","institution":"Emory University","correspondingAuthor":false,"prefix":"","firstName":"Xiaobo","middleName":"","lastName":"Sun","suffix":""}],"badges":[],"createdAt":"2026-01-27 10:16:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8708834/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8708834/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101753230,"identity":"26e0c6a4-479b-49c5-8292-95cc4e2f30bb","added_by":"auto","created_at":"2026-02-03 10:39:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":29848499,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8708834/v1_covered_9936467e-3925-4d4c-b1b6-bbd234c6258b.pdf"},{"id":101624750,"identity":"549cc053-beab-4fc4-963a-63c9bf3c6b06","added_by":"auto","created_at":"2026-02-02 03:36:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":179059,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1SupplementaryNotes.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8708834/v1/04ccae45c55003404453a676.pdf"},{"id":101624752,"identity":"7af4cb2a-6934-4b86-bb27-2f6967156e8b","added_by":"auto","created_at":"2026-02-02 03:36:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":151063630,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2SupplementaryFigures.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8708834/v1/03367a6944fab710fce1a034.pdf"},{"id":101624751,"identity":"49278511-090c-46c9-a7a0-593405cab9c0","added_by":"auto","created_at":"2026-02-02 03:36:07","extension":"xlsx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":106577,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile3SupplementaryTables.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-8708834/v1/6ee67f24d4a8ccc9309bc7e2.xlsx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Comprehensive benchmarking of RNA velocity methods across single-cell datasets","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"genome-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"gbio","sideBox":"Learn more about [Genome Biology](https://genomebiology.biomedcentral.com/)","snPcode":"13059","submissionUrl":"https://submission.springernature.com/new-submission/13059/3","title":"Genome Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Benchmarking, RNA velocity, Single-cell RNA sequencing, Splicing dynamics, Multimodal","lastPublishedDoi":"10.21203/rs.3.rs-8708834/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8708834/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e: RNA velocity provides a powerful framework for inferring cellular dynamics from single-cell RNA sequencing data. The rapid proliferation of computational methods within this field has prompted a need for systematic evaluation. However, existing comparisons often suffer from limited scope or incomplete task design, leaving users without clear guidance. Consequently, there is a lack of a comprehensive and standardized benchmark that evaluates methods across diverse biological and technical scenarios using appropriate, context-specific metrics.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e: In this study, we present a comprehensive benchmark of 19 computational RNA velocity tools covering 30 distinct methods. We systematically evaluate 25 splicing dynamics--based methods across eight evaluation tasks, designating directional consistency, temporal precision, negative control robustness, and sequencing depth stability as core tasks, while assessing five multimodal-enhanced methods specifically on the multimodal integration task. These assessments utilize 30 datasets spanning 22 real-world and eight simulated scenarios. Our results reveal a clear trade-off between directional consistency and negative control robustness, distinct group-wise behaviors across temporal modeling strategies, and variability driven by sequencing depth and quantification choices. This study also identifies several methodological gaps, including the need for improved modeling of gene dependence, more accurate temporal inference strategies, and better-designed multimodal architectures.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e: This benchmark establishes a unified framework for evaluating RNA velocity methods. Crucially, we provide task-aware guidance to facilitate method selection based on specific biological contexts and technical constraints, rather than relying on a single overall ranking.\u003c/p\u003e","manuscriptTitle":"Comprehensive benchmarking of RNA velocity methods across single-cell datasets","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-02 03:36:03","doi":"10.21203/rs.3.rs-8708834/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-31T19:42:54+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-29T20:29:26+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-09T20:31:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"34823117930808560856585391344024315297","date":"2026-02-19T23:15:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"196069779951000970371468340340569498681","date":"2026-02-09T21:42:51+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-29T20:05:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-28T21:29:23+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-28T14:20:40+00:00","index":"","fulltext":""},{"type":"submitted","content":"Genome Biology","date":"2026-01-27T09:23:49+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"genome-biology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"gbio","sideBox":"Learn more about [Genome Biology](https://genomebiology.biomedcentral.com/)","snPcode":"13059","submissionUrl":"https://submission.springernature.com/new-submission/13059/3","title":"Genome Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7986287b-35db-4ba9-9d51-9a95f3feed39","owner":[],"postedDate":"February 2nd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-17T09:39:19+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-02 03:36:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8708834","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8708834","identity":"rs-8708834","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}