Complete Simulation of timsTOF PASEF Raw Datasets with Timsim Enables Precise Evaluation of False Discovery and Phosphosite Localization Error Rates

Complete Simulation of timsTOF PASEF Raw Datasets with Timsim Enables Precise Evaluation of False Discovery and Phosphosite Localization Error Rates | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Complete Simulation of timsTOF PASEF Raw Datasets with Timsim Enables Precise Evaluation of False Discovery and Phosphosite Localization Error Rates Stefan Tenzer, David Teschner, Zixuan Xiao, Tim Maier, David Gomez-Zepeda, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9032301/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Accurate control of false discovery rates (FDR) and false localization rates (FLR) is central to quantitative proteomics and phosphoproteomics, yet rigorous validation is limited by the absence of high-complexity ground truth data. Here we introduce timsim, a simulation framework using machine-learning and first principle-driven prediction of peptide properties to generate native Bruker-format timsTOF dda-PASEF and dia-PASEF acquisition data with complete ground-truth annotation. Using timsim benchmarks, we show that several dia-PASEF workflows control FDR near the nominal 1% threshold at stripped-sequence level but exhibit inflated true FDR (3–5%) when modified peptidoforms are considered, driven by systematic misassignment of common modifications. In dda-PASEF analyses, match-between-runs produced peak-matching errors of up to 30% under high-density conditions. Simulated phosphoproteomics datasets enabled calibration of site localization scores, identifying a 0.65 site-probability cutoff as an optimal tradeoff between sensitivity and false localization. Timsim provides a scalable resource for rigorous benchmarking and development of proteomics software. Biological sciences/Computational biology and bioinformatics/Proteome informatics Biological sciences/Computational biology and bioinformatics/Software Biological sciences/Computational biology and bioinformatics/Computational models Biological sciences/Biological techniques/Proteomic analysis Biological sciences/Biological techniques/Mass spectrometry Full Text Additional Declarations There is NO Competing Interest. Supplementary Files si3softwaresetup.pdf Supporting Information Software Installation and Usage Guide si1mm.pdf Supporting Information Materials and Methods si2figures.pdf Supporting Information Supplementary Figures Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9032301","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":600755434,"identity":"c13129ac-2434-4211-89ae-d3641a7833f8","order_by":0,"name":"Stefan Tenzer","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1ElEQVRIie2OMQuCQBSAX0R3ywNXh6K/cOIa+VcUoUmo/+DQYns/wy2ChoNbRdfEhiYnB1vCwaHTlqA4cWu4b7gHj/fxHYBG87+sAKbdZN3jqm/xPTbjFfG5G1AcehD145J5JwqzuNndHKA+V1cwda1jWXjnEEgesXIKWKozaAbMR154sTCeV2SCgNyolWXFRMtTqchKywSCua0HKmiFwHmvFLJiyorSAEwCexJx3+6VOROMYKn+GN0ndt3w9SLOOMmrVjgG9e/qzBdk5L1Go9FofvACeP1EdyChWBUAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-3034-0017","institution":"University Medical Center of the Johannes-Gutenberg-University Mainz","correspondingAuthor":true,"prefix":"","firstName":"Stefan","middleName":"","lastName":"Tenzer","suffix":""},{"id":600755435,"identity":"8226899b-00dc-4a64-9c50-b634e41b2ee9","order_by":1,"name":"David Teschner","email":"","orcid":"","institution":"Johannes-Gutenberg University Mainz","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"","lastName":"Teschner","suffix":""},{"id":600755436,"identity":"834a1a81-db0a-446b-a0db-1ac4372b504b","order_by":2,"name":"Zixuan Xiao","email":"","orcid":"","institution":"Technical University of Munich","correspondingAuthor":false,"prefix":"","firstName":"Zixuan","middleName":"","lastName":"Xiao","suffix":""},{"id":600755437,"identity":"0b960a52-ca93-45e6-a14c-e2a376d705db","order_by":3,"name":"Tim Maier","email":"","orcid":"","institution":"Johannes Gutenberg University Mainz","correspondingAuthor":false,"prefix":"","firstName":"Tim","middleName":"","lastName":"Maier","suffix":""},{"id":600755438,"identity":"2e58f7fc-0bca-4381-8daf-42cf58582a6b","order_by":4,"name":"David Gomez-Zepeda","email":"","orcid":"","institution":"Helmholtz Institute for Translational Oncology Mainz (HI-TRON Mainz) - A Helmholtz Institute of the DKFZ, Mainz, Germany","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"","lastName":"Gomez-Zepeda","suffix":""},{"id":600755439,"identity":"7b23d05a-fa97-4228-a64c-ccc45d260f68","order_by":5,"name":"Mateusz Łącki","email":"","orcid":"","institution":"University Medical Center of the Johannes Gutenberg University Mainz","correspondingAuthor":false,"prefix":"","firstName":"Mateusz","middleName":"","lastName":"Łącki","suffix":""},{"id":600755440,"identity":"63d0f204-9f1a-4f3a-90c1-61b28abb42ba","order_by":6,"name":"Michal Startek","email":"","orcid":"","institution":"University of Warsaw","correspondingAuthor":false,"prefix":"","firstName":"Michal","middleName":"","lastName":"Startek","suffix":""},{"id":600755441,"identity":"f302a99b-9d29-4adf-a9a2-13b627ab2d3f","order_by":7,"name":"Ute Distler","email":"","orcid":"https://orcid.org/0000-0002-8031-6384","institution":"University Medical Center of the Johannes Gutenberg University Mainz","correspondingAuthor":false,"prefix":"","firstName":"Ute","middleName":"","lastName":"Distler","suffix":""},{"id":600755442,"identity":"1e2b5bea-d9e1-4902-b4e5-3dca42f44421","order_by":8,"name":"Tanja Ziesmann","email":"","orcid":"https://orcid.org/0009-0009-0399-258X","institution":"University Medical Center Mainz","correspondingAuthor":false,"prefix":"","firstName":"Tanja","middleName":"","lastName":"Ziesmann","suffix":""},{"id":600755443,"identity":"eea1838f-bdca-4880-98a2-3c0b041be3fb","order_by":9,"name":"Mathias Wilhelm","email":"","orcid":"https://orcid.org/0000-0002-9224-3258","institution":"Technical University of Munich","correspondingAuthor":false,"prefix":"","firstName":"Mathias","middleName":"","lastName":"Wilhelm","suffix":""},{"id":600755444,"identity":"228e019c-df55-4f58-b382-25d352b3f61f","order_by":10,"name":"Andreas Hildebrand","email":"","orcid":"https://orcid.org/0000-0003-2180-6516","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Andreas","middleName":"","lastName":"Hildebrand","suffix":""}],"badges":[],"createdAt":"2026-03-04 16:15:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9032301/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9032301/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105904592,"identity":"1a08266b-de4b-4e1e-8fe4-37016bdc34de","added_by":"auto","created_at":"2026-04-01 10:09:46","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1624841,"visible":true,"origin":"","legend":"Article File","description":"","filename":"timsimmainmanuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9032301/v1_covered_6dadf069-c61b-4378-bb20-25b661fd6aa1.pdf"},{"id":104457917,"identity":"4a99b9d7-cb5c-4bcc-90d6-3a80b35fb283","added_by":"auto","created_at":"2026-03-12 02:55:11","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":405530,"visible":true,"origin":"","legend":"Supporting Information Software Installation and Usage Guide","description":"","filename":"si3softwaresetup.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9032301/v1/6e794dcccd0c69a1bc58ce32.pdf"},{"id":104457937,"identity":"0a744267-65ac-4699-b53b-34451bd2143e","added_by":"auto","created_at":"2026-03-12 02:55:23","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":823103,"visible":true,"origin":"","legend":"Supporting Information Materials and Methods","description":"","filename":"si1mm.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9032301/v1/df40ee33a170142f1737250a.pdf"},{"id":104457934,"identity":"ebfcf26f-3828-4ff1-a82d-153ab2ec9e08","added_by":"auto","created_at":"2026-03-12 02:55:17","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":8782331,"visible":true,"origin":"","legend":"Supporting Information Supplementary Figures","description":"","filename":"si2figures.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9032301/v1/f185f2d02549456642524df7.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Complete Simulation of timsTOF PASEF Raw Datasets with Timsim Enables Precise Evaluation of False Discovery and Phosphosite Localization Error Rates","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-9032301/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9032301/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Accurate control of false discovery rates (FDR) and false localization rates (FLR) is central to quantitative proteomics and phosphoproteomics, yet rigorous validation is limited by the absence of high-complexity ground truth data. Here we introduce timsim, a simulation framework using machine-learning and first principle-driven prediction of peptide properties to generate native Bruker-format timsTOF dda-PASEF and dia-PASEF acquisition data with complete ground-truth annotation. Using timsim benchmarks, we show that several dia-PASEF workflows control FDR near the nominal 1% threshold at stripped-sequence level but exhibit inflated true FDR (3–5%) when modified peptidoforms are considered, driven by systematic misassignment of common modifications. In dda-PASEF analyses, match-between-runs produced peak-matching errors of up to 30% under high-density conditions. Simulated phosphoproteomics datasets enabled calibration of site localization scores, identifying a 0.65 site-probability cutoff as an optimal tradeoff between sensitivity and false localization. Timsim provides a scalable resource for rigorous benchmarking and development of proteomics software.","manuscriptTitle":"Complete Simulation of timsTOF PASEF Raw Datasets with Timsim Enables Precise Evaluation of False Discovery and Phosphosite Localization Error Rates","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-12 02:54:52","doi":"10.21203/rs.3.rs-9032301/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"25da9240-d8db-436a-a464-acba30172f96","owner":[],"postedDate":"March 12th, 2026","published":true,"recentEditorialEvents":[{"type":"transferred","content":"Nature Communications","date":"2026-05-11T13:59:06+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":63931612,"name":"Biological sciences/Computational biology and bioinformatics/Proteome informatics"},{"id":63931613,"name":"Biological sciences/Computational biology and bioinformatics/Software"},{"id":63931614,"name":"Biological sciences/Computational biology and bioinformatics/Computational models"},{"id":63931615,"name":"Biological sciences/Biological techniques/Proteomic analysis"},{"id":63931616,"name":"Biological sciences/Biological techniques/Mass spectrometry"}],"tags":[],"updatedAt":"2026-05-11T13:59:06+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-12 02:54:52","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9032301","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9032301","identity":"rs-9032301","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}