The replicability of real-world evidence for the surgical treatment of proximal humeral fractures in older patients evaluated using sceptical p-value

The replicability of real-world evidence for the surgical treatment of proximal humeral fractures in older patients evaluated using sceptical p-value | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article The replicability of real-world evidence for the surgical treatment of proximal humeral fractures in older patients evaluated using sceptical p-value Jeanette Köppe, Karen Fischhuber, Josef Stolberg-Stolberg, Ursula Marschall, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6679075/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 16 You are reading this latest preprint version Abstract Background: The replicability of study results plays a central role, particularly in the context of real-world evidence (RWE). However, the evaluation of the agreement between two study results is not straightforward and different metrics are available. As an example for RWE, the surgical treatment of proximal humeral fractures in older patients using locked plate fixation (LPF) or reverse total shoulder arthroplasty (RTSA) was analyzed by two different studies based on the data of the largest German health insurance company [1, 2], but the generalizability of the results should be further addressed. Methods: Independent data of a different German insurance company were used to emulate two existing studies for the surgical treatment of proximal humeral fracture in older patients using RTSA vs LPF. Study protocol was a priori registered. The sceptical p-value was used to judge the agreement between known original effect estimate (OES) and the observed emulated treatment effect estimate (EES) and compared to other existing binary agreement metrics. Results: Overall, 7 (64%) of the 11 evaluated endpoints were successfully replicated. The Pearson correlation coefficient (rp = 0.97) and the intra-class correlation coefficient (ICC3 = 0.94) indicated a good, positive correlation between original and emulation study. OES was successfully replicated for all short-term endpoints and surgical complications during follow-up (all pscep < 0.025). Conclusion: All treatment effects of the original study for short-term endpoints and for surgical complications during follow-up were successfully replicated using an independent data base. The sceptical p value was a useful and well-interpretable metric to evaluate the agreement between original and emulation study. Trial registration: A priori registered at ClinicalTrials.gov (Identifier: NCT06537024). Level of Evidence: Level III, retrospective replication study. Health sciences/Health care/Fracture repair Health sciences/Medical research/Epidemiology Health sciences/Medical research/Outcomes research real world evidence replication sceptical p-value health service research proximal humeral fracture reverse total shoulder arthroplasty Full Text Additional Declarations Competing interest reported. JK, JSS and JCK report research funding from the German Society for Trauma Surgery sponsored by Stryker, out-side the submitted work. All the remaining authors declare no conflict of interest. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 23 Apr, 2026 Reviews received at journal 17 Apr, 2026 Reviewers agreed at journal 12 Apr, 2026 Reviewers agreed at journal 28 Mar, 2026 Reviews received at journal 22 Mar, 2026 Reviewers agreed at journal 22 Mar, 2026 Reviews received at journal 20 Mar, 2026 Reviewers agreed at journal 20 Mar, 2026 Reviews received at journal 13 Nov, 2025 Reviewers agreed at journal 24 Oct, 2025 Reviewers agreed at journal 23 Oct, 2025 Reviewers invited by journal 23 Oct, 2025 Editor assigned by journal 21 Aug, 2025 Editor invited by journal 21 May, 2025 Submission checks completed at journal 21 May, 2025 First submitted to journal 16 May, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6679075","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":536985017,"identity":"c0a4414d-b24b-497e-be05-9feafba6fb03","order_by":0,"name":"Jeanette Köppe","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABOklEQVRIiWNgGAWjYBACxgYYiwdCycEYYCCBU0uCAVyLMYwhgUsLBCBpSWwgpIW5vfnZg48//gDdc/jg48IdNukbzpxOk/i4o66OX/oA440PWBzWc8zccEaCgTEDb1uy8cwzabkbzvZuk5x55rCEZF8Cs+UMLFpmJJhJ8yQYJDbw85hJ87Ydzt1wnncbkHFAwuAMA5s0DxYt859/k/6TYFAP1GL+m7ftf7oBSMvftjoJe5CWP9hsARoO9H4CA2+PGTPQ8AQDoMOkGduYJQx4gFqweJ+xJ6dMsifN2LCN51iy9MwzyYYzz5zdbNnbdlhyxhnGZsseTC2G7ce3SfywkZPn50k++Llwh50835ncjTd+ttXx8/cwH7zxA4uWBiiDDRTiiLQAcQIqFwrkkTnoWkbBKBgFo2AUgAEARohkLBzTNrQAAAAASUVORK5CYII=","orcid":"","institution":"University of Münster","correspondingAuthor":true,"prefix":"","firstName":"Jeanette","middleName":"","lastName":"Köppe","suffix":""},{"id":536985018,"identity":"8c0a2adc-c1b8-4fcc-87a0-a9eb3f21476b","order_by":1,"name":"Karen Fischhuber","email":"","orcid":"","institution":"University of Münster","correspondingAuthor":false,"prefix":"","firstName":"Karen","middleName":"","lastName":"Fischhuber","suffix":""},{"id":536985019,"identity":"0cc5f342-3cf9-4079-b145-eff6e21fd7f8","order_by":2,"name":"Josef Stolberg-Stolberg","email":"","orcid":"","institution":"University Hospital Münster","correspondingAuthor":false,"prefix":"","firstName":"Josef","middleName":"","lastName":"Stolberg-Stolberg","suffix":""},{"id":536985020,"identity":"a74d900f-6289-4b36-accc-05ec79b07429","order_by":3,"name":"Ursula Marschall","email":"","orcid":"","institution":"BARMER Institute for Health System Research","correspondingAuthor":false,"prefix":"","firstName":"Ursula","middleName":"","lastName":"Marschall","suffix":""},{"id":536985021,"identity":"feafbdd9-b920-4edb-afef-ea19de15cad3","order_by":4,"name":"Michael J. Raschke","email":"","orcid":"","institution":"University Hospital Münster","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"J.","lastName":"Raschke","suffix":""},{"id":536985022,"identity":"e93d024f-1014-43fc-bfdb-24a649bcd2a7","order_by":5,"name":"Andreas Faldum","email":"","orcid":"","institution":"University of Münster","correspondingAuthor":false,"prefix":"","firstName":"Andreas","middleName":"","lastName":"Faldum","suffix":""},{"id":536985023,"identity":"d938d0b9-e668-42e2-b213-98981efd178e","order_by":6,"name":"J. Christoph Katthagen","email":"","orcid":"","institution":"University Hospital Münster","correspondingAuthor":false,"prefix":"","firstName":"J.","middleName":"Christoph","lastName":"Katthagen","suffix":""}],"badges":[],"createdAt":"2025-05-16 08:53:28","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6679075/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6679075/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":95078150,"identity":"ab903364-4bd4-4db6-b615-32a9ee91e379","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9395,"visible":true,"origin":"","legend":"","description":"","filename":"5c8c8c5d966c40af9fa9788b3cb6424a.json","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/d19d3d8d0cb27bd5d0dac943.json"},{"id":95078155,"identity":"f429947d-6579-4b99-81c6-06380fc0b4d0","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":148130,"visible":true,"origin":"","legend":"","description":"","filename":"20240807PSOverlap.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/cffc47dc57790f75c6152418.pdf"},{"id":95078157,"identity":"54729312-277d-4430-8bd3-088974fe5aa2","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":171551,"visible":true,"origin":"","legend":"","description":"","filename":"20241108ReploutcomescorrBAPbearb.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/a645e0e12ceb635caa6decb6.pdf"},{"id":95078151,"identity":"f62ef5e5-9335-43d4-ade5-b7e5b8f93f9d","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":17125,"visible":true,"origin":"","legend":"","description":"","filename":"20241108forestlongtermpublbearb.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/8ac87715a923321d2a467c5d.pdf"},{"id":95224164,"identity":"9af00b0b-2e94-4d2c-8332-a994ac9cfcfa","added_by":"auto","created_at":"2025-11-05 16:23:28","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":45920,"visible":true,"origin":"","legend":"","description":"","filename":"20241108forestshorttermpublbearb.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/872fbb9f7ea52533e01347d6.pdf"},{"id":95078156,"identity":"59677336-e9d2-4586-89d5-1a0a5636ff4e","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":319340,"visible":true,"origin":"","legend":"","description":"","filename":"20250516CoverLetterKoeppeetal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/38ae104cc7e4d95159e5ec87.pdf"},{"id":95078154,"identity":"bf41e30c-a12b-4d14-91d4-77860974796e","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"cls","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55857,"visible":true,"origin":"","legend":"","description":"","filename":"snjnl.cls","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/75b78d2770552eb917dbca6f.cls"},{"id":95224032,"identity":"9fdf420f-a908-413e-ad6b-46aab57f6711","added_by":"auto","created_at":"2025-11-05 16:23:13","extension":"bst","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64164,"visible":true,"origin":"","legend":"","description":"","filename":"snmathphysnum.bst","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/282928374689d01baeaafd3b.bst"},{"id":95078158,"identity":"7ef5d7f5-d791-474c-b917-c4a5cdab3862","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"pdf","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":803407,"visible":true,"origin":"","legend":"","description":"","filename":"upload.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/3d78745bcf2ceb0f46416529.pdf"},{"id":95078159,"identity":"886dcb57-721a-4a9a-b690-be02001aeca5","added_by":"auto","created_at":"2025-11-04 05:31:27","extension":"xml","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":170579,"visible":true,"origin":"","legend":"","description":"","filename":"5c8c8c5d966c40af9fa9788b3cb6424a1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1/bacf838126eabfa1c3aecae8.xml"},{"id":95230230,"identity":"2d66ecc8-68f3-4c66-9686-af924e552772","added_by":"auto","created_at":"2025-11-05 16:37:01","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":599303,"visible":true,"origin":"","legend":"","description":"","filename":"upload.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6679075/v1_covered_baccd25e-11c5-409a-9d22-c66329d080fe.pdf"}],"financialInterests":"Competing interest reported. JK, JSS and JCK report research funding from the German Society for Trauma Surgery sponsored by Stryker, out-side the submitted work. All the remaining authors declare no conflict of interest.","formattedTitle":"The replicability of real-world evidence for the surgical treatment of proximal humeral fractures in older patients evaluated using sceptical p-value","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"real world evidence, replication, sceptical p-value, health service research, proximal humeral fracture, reverse total shoulder arthroplasty","lastPublishedDoi":"10.21203/rs.3.rs-6679075/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6679075/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: The replicability of study results plays a central role, particularly in the context of real-world evidence (RWE). However, the evaluation of the agreement between two study results is not straightforward and different metrics are available. As an example for RWE, the surgical treatment of proximal humeral fractures in older patients using locked plate fixation (LPF) or reverse total shoulder arthroplasty (RTSA) was analyzed by two different studies based on the data of the largest German health insurance company [1, 2], but the generalizability of the results should be further addressed.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMethods: Independent data of a different German insurance company were used to emulate two existing studies for the surgical treatment of proximal humeral fracture in older patients using RTSA vs LPF. Study protocol was a priori registered. The sceptical p-value was used to judge the agreement between known original effect estimate (OES) and the observed emulated treatment effect estimate (EES) and compared to other existing binary agreement metrics.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eResults: Overall, 7 (64%) of the 11 evaluated endpoints were successfully replicated. The Pearson correlation coefficient (rp = 0.97) and the intra-class correlation coefficient (ICC3 = 0.94) indicated a good, positive correlation between original and emulation study. OES was successfully replicated for all short-term endpoints and surgical complications during follow-up (all pscep \u0026lt; 0.025).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConclusion: All treatment effects of the original study for short-term endpoints and for surgical complications during follow-up were successfully replicated using an independent data base. The sceptical p value was a useful and well-interpretable metric to evaluate the agreement between original and emulation study.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTrial registration: A priori registered at ClinicalTrials.gov (Identifier: NCT06537024).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLevel of Evidence: \u0026nbsp;Level III, retrospective replication study.\u003c/p\u003e","manuscriptTitle":"The replicability of real-world evidence for the surgical treatment of proximal humeral fractures in older patients evaluated using sceptical p-value","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-04 05:31:22","doi":"10.21203/rs.3.rs-6679075/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-04-23T08:26:37+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-17T07:25:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"132028996065161788618844963715338297108","date":"2026-04-12T15:43:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"222363166892075714482225337431675570562","date":"2026-03-28T23:40:33+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-22T11:45:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"197252738123637465923837408955576043920","date":"2026-03-22T11:17:33+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-21T02:35:37+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"176499192162922496855983181194090209180","date":"2026-03-21T01:56:14+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-13T15:48:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"26503879511245914091120548712413189611","date":"2025-10-24T15:32:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"284351447146330984031872945523905915768","date":"2025-10-23T15:56:41+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-10-23T10:29:57+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-21T10:16:09+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-05-21T11:03:40+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-05-21T07:38:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-05-16T08:47:40+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8466ec98-01c5-4f33-8338-ecfceaec91b0","owner":[],"postedDate":"November 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":57103801,"name":"Health sciences/Health care/Fracture repair"},{"id":57103802,"name":"Health sciences/Medical research/Epidemiology"},{"id":57103803,"name":"Health sciences/Medical research/Outcomes research"}],"tags":[],"updatedAt":"2026-05-15T08:48:59+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-04 05:31:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6679075","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6679075","identity":"rs-6679075","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}