Neutrality Boundary Robustness for Meta-Analyses

Neutrality Boundary Robustness for Meta-Analyses | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article Neutrality Boundary Robustness for Meta-Analyses Thomas F Heston This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8457715/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: Meta-analyses conventionally report pooled effect sizes, confidence intervals (CIs), and p-values, addressing statistical significance and precision but not distance from therapeutic neutrality on a standardized robustness scale. The Neutrality Boundary Framework (NBF) provides a 0–1 robustness metric (nb) for individual studies, but its extension to meta-analytic evidence has not been formalized. Objective: To demonstrate empirical robustness synthesis across heterogeneous trial designs and propose a theoretical framework for pooled meta-analytic robustness (nbmeta) applicable when study-level effect estimates are available (and, for inverse-variance pooling, their variances/SEs). Methods: We formalize meta-analytic robustness using the NBF formula nbmeta = |T − T0|/(|T − T0| + S), where T is a pooled effect estimate, T0 is therapeutic neutrality (e.g., log(RR) = 0), and S is a cross-study scale parameter (e.g., between-study standard deviation ˆτ or median absolute deviation (MAD)). For empirical illustration, we analyzed a convenience sample of N = 161 clinical trials with pre-computed trial-level nb values, where nb ∈ [0, 1] is the NBF robustness index measuring distance from therapeutic neutrality. We summarized the distribution of nb overall and examined the correlation between nb and − log10(p). Results: Across N = 161 trials, median nb = 0.147 (IQR 0.038–0.390; range 0.000–0.902). Using empirically derived, provisionally recommended robustness bands (nb < 0.075 weak; 0.075 ≤ nb < 0.227 moderate; nb ≥ 0.227 strong), 35.4% of trials showed weak robustness, 24.8% moderate, and 39.8% strong. Binary 2×2 trials tended to show lower nb than continuous-outcome trials. Robustness nb was moderately correlated with − log10(p) (r = 0.35, p < 0.001, n = 137; 24 trials with p = 0 excluded), suggesting that robustness captures geometric distance from neutrality, a dimension distinct from statistical significance. Conclusions: This methods note establishes trial-level nb distribution as a simple robustness synthesis approach and formalizes nbmeta for future implementation when study-level effect estimates are available (and, for inverse-variance pooling, their variances/SEs). Even without computing nbmeta, the distribution of trial-level nb provides a cross-design summary of how far the evidence base lies from therapeutic neutrality. Integrating robustness assessment alongside p-values in routine evidence synthesis yields a more complete picture of evidential strength Biostatistics robustness neutrality boundary framework reproducibility crisis Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8457715","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":565994391,"identity":"4513d22f-d476-4bce-b68c-2e0fd9172e12","order_by":0,"name":"Thomas F Heston","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAqklEQVRIiWNgGAWjYDACCQglByZ5iNHBA9JygIHBmHQtiQ1Ea7GXbj78+UPFnfQNNxIYH7xtI8YWmWNpEgfOPMsFamE2nEuUFokcM4aDbYdzN9xOYJPmJU5L/ucPQC3pBrcT2H8TqSWHQQKoJQGohY2ZOC030swkzpw5bDjz/sNmyTnniNDCPiP58YeKisPyfGcOH/zwpowILUiAsYE09aNgFIyCUTAKcAMA+0U5crEgZPIAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-5655-2512","institution":"University of Washington","correspondingAuthor":true,"prefix":"","firstName":"Thomas","middleName":"F","lastName":"Heston","suffix":""}],"badges":[],"createdAt":"2025-12-26 22:06:51","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8457715/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8457715/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99796924,"identity":"1ba476b8-8f14-41e6-b4c3-62d0719beb06","added_by":"auto","created_at":"2026-01-08 13:43:58","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":191270,"visible":true,"origin":"","legend":"","description":"","filename":"metaanalysis7.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8457715/v1_covered_743013c0-651d-44f4-b4f9-a96c64f4317f.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eNeutrality Boundary Robustness for Meta-Analyses\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Washington","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"robustness, neutrality boundary framework, reproducibility crisis","lastPublishedDoi":"10.21203/rs.3.rs-8457715/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8457715/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: Meta-analyses conventionally report pooled effect sizes, confidence intervals (CIs), and p-values, addressing statistical significance and precision but not distance from therapeutic neutrality on a standardized robustness scale. The Neutrality Boundary Framework (NBF) provides a 0–1 robustness metric (nb) for individual studies, but its extension to meta-analytic evidence has not been formalized.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eObjective: To demonstrate empirical robustness synthesis across heterogeneous trial designs and propose a theoretical framework for pooled meta-analytic robustness (nbmeta) applicable when study-level effect estimates are available (and, for inverse-variance pooling, their variances/SEs).\u003c/p\u003e\n\u003cp\u003eMethods: We formalize meta-analytic robustness using the NBF formula nbmeta = |T − T0|/(|T − T0| + S), where T is a pooled effect estimate, T0 is therapeutic neutrality (e.g., log(RR) = 0), and S is a cross-study scale parameter (e.g., between-study standard deviation ˆτ or median absolute deviation (MAD)). For empirical illustration, we analyzed a convenience sample of N = 161 clinical trials with pre-computed trial-level nb values, where nb ∈ [0, 1] is the NBF robustness index measuring distance from therapeutic neutrality. We summarized the distribution of nb overall and examined the correlation between nb and − log10(p).\u003c/p\u003e\n\u003cp\u003eResults: Across N = 161 trials, median nb = 0.147 (IQR 0.038–0.390; range 0.000–0.902). Using empirically derived, provisionally recommended robustness bands (nb \u0026lt; 0.075 weak; 0.075 ≤ nb \u0026lt; 0.227 moderate; nb ≥ 0.227 strong), 35.4% of trials showed weak robustness, 24.8% moderate, and 39.8% strong. Binary 2×2 trials tended to show lower nb than continuous-outcome trials. Robustness nb was moderately correlated with − log10(p) (r = 0.35, p \u0026lt; 0.001, n = 137; 24 trials with p = 0 excluded), suggesting that robustness captures geometric distance from neutrality, a dimension distinct from statistical significance.\u003c/p\u003e\n\u003cp\u003eConclusions: This methods note establishes trial-level nb distribution as a simple robustness synthesis approach and formalizes nbmeta for future implementation when study-level effect estimates are available (and, for inverse-variance pooling, their variances/SEs). Even without computing nbmeta, the distribution of trial-level nb provides a cross-design summary of how far the evidence base lies from therapeutic neutrality. Integrating robustness assessment alongside p-values in routine evidence synthesis yields a more complete picture of evidential strength\u003c/p\u003e","manuscriptTitle":"Neutrality Boundary Robustness for Meta-Analyses","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-07 09:03:36","doi":"10.21203/rs.3.rs-8457715/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e871e90c-d592-4ce8-a365-dd2c3a1d4d08","owner":[],"postedDate":"January 7th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":60254921,"name":"Biostatistics"}],"tags":[],"updatedAt":"2026-01-07T09:03:36+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-07 09:03:36","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8457715","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8457715","identity":"rs-8457715","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}