AppRaise: Software for quantifying evidence uncertainty in systematic reviews using a Bayesian mixture model

AppRaise: Software for quantifying evidence uncertainty in systematic reviews using a Bayesian mixture model | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article AppRaise: Software for quantifying evidence uncertainty in systematic reviews using a Bayesian mixture model Conrad Kabali This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6026144/v2 This work is licensed under a CC BY 4.0 License Status: Posted Version 2 posted You are reading this latest preprint version Show more versions Abstract Rationale: Systematic reviews are essential for evidence-based healthcare decision-making. While it is relatively straightforward to quantitatively assess random errors in systematic reviews, as these are typically reported in primary studies, the assessment of biases often remains narrative. Primary studies seldom provide quantitative estimates of biases and their uncertainties, resulting in systematic reviews rarely including such measurements. Additionally, evidence appraisers often face time constraints and technical challenges that prevent them from conducting quantitative bias assessments themselves. Given that multiple biases and random errors collectively skew the point estimate from the truth, it is important to incorporate comprehensive quantitative methods of uncertainty in systematic reviews. These methods should integrate random errors and biases into a unified measure of uncertainty and be easily accessible to evidence appraisers, preferably through user-friendly software. Aims and objectives: To address this need, we propose a Bayesian mixture model and introduce AppRaise, a free, web-based interactive software designed to implement this approach. Method: We showcase its application through a health technology assessment (HTA) report on the effectiveness of continuous glucose monitoring in reducing A1c levels among individuals with type 1 diabetes. Results: Applying the AppRaise software to the HTA report revealed a high level of certainty (86% probability) that continuous glucose monitoring would, on average, result in a reduction in A1c levels compared with self-monitoring of blood glucose among Ontarians with type 1 diabetes. These findings were nearly identical or similar to other quantitative bias-adjusted approaches in systematic reviews. Conclusion: AppRaise can be utilized as a standalone tool or as a complement to validate the quality of evidence assessed using qualitative-based scoring methods. Decision Sciences AppRaise quantifying uncertainty health technology assessment decision making Bayesian mixture model systematic review Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 2 posted You are reading this latest preprint version Show more versions Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6026144","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":416662801,"identity":"e6e5dcca-f40d-48e4-aac2-d749cf083e5c","order_by":0,"name":"Conrad Kabali","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIiWNgGAWjYNACNgYDBgkGxgMJFSRqYTiQcAZEAcEBorUwthGhhb+999iHH2UMxvyzmx8ceDjvcB2/9OFjjz9UMMjzN2DXInHmXPLMnnMMZhJ3jhkcSNx2WEKyLy3d4MAZBsMZOKwykMgxZuBtY7BhuJEA0WJwhsdM4mAbQwIu14G0MP4FapG/kf7hQOIcJC3yeLQwA20xM7iRA7SlAUmLAQ4tEmfOGDPLnJMwNrxzpuBAwrF0yZk9bGlAUQnDjTi08Lf3GDO+KbMxnHe7fePDHzXW/Pw8zMckKips5OVwaIFZRoTIKBgFo2AUjALiAQBBBlryu02RsgAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-0553-0305","institution":"University of Toronto","correspondingAuthor":true,"prefix":"","firstName":"Conrad","middleName":"","lastName":"Kabali","suffix":""}],"badges":[],"createdAt":"2025-02-13 23:06:24","currentVersionCode":2,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6026144/v2","doiUrl":"https://doi.org/10.21203/rs.3.rs-6026144/v2","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":83161057,"identity":"d24e5b36-13df-496d-a458-a8a7c6f6e4be","added_by":"auto","created_at":"2025-05-20 15:12:32","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":467353,"visible":true,"origin":"","legend":"","description":"","filename":"appraise.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6026144/v2_covered_1a081147-01c7-400f-aacc-6b5f5b2a8e39.pdf"}],"financialInterests":"\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e","formattedTitle":"AppRaise: Software for quantifying evidence uncertainty in systematic reviews using a Bayesian mixture model","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"AppRaise, quantifying uncertainty, health technology assessment, decision making, Bayesian mixture model, systematic review","lastPublishedDoi":"10.21203/rs.3.rs-6026144/v2","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6026144/v2","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eRationale: Systematic reviews are essential for evidence-based healthcare decision-making.\u003c/p\u003e\n\u003cp\u003eWhile it is relatively straightforward to quantitatively assess random errors in systematic\u003c/p\u003e\n\u003cp\u003ereviews, as these are typically reported in primary studies, the assessment of biases often\u003c/p\u003e\n\u003cp\u003eremains narrative. Primary studies seldom provide quantitative estimates of biases and\u003c/p\u003e\n\u003cp\u003etheir uncertainties, resulting in systematic reviews rarely including such measurements.\u003c/p\u003e\n\u003cp\u003eAdditionally, evidence appraisers often face time constraints and technical challenges that\u003c/p\u003e\n\u003cp\u003eprevent them from conducting quantitative bias assessments themselves. Given that multiple\u003c/p\u003e\n\u003cp\u003ebiases and random errors collectively skew the point estimate from the truth, it is important to\u003c/p\u003e\n\u003cp\u003eincorporate comprehensive quantitative methods of uncertainty in systematic reviews. These\u003c/p\u003e\n\u003cp\u003emethods should integrate random errors and biases into a unified measure of uncertainty and\u003c/p\u003e\n\u003cp\u003ebe easily accessible to evidence appraisers, preferably through user-friendly software.\u003c/p\u003e\n\u003cp\u003eAims and objectives: To address this need, we propose a Bayesian mixture model and\u003c/p\u003e\n\u003cp\u003eintroduce AppRaise, a free, web-based interactive software designed to implement this\u003c/p\u003e\n\u003cp\u003eapproach.\u003c/p\u003e\n\u003cp\u003eMethod: We showcase its application through a health technology assessment (HTA) report\u003c/p\u003e\n\u003cp\u003eon the effectiveness of continuous glucose monitoring in reducing A1c levels among individuals\u003c/p\u003e\n\u003cp\u003ewith type 1 diabetes.\u003c/p\u003e\n\u003cp\u003eResults: Applying the AppRaise software to the HTA report revealed a high level of certainty\u003c/p\u003e\n\u003cp\u003e(86% probability) that continuous glucose monitoring would, on average, result in a reduction\u003c/p\u003e\n\u003cp\u003ein A1c levels compared with self-monitoring of blood glucose among Ontarians with type 1\u003c/p\u003e\n\u003cp\u003ediabetes. These findings were nearly identical or similar to other quantitative bias-adjusted\u003c/p\u003e\n\u003cp\u003eapproaches in systematic reviews.\u003c/p\u003e\n\u003cp\u003eConclusion: AppRaise can be utilized as a standalone tool or as a complement to validate\u003c/p\u003e\n\u003cp\u003ethe quality of evidence assessed using qualitative-based scoring methods.\u003c/p\u003e","manuscriptTitle":"AppRaise: Software for quantifying evidence uncertainty in systematic reviews using a Bayesian mixture model","msid":"","msnumber":"","nonDraftVersions":[{"code":2,"date":"2025-05-20 14:56:24","doi":"10.21203/rs.3.rs-6026144/v2","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}},{"code":1,"date":"2025-02-17 05:58:53","doi":"10.21203/rs.3.rs-6026144/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a8222ffd-8daf-4abe-a3ae-1acc34288587","owner":[],"postedDate":"May 20th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":44422224,"name":"Decision Sciences"}],"tags":[],"updatedAt":"2025-02-17T05:58:53+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-20 14:56:24","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v2","identity":"rs-6026144","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6026144","identity":"rs-6026144","version":["v2"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}