Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies

Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies Moritz Felsch, Lars Beckmann, Ralf Bender, Oliver Kuss, Guido Skipka, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1712108/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background Meta-analyses are used to summarize the results of several studies to a specific research question. Standard methods for meta-analyses, namely inverse variance random effects models have unfavourable properties if only very few (2-4) studies are available. Therefore, alternative meta-analytic methods are needed. In case of binary data, the “common-rho” beta-binomial model has shown good results in situations with spare data or few studies. The major concern of this model is that it ignores the fact that each treatment arm is paired with a respective control arm from the same study. Thus, the randomisation to a study arm of a specific study is disrespected, which may lead to compromised estimates of the treatment effect. Therefore, we extended this model to a version that respects randomisation. The aim of this simulation study was to compare “common-rho” beta-binomial model and several other beta-binomial models to standard meta-analyses models including generalised linear mixed models and several inverse variance random effects models. Methods We conducted a simulation study in which beta-binomial models and various standard meta-analysis methods were compared. The design of the simulation aimed to consider meta-analytic situations occurring in practice. Results In summary, no method performed well in scenarios with only 2 studies in the random effects scenario. In this situation, a fixed effect model or a qualitative summary of the study results may be preferable. In scenarios with 3 or 4 studies the “common-rho” beta-binomial model performed at least slightly better than other models, whereas performance was not improved by the beta-binomial model respecting randomisation. Conclusion The “common-rho” beta-binomial appears to be a good option for meta-analyses of very few studies. Because residual concerns about the consequences of disrespecting the randomisation may still exist, we recommend a sensitivity analysis with a standard meta-analysis method that respects randomisation. beta-binomial model generalised linear mixed models meta-analyses simulation study few studies Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Full Text Additional Declarations No competing interests reported. Supplementary Files Supportinglnformation1.docx Supportinglnformation2.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Major revision 26 Jul, 2022 Reviews received at journal 12 Jul, 2022 Reviewers agreed at journal 05 Jul, 2022 Reviewers agreed at journal 01 Jul, 2022 Reviewers invited by journal 29 Jun, 2022 Editor assigned by journal 02 Jun, 2022 Editor invited by journal 02 Jun, 2022 Submission checks completed at journal 02 Jun, 2022 First submitted to journal 31 May, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1712108","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":110659472,"identity":"15aa9db6-58d9-4d9c-8842-5d7a1d131d68","order_by":0,"name":"Moritz Felsch","email":"","orcid":"","institution":"Institute for Quality and Efficiency in Health Care","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Moritz","middleName":"","lastName":"Felsch","suffix":""},{"id":110659476,"identity":"dba447ee-9ecc-41c4-a546-019acbdd03e4","order_by":1,"name":"Lars Beckmann","email":"","orcid":"","institution":"Institute for Quality and Efficiency in Health Care","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Lars","middleName":"","lastName":"Beckmann","suffix":""},{"id":110659477,"identity":"b19456c9-6170-44bd-b9e7-54680a45d7b4","order_by":2,"name":"Ralf Bender","email":"","orcid":"","institution":"Institute for Quality and Efficiency in Health Care","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ralf","middleName":"","lastName":"Bender","suffix":""},{"id":110659478,"identity":"c9dd42ef-d18b-475a-a9e9-f8dba4575998","order_by":3,"name":"Oliver Kuss","email":"","orcid":"","institution":"Leibniz Institute for Diabetes Research at Heinrich Heine University Düsseldorf","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Oliver","middleName":"","lastName":"Kuss","suffix":""},{"id":110659480,"identity":"ffb342a6-d35b-411c-b6d0-e37d258f29b3","order_by":4,"name":"Guido Skipka","email":"","orcid":"","institution":"Institute for Quality and Efficiency in Health Care","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Guido","middleName":"","lastName":"Skipka","suffix":""},{"id":110659481,"identity":"17fab254-2c4f-43e3-b16e-a98a1646250b","order_by":5,"name":"Tim Mathes","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABS0lEQVRIie2RMUvDQBTHXz24DubMmlJtv8LJgSAV/Cq5pVkSEQqlQ9GCEBcx632MuFScLAQ6xRa3lC5KoDgGxCJYi5e01FBTcBTJb7l3x/3u/bkHkJPzF0GLRZdVDwB/nyR1artCWZxQqWA9rVBtowIrRaGrlxMFNijHxW19K2jPTlTVf3OjJvDbIqCXqDU72ylfPoanUKv8CEZcZPZpoySs7kgMgN9dAD4Uvgy26zeYAINlKphyNyDdMbGBu546ZcSWimbWywp4vJOlzCm/H/qT8WeiAGZknijGh1TOsxTLll3AxOPCQkEh6cSK0UdS0deDecT1rGvGRVA/GF0NNBZ3KYg+K9maicoKNfbXuhSdh5vQnFa443iT4L15tOcOeyiK2hW1KoznV6VVq67/sqSXqpNp4OVM4knRDCEDFC0jPP3ufk5OTs4/5wtbjmr0JKeeHwAAAABJRU5ErkJggg==","orcid":"","institution":"University Medical Centre Göttingen","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Tim","middleName":"","lastName":"Mathes","suffix":""}],"badges":[],"createdAt":"2022-05-31 13:44:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1712108/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1712108/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":22507597,"identity":"e826b4b0-7f6f-47e8-90c4-5087f28b9433","added_by":"auto","created_at":"2022-06-10 15:34:35","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":479122,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure1ConvergenceORH0.jpg","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/1df92b264e8a0d838b6377c5.jpg"},{"id":22508396,"identity":"d5240dd7-4b3f-40bf-9489-45c37ffa1a89","added_by":"auto","created_at":"2022-06-10 15:39:35","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":454260,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure2CoverageprobabilityORH0new.jpg","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/d7e05073c1c75a698aa0093a.jpg"},{"id":22507591,"identity":"86d4e56c-fd22-4207-9300-e2a97911cdfb","added_by":"auto","created_at":"2022-06-10 15:34:35","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":475262,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure3CoverageprobabilityORH1new.jpg","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/1c35e60122d1037741da7c4b.jpg"},{"id":22507592,"identity":"2ab343a4-e521-486f-b953-ca6b29774db1","added_by":"auto","created_at":"2022-06-10 15:34:35","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":486754,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure4PowerORH1new.jpg","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/909ca047cc02d80c536938b0.jpg"},{"id":22507596,"identity":"d814eec3-7420-414f-876f-77025564dfe2","added_by":"auto","created_at":"2022-06-10 15:34:35","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":451307,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure5CoverageprobabilityORH0qnew.jpg","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/0b2ab9e18faa60ac0c6b0480.jpg"},{"id":22508398,"identity":"65ac6e89-fafa-4f28-bdcb-99a78785e71f","added_by":"auto","created_at":"2022-06-10 15:39:42","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":841414,"visible":true,"origin":"","legend":"","description":"","filename":"BBMAveryfewstudiesfinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1_covered.pdf"},{"id":22508397,"identity":"67cd6fa4-18d4-492c-84ce-8b54608ec3a0","added_by":"auto","created_at":"2022-06-10 15:39:35","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":73625,"visible":true,"origin":"","legend":"","description":"","filename":"Supportinglnformation1.docx","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/d0b41be924cbaf87b0105a96.docx"},{"id":22507594,"identity":"9d199df2-fe53-4585-aae0-35912683bc18","added_by":"auto","created_at":"2022-06-10 15:34:35","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":105303,"visible":true,"origin":"","legend":"","description":"","filename":"Supportinglnformation2.docx","url":"https://assets-eu.researchsquare.com/files/rs-1712108/v1/6bc3a7cf344c2121d730b8c4.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1712108/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"beta-binomial model, generalised linear mixed models, meta-analyses, simulation study, few studies","lastPublishedDoi":"10.21203/rs.3.rs-1712108/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1712108/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003e\u003cem\u003eBackground\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\u003cp\u003eMeta-analyses are used to summarize the results of several studies to a specific research question. Standard methods for meta-analyses, namely inverse variance random effects models have unfavourable properties if only very few (2-4) studies are available. Therefore, alternative meta-analytic methods are needed. In case of binary data, the “common-rho” beta-binomial model has shown good results in situations with spare data or few studies. The major concern of this model is that it ignores the fact that each treatment arm is paired with a respective control arm from the same study. Thus, the randomisation to a study arm of a specific study is disrespected, which may lead to compromised estimates of the treatment effect. Therefore, we extended this model to a version that respects randomisation. \u003c/p\u003e\u003cp\u003eThe aim of this simulation study was to compare “common-rho” beta-binomial model and several other beta-binomial models to standard meta-analyses models including generalised linear mixed models and several inverse variance random effects models.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003e\u003cem\u003eMethods\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\u003cp\u003eWe conducted a simulation study in which beta-binomial models and various standard meta-analysis methods were compared. The design of the simulation aimed to consider meta-analytic situations occurring in practice. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003e\u003cem\u003eResults\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\u003cp\u003eIn summary, no method performed well in scenarios with only 2 studies in the random effects scenario. In this situation, a fixed effect model or a qualitative summary of the study results may be preferable. In scenarios with 3 or 4 studies the “common-rho” beta-binomial model performed at least slightly better than other models, whereas performance was not improved by the beta-binomial model respecting randomisation. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003e\u003cem\u003eConclusion\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\u003cp\u003eThe “common-rho” beta-binomial appears to be a good option for meta-analyses of very few studies. Because residual concerns about the consequences of disrespecting the randomisation may still exist, we recommend a sensitivity analysis with a standard meta-analysis method that respects randomisation.\u003c/p\u003e","manuscriptTitle":"Performance of several types of beta-binomial models in comparison to standard approaches for meta-analyses with very few studies","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-06-10 15:34:33","doi":"10.21203/rs.3.rs-1712108/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2022-07-26T11:23:15+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-07-12T05:19:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"fb10e3b7-0eb6-462f-b9fd-b95decd53d24","date":"2022-07-05T10:30:45+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"40e684e3-1bf5-4bb1-a083-d9fbd591f8d1","date":"2022-07-01T20:42:50+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-06-29T16:25:56+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-06-02T12:07:50+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2022-06-02T11:59:46+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2022-06-02T11:56:43+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2022-05-31T13:32:38+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"667196f6-46bc-4b98-9324-bccc21490994","owner":[],"postedDate":"June 10th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2022-10-29T10:44:22+00:00","versionOfRecord":[],"versionCreatedAt":"2022-06-10 15:34:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1712108","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1712108","identity":"rs-1712108","version":["v1"]},"buildId":"FbvkV6FR0MCFSLy54lSbu","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}