The Composite Quality Score for the appraisal of prospective controlled clinical therapy trials in systematic reviews and its limits (Preprint)

The Composite Quality Score for the appraisal of prospective controlled clinical therapy trials in systematic reviews and its limits (Preprint) | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article The Composite Quality Score for the appraisal of prospective controlled clinical therapy trials in systematic reviews and its limits (Preprint) Steffen Mickenautsch, Stefan Rupf, Veerasamy Yengopal This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2785700/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 27 Jun, 2023 Read the published version in Frontiers in Medicine → Version 1 posted You are reading this latest preprint version Abstract Systematic reviews of prospective controlled clinical therapy trials are one of the most important sources of information in modern medicine. Besides the systematic search for and statistical pooling of current clinical trial data for a particular type of therapy, systematic reviews also have the task of appraising the quality of trial results. The quality of trial results may be diminished by low internal trial validity, due to systematic error (bias). A high risk of bias may likely cause the reported trial results to be diverted from the actual true therapeutic effect and thus render it unsuitable for clinical guidance. According to the Cochrane Collaboration, the risk of bias in clinical therapy trials should be assessed using its Risk of Bias tool, Version 2 (RoB 2). However, the tool has been established to have poor inter-rater reliability, with a limited empirical evidence base and described as complex and demanding. Against this background, the Composite Quality Score (CQS) has been developed as a possible alternative trial appraisal tool, characterised by high epistemic rigour, empirical evidence base, inter-rater reliability and ease of use. This article presents the current evidence of the CQS and its limitations. Internal Medicine composite quality score clinical trial trial appraisal risk of bias tool Introduction According to the Cochrane Collaboration, the risk of bias in clinical therapy trials should be assessed using its Risk of Bias tool, Version 2 (RoB 2). The tool consists of 22 signalling questions to five bias domains (randomisation process, deviation from intended interventions, missing outcome data, measurement of outcome, and selection of the reported result) [ 1 ]. The tool was developed between 2015–19, based on a lengthy process of expert consensus, repeated cycles of discussion, piloting and rewording. The development was further supported by the results of a systematic review of meta-epidemiological studies concerning empirical evidence of bias effect on clinical trial results [ 2 ], a systematic review of types of bias in epidemiology [ 1 ] and a cross-sectional study on selective outcome reporting [ 3 ]. Despite its rigorous development process, the RoB 2 tool has to date been shown to have poor inter-rater reliability (IRR: Fleiss’ Kappa 0.16; 95% CI: 0.08–0.24) and its application has been described as complex and demanding, therefore requiring intensive formal training and the conduct of pilot runs before it may correctly be applied [ 4 ]. Even after the inclusion of an intensive rater-calibration process, including a total of 40 hours over a period of three months, the overall IRR remained moderate (IRR = 0.42) only [ 5 ]. During its developmental process, the RoB 2 tool is claimed to have been based on empirical evidence from meta-epidemiological studies [ 2 ]. However, the systematic review by Page et al. (2016) could provide evidence in support of only 5 out of the 22 signalling questions of the tool. This evidence supports the signalling question number 1.1 and 1.2, namely whether the allocation concealment was random (7% overestimation of trials with inadequate or unclear randomisation: ROR 0.93, 95% CI: 0.86–0.99, I 2 = 0%) and whether the allocation sequence was concealed (7% overestimation of trials with inadequate or unclear allocation concealment: ROR 0.90, 95% CI: 0.84–0.97, I 2 = 28%), respectively. The evidence also supports in principle the signalling questions number 4.3–4.5 in the domain “bias in measurement of the outcome; that is, in support for blinded assessment of susceptive outcomes and double-blinding (23% overestimation of trials with inadequate or unclear double-blinding: ROR 0.77, 95% CI: 0.61–0.93). However, “blinding” was not explicitly included in formulating the tool’s questions. The systematic review did not identify empirical evidence supporting 3 out of the tool’s 5 bias domains (including 17 out of its total 22 signalling questions), namely “deviation from intended intervention”, “missing outcome data” and “selective reporting” [ 2 ]. Besides its poor inter-rater reliability (IRR) and limited evidence base, its complex and demanding application may be the reason why a low adherence to the RoB 2 tool among Cochrane protocols and systematic reviews was observed and why most Cochrane reviewers themselves choose not to use the tool [ 6 ]. The complexity of applying the RoB 2 tool stands in contrast to the steadily increasing volume of clinical intervention trials worldwide [ 7 ] and thus the subsequent need arises for more timely, less complicated, yet effective and reliable trial appraisal methods. In addition to the above, the RoB 2 tool assigns an overall “low-bias risk” status to trials that have been judged to be of “low-bias risk” in all of its five single bias domains, even though any methodological trial error that lies outside its five domains may completely invalidate the reported trial results. Against this background, the Composite Quality Score (CQS) has been developed as a possible alternative for the appraisal of prospective controlled clinical therapy trials in systematic reviews. The Composite Quality Score (Cqs) The development of the CQS as a possible trial appraisal tool started in 2019 with the epistemic consideration that regardless of how many trial appraisal criteria a trial has fully complied with, the judgment of such a trial as being of “low-bias risk” cannot be justified but a single methodological error may render a trial to actually be of “high-bias risk”. Accordingly, three evidence-based criteria describing trial design requirements (random allocation sequence generation, allocation sequence concealment, minimum sample size) that are essential (albeit not sufficient) for the trial results to reflect therapeutic truth were established. The criteria were intentionally worded in a simplified and least restrictive manner, based on the rationale that the lower the stringency of an appraisal criterion, the higher the certainty of “high-bias risk” when such criterion is not met [ 8 ]. To explore the applicability of the novel CQS approach for trial appraisal, it was applied in the field of restorative dentistry. Based on a systematic literature search, 683 prospective clinical controlled trial reports were identified out of a total of 14 694 citations. Of these, the bias risk of 99.7% of trials could be appraised and identified as high, despite the low stringency of the applied criteria [ 9 ]. In 2021, the inter-rater reliability of the CQS approach was investigated for the first time. The results showed a high inter-rater reliability: Brennan-Prediger coefficient (BPC) of 0.95; 95% CI: (0.87–1.00), which compared favourably to that of the first RoB tool version, which ranged from BPC − 0.07; 95% CI: -0.42, – 0.28 and 0.34; 95% CI: -0.05–0.73. Most of the differences between the RoB and the CQS were statistically significant (p < 0.05) in favour of the CQS [ 10 ]. The findings of all three preliminary investigations [ 8 – 10 ] were summarised and presented as the first version of the CQS (CQS-1) in 2023 [ 11 ]. In order to extend the applicability and evidence base of the CQS-1 for all fields of clinical therapy, a systematic review of meta-epidemiological studies was conducted. Based on its results, one new criterion concerning double-blinding was added, and the original criteria concerning random allocation sequence generation and minimum sample size were amended [ 12 ]. Consequently, all four criteria of the resulting new CQS version (CQS-2) were based on meta-epidemiological evidence, indicating statistically significant (p < 0.05) over- or underestimation of the true therapeutic effect estimate for trials where such criteria were not met. This systematic review was partially updated to improve the wording and applicability of the criterion concerning the appraisal of allocation concealment, resulting in a further refined CQS version (CQS-2B) [ 13 ]. Following the extension of the original CQS-1, the CQS-2 was assessed to establish whether its extended format extension would have negatively affected the high inter-rater reliability of the first version. The results of this study showed that CQS extension had no negative effect and that the CQS-2 version was also associated with very high inter-rater reliability (BPC 1.00; 95% CI: 0.94–1.00) and did not statistically differ significantly (p > 0.05) from that of the CQS-1 (BPC 0.85; 95% CI: 0.64 − 1.00) [ 14 ]. The result of this assessment led to the development of a CQS version (CQS-2B) that included four criteria related to the random allocation to treatment groups, concealment of such allocation, double-blinding and sample size minimum (Table 1 ). Table 1 CQS-2B appraisal criteria Criterion I ‘Randomisation’ for allocation to treatment groups is in some form reported in the text Yes No Criterion II Any assurance that the patient allocation to treatment groups according to the random sequence was applied by an independent agent or agency, not otherwise involved in the trial, is in some form reported in the text Criterion III Double-blinding or the blinding of at least two out of the three groups: trial participants, trial personnel and trial outcome assessors in some form reported in the text Criterion IV The sample size of any particular treatment group reported in the trial is not less than N = 100 Its application comprised: (i) binary trial report rating per appraisal criterion (Scores: 0 = No/invalid/falsified, 1 = Yes/corroborated); (ii) multiplication of all criterion scores to an overall appraisal score, (iii) identification of invalid/falsified trial reports based on a zero overall appraisal score. During its application, several corroboration (C-) levels are recognised. These levels indicate the number of consecutive criteria that a trial has complied with (e.g. level C3 indicates compliance with Criterion I - III; level C4 indicates compliance with Criterion I - IV, etc.). A corroboration level for a particular trial is reached before one criterion is rated with a 0-score or when all criteria are rated with a 1-score, for example Corroboration level C3: Criterion I–III = 1-score, Criterion IV = 0-score; Corroboration level C4: All criteria = 1-score. After a criterion has been rated with a 0-score, the C-level of a trial remains the same even if a following criterion is rated with a 1-score, for example Corroboration level C1: Criterion I = 1-score, Criterion II = 0-score, Criterion III = 1-score. No certainty of “low-bias risk” is ascribed to an overall 1-score appraisal result, which indicates only that during the appraisal process no evidence for high-bias risk could be established so far. It was explored whether systematic review conclusions originally based on Cochrane’s second version of its Risk of Bias tool (RoB 2) would differ when the CQS-2B is used for trial appraisal instead. The results of this study provided justification for the testable hypothesis that trial appraisal using the CQS-2B provides more conservative conclusions based on similar data than trial appraisal using Cochrane’s RoB 2 tool [ 15 ]. CQS Limitations Despite its high epistemic rigour, evidence base and inter-rater reliability, the current version of the CQS has certain limitations that need to be taken into account when considering its application in systematic reviews of prospective controlled clinical therapy trials. These include the following items. Verification of low-bias risk As per the design, the CQS-2B is not a tool for verifying low-bias risk. Trials that are awarded 1-scores for all four appraisal criteria may still have serious flaws in other aspects of their methodology. This is a fully intended aspect of the CQS approach in line with the epistemic principles of deductive falsification [ 8 ]. One example is the clinical trial by Sitthisettapong et al. (2012) concerning the clinical effect of 10% w/v calcium phosphopeptide-amorphous calcium phosphate (CPP-ACP) paste for one year when added to regular toothbrushing with fluoridated toothpaste to prevent dental caries in pre-school children [ 16 ]. The trial reports to be “randomised” (Criterion I = 1-score) with the allocation of experimental or control group were determined by an assistant who was not involved in the clinical aspects of the study (Criterion II = 1-score), “double-blind” (Criterion III = 1-score) and included patients of which “150 were assigned to the experimental condition and 146 to the control condition” (Criterion IV = 1-score). However, subsequent further in-depth appraisal reveals that patient allocation was neither random nor concealed but alternated with patients “with odd ID numbers assigned to the experimental treatment, and those with even ID numbers to the control treatment”. This form of alternation may cause the allocation of patients to either group to be visible, which renders the study open to a high risk of selection bias. Furthermore, the important baseline variable “falling asleep with a bottle” was statistically significantly more distributed among the experimental group (p = 0.008). If randomisation had been effective, then all baseline variables should have been equally distributed between the groups. This example shows that corroborated trials according to the CQS-2B cannot be considered as low-bias risk. Instead, corroborated trials may be considered eligible for further in-depth appraisal as shown in the example above. Such appraisal may not be confined within the criteria of any specific trial appraisal tool but may need to be tailored specifically to the particularities of each trial. In contrast, a further in-depth appraisal may not be warranted for trials that have been already falsified at C1–4 levels since the appraisal process has already provided sufficient reason to consider such trials as of high-bias risk. In that way, the CQS-2B may contribute to a more timely and efficient trial appraisal, particularly of a large volume of prospective controlled clinical therapy trials [ 7 ]. The discrepancy between not reported versus actual trial characteristics Not all actual trial characteristics that were performed during the conduct of a trial, required by the CQS criteria, may have been reported by trial authors. For example, trial randomisation may have included central allocation, but this was not made explicit in the published trial report. Trial appraisal using the CQS-2B would thus have assigned an erroneous 0–score to its Criterion II. Contacting trial authors during a systematic review of trials may appear to be a possible solution but can be affected by recall bias, the fact that such author responses have not been part of the original peer-review process and the reduction of possible reproducibility of the systematic review results at a later stage. Trial appraisal remains by its nature qualitative text analysis only. This is valid for any trial appraisal tool, such as the CQS-2B or Cochrane’s RoB 2. The consequence is false-negative trial appraisal results that may not be remedied. The discrepancy between not performed trial methods and systematic error effect The lack of essential trial characteristics may not always lead to systematic error. This limitation is also valid for any trial appraisal tool. Trial appraisal based on qualitative text analysis can assume whether a trial is susceptible to systematic error only when essential trial characteristics are missing. Quantitative tests that can be applied during the systematic review of clinical trials may solve this problem. Currently, there are only a few tests available for this purpose, such as the highly accurate Berger-Exner test for 3rd -order selection bias [ 17 ]. However, the currently existing tools do not yet cover all bias domains and thus do not replace trial appraisal based on qualitative text analysis. Therefore, the CQS-2B can establish only a “high-bias risk” of a trial but cannot actually provide proof that a trial result is biased. For this reason and in agreement with guidelines for the RoB 2 tool [ 1 ], it is strongly recommended that systematic reviews with the CQS-2B should not exclude high-bias risk trial data but always include stratification by overall bias risk for trial outcomes or endpoints during meta-analysis. Conclusion The CQS has been developed based on high epistemic rigour, meta-epidemiological evidence and inter-rater reliability. However, like all trial appraisal tools, it has limitations that need to be considered during its application in systematic reviews of prospective controlled clinical therapy trials. Declarations Financial Disclosure The authors received no specific funding for this work. Data availability All data is made fully available without restriction as part of the preprint. Competing Interests The authors declare that no competing interests exist References Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366: l4898. Page MJ, Higgins JP, Clayton G, Sterne JA, Hróbjartsson A, Savović J. Empirical evidence of study design biases in randomized trials: systematic review of meta-epidemiological studies. PLoS One. 2016; 11(7):e0159267. Page MJ, Higgins JPT. Rethinking the assessment of risk of bias due to selective reporting: a cross-sectional study. Syst Rev 2016;5:108. Minozzi S, Cinquini M, Gianola S, Gonzalez-Lorenzo M, Banzi R. The revised Cochrane risk of bias tool for randomized trials (RoB 2) showed low interrater reliability and challenges in its application. J Clin Epidemiol. 2020; 126:37-44. Minozzi S, Dwan K, Borrelli F, Filippini G. Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022 Jan;141:99-105. doi: 10.1016/j.jclinepi.2021.09.021. Epub 2021 Sep 16. PMID: 34537386. Martimbianco ALC, Sá KMM, Santos GM, Santos EM, Pacheco RL, Riera R. Most Cochrane systematic reviews and protocols did not adhere to the Cochrane's risk of bias 2.0 tool. Rev Assoc Med Bras (1992). 2023 Feb 20;69(3):469-472. doi: 10.1590/1806-9282.20221593. PMID: 36820779; PMCID: PMC10004297. World Health Organization. Number of clinical trial registrations by location, disease, phase of development, age and sex of trial participants (1999-2021). Website: https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-trial-registrations-by-year-location-disease-and-phase-of-development, 2022 (accessed: November 13, 2022). Mickenautsch S. Is the Deductive Falsification Approach a Better Basis for Clinical Trial Appraisal? Rev Recent Clin Trials. 2019; 14:224-228. Mickenautsch S. Are Most of the Published Clinical Trial Results in Restorative Dentistry Invalid? An Empirical Investigation. Rev Recent Clin Trials. 2020;15(2):122-130. Mickenautsch S, Miletić I, Rupf S, Renteria J, Göstemeyer G. The Composite Quality Score (CQS) as a trial appraisal tool: inter-rater reliability and rating time. Clin Oral Investig. 2021; 25:6015-23. Mickenautsch S, Rupf S, Miletić I, Yengopal V. The Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials: rationale and current evidence. Rev Recent Clin Trials. 2023; 18:28-33. Mickenautsch S, Rupf S, Miletić I, Yengopal V. Extension of the Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials-A systematic review of meta-epidemiological evidence. PLoS One. 2022; 17:e0279645. Mickenautsch S, Yengopal V. Allocation concealment appraisal of clinical therapy trials using the extended Composite Quality Score (CQS-2) – An empirically based update (Preprint), 14 February 2023, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2582208/v1]. Mickenautsch S, Rupf S, Miletić I, Strähle UT, Sturm R, Kimmie-Dhansay F, Vidosusić K, Yengopal V. Inter-rater reliability of the extended Composite Quality Score (CQS-2) – a pilot study, 23 November 2022, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2297364/v1] Mickenautsch S, Rupf S, Yengopal V. Application of the Composite Quality Score (CQS-2B) in systematic reviews of prospective, controlled, clinical therapy trials – an exploratory study (Preprint), 23 March 2023, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2718841/v1]. Sitthisettapong T, Phantumvanit P, Huebner C, Derouen T. Effect of CPP-ACP paste on dental caries in primary teeth: a randomized trial. J Dent Res. 2012 Sep;91(9):847-52. Mickenautsch S, Fu B, Gudehithlu S, Berger VW. Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation. BMC Med Res Methodol. 2014 Oct 6;14:114. Cite Share Download PDF Status: Published Journal Publication published 27 Jun, 2023 Read the published version in Frontiers in Medicine → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2785700","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":189716555,"identity":"9056eca2-fe9f-41cf-b497-68574faf3085","order_by":0,"name":"Steffen Mickenautsch","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAy0lEQVRIiWNgGAWjYLCCDwwWDAYMzA3E62CcwSAB1MJIghZmHpK0mLM3H91s80tCzpz9YAPDjwoGe35COi17jqXdzu2TMLbsSWxg7DnDwCxxgIAWgxs5ZrdzeyQSNxxIbGBmbGNgYyCo5f77b7ctQVrOPwRr4ZEnbAsP222GH0AtNyC2SBgQ1HImzexmbwPQLzMeNhzsOSNhYEhQy/HDz278+GMjZ86ffPDBjwobezlCWsAA6B4wACqWIEY9CPwhVuEoGAWjYBSMSAAAHuZCQA+ZtRUAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-4994-2328","institution":"University of the Western Cape","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Steffen","middleName":"","lastName":"Mickenautsch","suffix":""},{"id":189716556,"identity":"129814b8-1868-4dee-9b3f-98608e3ea08f","order_by":1,"name":"Stefan Rupf","email":"","orcid":"","institution":"Saarland University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Stefan","middleName":"","lastName":"Rupf","suffix":""},{"id":189716557,"identity":"8853cfdb-2358-4be3-a542-85413427eb2f","order_by":2,"name":"Veerasamy Yengopal","email":"","orcid":"","institution":"University of the Western Cape","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Veerasamy","middleName":"","lastName":"Yengopal","suffix":""}],"badges":[],"createdAt":"2023-04-06 12:07:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2785700/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2785700/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.3389/fmed.2023.1201951","type":"published","date":"2023-06-28T00:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":42101511,"identity":"64a0b8bc-9fbf-4796-b6d9-ec40f7ecd5a4","added_by":"auto","created_at":"2023-08-24 18:37:36","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":259281,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2785700/v1/ede0b47d-6e2c-48be-9983-5ec5274f89d9.pdf"}],"financialInterests":"","formattedTitle":"\u003cp\u003eThe Composite Quality Score for the appraisal of prospective controlled clinical therapy trials in systematic reviews and its limits (Preprint)\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eAccording to the Cochrane Collaboration, the risk of bias in clinical therapy trials should be assessed using its Risk of Bias tool, Version 2 (RoB 2). The tool consists of 22 signalling questions to five bias domains (randomisation process, deviation from intended interventions, missing outcome data, measurement of outcome, and selection of the reported result) [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe tool was developed between 2015\u0026ndash;19, based on a lengthy process of expert consensus, repeated cycles of discussion, piloting and rewording. The development was further supported by the results of a systematic review of meta-epidemiological studies concerning empirical evidence of bias effect on clinical trial results [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e], a systematic review of types of bias in epidemiology [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e] and a cross-sectional study on selective outcome reporting [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDespite its rigorous development process, the RoB 2 tool has to date been shown to have poor inter-rater reliability (IRR: Fleiss\u0026rsquo; Kappa 0.16; 95% CI: 0.08\u0026ndash;0.24) and its application has been described as complex and demanding, therefore requiring intensive formal training and the conduct of pilot runs before it may correctly be applied [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Even after the inclusion of an intensive rater-calibration process, including a total of 40 hours over a period of three months, the overall IRR remained moderate (IRR\u0026thinsp;=\u0026thinsp;0.42) only [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDuring its developmental process, the RoB 2 tool is claimed to have been based on empirical evidence from meta-epidemiological studies [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. However, the systematic review by Page et al. (2016) could provide evidence in support of only 5 out of the 22 signalling questions of the tool. This evidence supports the signalling question number 1.1 and 1.2, namely whether the allocation concealment was random (7% overestimation of trials with inadequate or unclear randomisation: ROR 0.93, 95% CI: 0.86\u0026ndash;0.99, I\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;0%) and whether the allocation sequence was concealed (7% overestimation of trials with inadequate or unclear allocation concealment: ROR 0.90, 95% CI: 0.84\u0026ndash;0.97, I\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;28%), respectively. The evidence also supports in principle the signalling questions number 4.3\u0026ndash;4.5 in the domain \u0026ldquo;bias in measurement of the outcome; that is, in support for blinded assessment of susceptive outcomes and double-blinding (23% overestimation of trials with inadequate or unclear double-blinding: ROR 0.77, 95% CI: 0.61\u0026ndash;0.93). However, \u0026ldquo;blinding\u0026rdquo; was not explicitly included in formulating the tool\u0026rsquo;s questions. The systematic review did not identify empirical evidence supporting 3 out of the tool\u0026rsquo;s 5 bias domains (including 17 out of its total 22 signalling questions), namely \u0026ldquo;deviation from intended intervention\u0026rdquo;, \u0026ldquo;missing outcome data\u0026rdquo; and \u0026ldquo;selective reporting\u0026rdquo; [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eBesides its poor inter-rater reliability (IRR) and limited evidence base, its complex and demanding application may be the reason why a low adherence to the RoB 2 tool among Cochrane protocols and systematic reviews was observed and why most Cochrane reviewers themselves choose not to use the tool [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. The complexity of applying the RoB 2 tool stands in contrast to the steadily increasing volume of clinical intervention trials worldwide [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] and thus the subsequent need arises for more timely, less complicated, yet effective and reliable trial appraisal methods.\u003c/p\u003e \u003cp\u003eIn addition to the above, the RoB 2 tool assigns an overall \u0026ldquo;low-bias risk\u0026rdquo; status to trials that have been judged to be of \u0026ldquo;low-bias risk\u0026rdquo; in all of its five single bias domains, even though any methodological trial error that lies outside its five domains may completely invalidate the reported trial results.\u003c/p\u003e \u003cp\u003eAgainst this background, the Composite Quality Score (CQS) has been developed as a possible alternative for the appraisal of prospective controlled clinical therapy trials in systematic reviews.\u003c/p\u003e"},{"header":"The Composite Quality Score (Cqs)","content":"\u003cp\u003eThe development of the CQS as a possible trial appraisal tool started in 2019 with the epistemic consideration that regardless of how many trial appraisal criteria a trial has fully complied with, the judgment of such a trial as being of \u0026ldquo;low-bias risk\u0026rdquo; cannot be justified but a single methodological error may render a trial to actually be of \u0026ldquo;high-bias risk\u0026rdquo;. Accordingly, three evidence-based criteria describing trial design requirements (random allocation sequence generation, allocation sequence concealment, minimum sample size) that are essential (albeit not sufficient) for the trial results to reflect therapeutic truth were established. The criteria were intentionally worded in a simplified and least restrictive manner, based on the rationale that the lower the stringency of an appraisal criterion, the higher the certainty of \u0026ldquo;high-bias risk\u0026rdquo; when such criterion is not met [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eTo explore the applicability of the novel CQS approach for trial appraisal, it was applied in the field of restorative dentistry. Based on a systematic literature search, 683 prospective clinical controlled trial reports were identified out of a total of 14 694 citations. Of these, the bias risk of 99.7% of trials could be appraised and identified as high, despite the low stringency of the applied criteria [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. In 2021, the inter-rater reliability of the CQS approach was investigated for the first time. The results showed a high inter-rater reliability: Brennan-Prediger coefficient (BPC) of 0.95; 95% CI: (0.87\u0026ndash;1.00), which compared favourably to that of the first RoB tool version, which ranged from BPC \u0026minus;\u0026thinsp;0.07; 95% CI: -0.42, \u0026ndash; 0.28 and 0.34; 95% CI: -0.05\u0026ndash;0.73. Most of the differences between the RoB and the CQS were statistically significant (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) in favour of the CQS [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe findings of all three preliminary investigations [\u003cspan additionalcitationids=\"CR9\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] were summarised and presented as the first version of the CQS (CQS-1) in 2023 [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn order to extend the applicability and evidence base of the CQS-1 for all fields of clinical therapy, a systematic review of meta-epidemiological studies was conducted. Based on its results, one new criterion concerning double-blinding was added, and the original criteria concerning random allocation sequence generation and minimum sample size were amended [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Consequently, all four criteria of the resulting new CQS version (CQS-2) were based on meta-epidemiological evidence, indicating statistically significant (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) over- or underestimation of the true therapeutic effect estimate for trials where such criteria were not met. This systematic review was partially updated to improve the wording and applicability of the criterion concerning the appraisal of allocation concealment, resulting in a further refined CQS version (CQS-2B) [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFollowing the extension of the original CQS-1, the CQS-2 was assessed to establish whether its extended format extension would have negatively affected the high inter-rater reliability of the first version. The results of this study showed that CQS extension had no negative effect and that the CQS-2 version was also associated with very high inter-rater reliability (BPC 1.00; 95% CI: 0.94\u0026ndash;1.00) and did not statistically differ significantly (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05) from that of the CQS-1 (BPC 0.85; 95% CI: 0.64 \u0026minus;\u0026thinsp;1.00) [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe result of this assessment led to the development of a CQS version (CQS-2B) that included four criteria related to the random allocation to treatment groups, concealment of such allocation, double-blinding and sample size minimum (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eCQS-2B appraisal criteria\u003c/b\u003e\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriterion I\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026lsquo;Randomisation\u0026rsquo; for allocation to treatment groups is in some form reported in the text\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriterion II\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAny assurance that the patient allocation to treatment groups according to the random sequence was applied by an independent agent or agency, not otherwise involved in the trial, is in some form reported in the text\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriterion III\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDouble-blinding or the blinding of at least two out of the three groups: trial participants, trial personnel and trial outcome assessors in some form reported in the text\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCriterion IV\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eThe sample size of any particular treatment group reported in the trial is not less than N\u0026thinsp;=\u0026thinsp;100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIts application comprised: (i) binary trial report rating per appraisal criterion (Scores: 0\u0026thinsp;=\u0026thinsp;No/invalid/falsified, 1\u0026thinsp;=\u0026thinsp;Yes/corroborated); (ii) multiplication of all criterion scores to an overall appraisal score, (iii) identification of invalid/falsified trial reports based on a zero overall appraisal score.\u003c/p\u003e \u003cp\u003eDuring its application, several corroboration (C-) levels are recognised. These levels indicate the number of consecutive criteria that a trial has complied with (e.g. level C3 indicates compliance with Criterion I - III; level C4 indicates compliance with Criterion I - IV, etc.). A corroboration level for a particular trial is reached before one criterion is rated with a 0-score or when all criteria are rated with a 1-score, for example Corroboration level C3: Criterion I\u0026ndash;III\u0026thinsp;=\u0026thinsp;1-score, Criterion IV\u0026thinsp;=\u0026thinsp;0-score; Corroboration level C4: All criteria\u0026thinsp;=\u0026thinsp;1-score. After a criterion has been rated with a 0-score, the C-level of a trial remains the same even if a following criterion is rated with a 1-score, for example Corroboration level C1: Criterion I\u0026thinsp;=\u0026thinsp;1-score, Criterion II\u0026thinsp;=\u0026thinsp;0-score, Criterion III\u0026thinsp;=\u0026thinsp;1-score. No certainty of \u0026ldquo;low-bias risk\u0026rdquo; is ascribed to an overall 1-score appraisal result, which indicates only that during the appraisal process no evidence for high-bias risk could be established so far.\u003c/p\u003e \u003cp\u003eIt was explored whether systematic review conclusions originally based on Cochrane\u0026rsquo;s second version of its Risk of Bias tool (RoB 2) would differ when the CQS-2B is used for trial appraisal instead. The results of this study provided justification for the testable hypothesis that trial appraisal using the CQS-2B provides more conservative conclusions based on similar data than trial appraisal using Cochrane\u0026rsquo;s RoB 2 tool [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e"},{"header":"CQS Limitations","content":"\u003cp\u003eDespite its high epistemic rigour, evidence base and inter-rater reliability, the current version of the CQS has certain limitations that need to be taken into account when considering its application in systematic reviews of prospective controlled clinical therapy trials. These include the following items.\u003c/p\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eVerification of low-bias risk\u003c/h2\u003e \u003cp\u003eAs per the design, the CQS-2B is not a tool for verifying low-bias risk. Trials that are awarded 1-scores for all four appraisal criteria may still have serious flaws in other aspects of their methodology. This is a fully intended aspect of the CQS approach in line with the epistemic principles of deductive falsification [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eOne example is the clinical trial by Sitthisettapong et al. (2012) concerning the clinical effect of 10% w/v calcium phosphopeptide-amorphous calcium phosphate (CPP-ACP) paste for one year when added to regular toothbrushing with fluoridated toothpaste to prevent dental caries in pre-school children [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. The trial reports to be \u0026ldquo;randomised\u0026rdquo; (Criterion I\u0026thinsp;=\u0026thinsp;1-score) with the allocation of experimental or control group were determined by an assistant who was not involved in the clinical aspects of the study (Criterion II\u0026thinsp;=\u0026thinsp;1-score), \u0026ldquo;double-blind\u0026rdquo; (Criterion III\u0026thinsp;=\u0026thinsp;1-score) and included patients of which \u0026ldquo;150 were assigned to the experimental condition and 146 to the control condition\u0026rdquo; (Criterion IV\u0026thinsp;=\u0026thinsp;1-score).\u003c/p\u003e \u003cp\u003eHowever, subsequent further in-depth appraisal reveals that patient allocation was neither random nor concealed but alternated with patients \u0026ldquo;with odd ID numbers assigned to the experimental treatment, and those with even ID numbers to the control treatment\u0026rdquo;. This form of alternation may cause the allocation of patients to either group to be visible, which renders the study open to a high risk of selection bias. Furthermore, the important baseline variable \u0026ldquo;falling asleep with a bottle\u0026rdquo; was statistically significantly more distributed among the experimental group (p\u0026thinsp;=\u0026thinsp;0.008). If randomisation had been effective, then all baseline variables should have been equally distributed between the groups.\u003c/p\u003e \u003cp\u003eThis example shows that corroborated trials according to the CQS-2B cannot be considered as low-bias risk. Instead, corroborated trials may be considered eligible for further in-depth appraisal as shown in the example above. Such appraisal may not be confined within the criteria of any specific trial appraisal tool but may need to be tailored specifically to the particularities of each trial. In contrast, a further in-depth appraisal may not be warranted for trials that have been already falsified at C1\u0026ndash;4 levels since the appraisal process has already provided sufficient reason to consider such trials as of high-bias risk. In that way, the CQS-2B may contribute to a more timely and efficient trial appraisal, particularly of a large volume of prospective controlled clinical therapy trials [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eThe discrepancy between not reported versus actual trial characteristics\u003c/h2\u003e \u003cp\u003eNot all actual trial characteristics that were performed during the conduct of a trial, required by the CQS criteria, may have been reported by trial authors. For example, trial randomisation may have included central allocation, but this was not made explicit in the published trial report. Trial appraisal using the CQS-2B would thus have assigned an erroneous 0\u0026ndash;score to its Criterion II.\u003c/p\u003e \u003cp\u003eContacting trial authors during a systematic review of trials may appear to be a possible solution but can be affected by recall bias, the fact that such author responses have not been part of the original peer-review process and the reduction of possible reproducibility of the systematic review results at a later stage. Trial appraisal remains by its nature qualitative text analysis only. This is valid for any trial appraisal tool, such as the CQS-2B or Cochrane\u0026rsquo;s RoB 2. The consequence is false-negative trial appraisal results that may not be remedied.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eThe discrepancy between not performed trial methods and systematic error effect\u003c/h2\u003e \u003cp\u003eThe lack of essential trial characteristics may not always lead to systematic error. This limitation is also valid for any trial appraisal tool. Trial appraisal based on qualitative text analysis can assume whether a trial is susceptible to systematic error only when essential trial characteristics are missing. Quantitative tests that can be applied during the systematic review of clinical trials may solve this problem. Currently, there are only a few tests available for this purpose, such as the highly accurate Berger-Exner test for 3rd -order selection bias [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. However, the currently existing tools do not yet cover all bias domains and thus do not replace trial appraisal based on qualitative text analysis. Therefore, the CQS-2B can establish only a \u0026ldquo;high-bias risk\u0026rdquo; of a trial but cannot actually provide proof that a trial result is biased. For this reason and in agreement with guidelines for the RoB 2 tool [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e], it is strongly recommended that systematic reviews with the CQS-2B should not exclude high-bias risk trial data but always include stratification by overall bias risk for trial outcomes or endpoints during meta-analysis.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe CQS has been developed based on high epistemic rigour, meta-epidemiological evidence and inter-rater reliability. However, like all trial appraisal tools, it has limitations that need to be considered during its application in systematic reviews of prospective controlled clinical therapy trials.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFinancial Disclosure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors received no specific funding for this work.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data is made fully available without restriction as part of the preprint.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that no competing interests exist\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eSterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366: l4898.\u003c/li\u003e\n\u003cli\u003ePage MJ, Higgins JP, Clayton G, Sterne JA, Hr\u0026oacute;bjartsson A, Savović J. Empirical evidence of study design biases in randomized trials: systematic review of meta-epidemiological studies. PLoS One. 2016; 11(7):e0159267. \u003c/li\u003e\n\u003cli\u003ePage MJ, Higgins JPT. Rethinking the assessment of risk of bias due to selective reporting: a cross-sectional study. Syst Rev 2016;5:108.\u003c/li\u003e\n\u003cli\u003eMinozzi S, Cinquini M, Gianola S, Gonzalez-Lorenzo M, Banzi R. The revised Cochrane risk of bias tool for randomized trials (RoB 2) showed low interrater reliability and challenges in its application. J Clin Epidemiol. 2020; 126:37-44. \u003c/li\u003e\n\u003cli\u003eMinozzi S, Dwan K, Borrelli F, Filippini G. Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022 Jan;141:99-105. doi: 10.1016/j.jclinepi.2021.09.021. Epub 2021 Sep 16. PMID: 34537386. \u003c/li\u003e\n\u003cli\u003eMartimbianco ALC, S\u0026aacute; KMM, Santos GM, Santos EM, Pacheco RL, Riera R. Most Cochrane systematic reviews and protocols did not adhere to the Cochrane\u0026apos;s risk of bias 2.0 tool. Rev Assoc Med Bras (1992). 2023 Feb 20;69(3):469-472. doi: 10.1590/1806-9282.20221593. PMID: 36820779; PMCID: PMC10004297. \u003c/li\u003e\n\u003cli\u003eWorld Health Organization. Number of clinical trial registrations by location, disease, phase of development, age and sex of trial participants (1999-2021). Website: https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-trial-registrations-by-year-location-disease-and-phase-of-development, 2022 (accessed: November 13, 2022). \u003c/li\u003e\n\u003cli\u003eMickenautsch S. Is the Deductive Falsification Approach a Better Basis for Clinical Trial Appraisal? Rev Recent Clin Trials. 2019; 14:224-228.\u003c/li\u003e\n\u003cli\u003eMickenautsch S. Are Most of the Published Clinical Trial Results in Restorative Dentistry Invalid? An Empirical Investigation. Rev Recent Clin Trials. 2020;15(2):122-130. \u003c/li\u003e\n\u003cli\u003eMickenautsch S, Miletić I, Rupf S, Renteria J, G\u0026ouml;stemeyer G. The Composite Quality Score (CQS) as a trial appraisal tool: inter-rater reliability and rating time. Clin Oral Investig. 2021; 25:6015-23.\u003c/li\u003e\n\u003cli\u003eMickenautsch S, Rupf S, Miletić I, Yengopal V. The Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials: rationale and current evidence. Rev Recent Clin Trials. 2023; 18:28-33.\u003c/li\u003e\n\u003cli\u003eMickenautsch S, Rupf S, Miletić I, Yengopal V. Extension of the Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials-A systematic review of meta-epidemiological evidence. PLoS One. 2022; 17:e0279645.\u003c/li\u003e\n\u003cli\u003eMickenautsch S, Yengopal V. Allocation concealment appraisal of clinical therapy trials using the extended Composite Quality Score (CQS-2) \u0026ndash; An empirically based update (Preprint), 14 February 2023, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2582208/v1].\u003c/li\u003e\n\u003cli\u003eMickenautsch S, Rupf S, Miletić I, Str\u0026auml;hle UT, Sturm R, Kimmie-Dhansay F, Vidosusić K, Yengopal V. Inter-rater reliability of the extended Composite Quality Score (CQS-2) \u0026ndash; a pilot study, 23 November 2022, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2297364/v1]\u003c/li\u003e\n\u003cli\u003eMickenautsch S, Rupf S, Yengopal V. Application of the Composite Quality Score (CQS-2B) in systematic reviews of prospective, controlled, clinical therapy trials \u0026ndash; an exploratory study (Preprint), 23 March 2023, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-2718841/v1].\u003c/li\u003e\n\u003cli\u003eSitthisettapong T, Phantumvanit P, Huebner C, Derouen T. Effect of CPP-ACP paste on dental caries in primary teeth: a randomized trial. J Dent Res. 2012 Sep;91(9):847-52. \u003c/li\u003e\n\u003cli\u003eMickenautsch S, Fu B, Gudehithlu S, Berger VW. Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation. BMC Med Res Methodol. 2014 Oct 6;14:114. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"University of the Western Cape","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"composite quality score, clinical trial, trial appraisal, risk of bias tool","lastPublishedDoi":"10.21203/rs.3.rs-2785700/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2785700/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eSystematic reviews of prospective controlled clinical therapy trials are one of the most important sources of information in modern medicine. Besides the systematic search for and statistical pooling of current clinical trial data for a particular type of therapy, systematic reviews also have the task of appraising the quality of trial results. The quality of trial results may be diminished by low internal trial validity, due to systematic error (bias). A high risk of bias may likely cause the reported trial results to be diverted from the actual true therapeutic effect and thus render it unsuitable for clinical guidance. According to the Cochrane Collaboration, the risk of bias in clinical therapy trials should be assessed using its Risk of Bias tool, Version 2 (RoB 2). However, the tool has been established to have poor inter-rater reliability, with a limited empirical evidence base and described as complex and demanding. Against this background, the Composite Quality Score (CQS) has been developed as a possible alternative trial appraisal tool, characterised by high epistemic rigour, empirical evidence base, inter-rater reliability and ease of use. This article presents the current evidence of the CQS and its limitations.\u003c/p\u003e","manuscriptTitle":"The Composite Quality Score for the appraisal of prospective controlled clinical therapy trials in systematic reviews and its limits (Preprint)","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-04-07 04:21:12","doi":"10.21203/rs.3.rs-2785700/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d716492c-dddb-456c-a952-e31cadcb8572","owner":[],"postedDate":"April 7th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":20513818,"name":"Internal Medicine"}],"tags":[],"updatedAt":"2023-08-24T18:37:32+00:00","versionOfRecord":{"articleIdentity":"rs-2785700","link":"https://doi.org/10.3389/fmed.2023.1201951","journal":{"identity":"frontiers-in-medicine","isVorOnly":true,"title":"Frontiers in Medicine"},"publishedOn":"2023-06-28 00:00:00","publishedOnDateReadable":"June 28th, 2023"},"versionCreatedAt":"2023-04-07 04:21:12","video":"","vorDoi":"10.3389/fmed.2023.1201951","vorDoiUrl":"https://doi.org/10.3389/fmed.2023.1201951","workflowStages":[]},"version":"v1","identity":"rs-2785700","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2785700","identity":"rs-2785700","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}