The reporting quality and spin of randomized controlled trials of endometriosis pain: Methodological study based on CONSORT extension on abstracts

Randomized Controlled Trials (RCTs) are the best design to assess the efficacy and safety of therapeutic interventions in medicine; they are considered the foundation of evidence-based medicine [ 1 ]. The abstract of RCTs provides the reader with a summary account of the study objectives, methods, results, and conclusions. Often without access to full text articles, clinicians and others, only have access to read the abstract of a research report to help guide their clinical decision making. However, inappropriate study methodology with incomplete or inaccurate reporting impedes sufficient understanding of the clinical indications and restricts the reader’s comprehension of the result’s validity [ 2 , 3 ]. These inferences are likely higher among physicians in low- and middle-income countries due to the unavailability of full-text publications. Similarly, patients and the public do not always have access to journal articles. Furthermore, determining whether a full-text publication is worth purchasing is crucial. These factors highlight the importance of reporting quality, clarity, and accuracy during the initial evaluation of RCT abstracts [ 4 ]. To improve the completeness and transparency of reports of RCT abstracts, the Consolidation of the Standards of Reporting Trials for abstracts (CONSORT-A), was developed in 2008 [ 5 ]. It provides the basic information to properly evaluate the validity of trials and the applicability and clinical relevance of trial findings. CONSORT-A includes 16 checklist items describing the spectrum of the research study, including information about the objectives, design, participants, interventions and their effect on primary efficacy outcomes and harms, conclusions, registration name and number, and source of funding [ 6 ]. Endometriosis is a painful, chronic inflammatory disorder. More than 60% of women diagnosed with endometriosis have dysmenorrhea or chronic pelvic pain. RCTs of endometriosis pain have not been critically evaluated [ 7 ]. To enhance the quality of reporting for RCTs in the endometriosis and gynecology literature, and in all intervention studies, researchers are motivated to publish their reports in clear, transparent, and unambiguous language. They must provide a precise and thorough description of participants, who were recruited, excluded, lost to follow up or did not complete the study. Because failing to declare essential data may influence the interpretation of results [ 8 ]. Furthermore, interpretation bias, often referred to as spin, is defined as misleading reporting of findings of a scientific study to highlight that the intervention is beneficial, despite the ignorable differences of the primary endpoint, or to confound the reader from statistically non-significant results. The tool for spin estimation in RCTs identifies reporting practices that constitute an intentional or unintentional effort to spin the results [ 9 ]. Although multiple RCTs have been published on the treatment of endometriosis pelvic pain, the adherence to the CONSORT guidelines and the prevalence of spin has not been evaluated. Due to the prevalence of pain among endometriosis patients and the emphasis of guideline on conservative drug therapy instead of surgical interventions [ 10 ] as well as based on the above two statements the present study was designed to critically i) Assess the reporting quality of recently published RCT abstracts in endometriosis pelvic pain; (ii) Identify factors associated with reporting quality; (iii) Investigate the prevalence and characteristics of spin in these abstracts.

Study Characteristics of the Selected Reports: From 239 retrieved articles in databases, 47 clinical trials met the eligibility criteria and were included in the study. The study flow chart is shown in Fig 1 . Among these, most of articles were indexed in PubMed (45; 93.8%) and Scopus (45, 93.8%). Additionally, 43 (89.6%), 41 (85.4%) and 4 (8.3%) were indexed in EMBASE, ISI and ESCI, respectively. The article and the journal metrics for each study are shown in supplementary file ( S1 File ). Assessing the characteristics of article abstracts, the average word count was 272.34 (±106.05) (min: 99, max: 734, median: 251, IQR: 78) and more than one-third (38.3%) of them had between 200 and 250 words. The mean of sample size was 227.59(±378.98) (min: 9, max: 1689, median: 83, IQR: 197). The mean Scopus citations was 41.11(±63.25) (min: 0, max: 321, median: 16). Taking into account the year of publication, document type, and disciplines the mean of FWCI is 3.24 (±4.05) (min: 0.14, max: 21.33, median: 1.70). Furthermore, we found that most articles described single center (n = 30, 63.8%) studies and were published in specific gynecological journals (n = 28, 59.6%). Almost half of the geographical area of the articles were in Europe and Asia (34.0% and 38.3%, respectively) ( Table 1 ). Assessing the journal metrics, we found than the mean H-index of the journals was 167.20 (±198.08). Evaluating the FWCI values according to the QoR (CONSORT values) shows a statistically significant correlation (r = 0.45, bootstrapping p-value: 0.003). As seen, the more the QoR the more is the FWCI. Assessing the quality of the abstracts, we saw the total score for the quality varied between 5 and 15 (mean: 9.59, SD: 3.03, median: 9, IQR: 5). Furthermore, evaluating each item of CONSORT-A checklist, in titles of 26 (55.3%) papers, identified the study as randomized. In most of abstracts the eligibility criteria for participants was described. The detail of CONSORT items is shown in Table 2 . We compared the CONSORT-A total scores (QoR) for each indexing databases. Evaluating the quality of papers that indexed in Scopus, we observed a statistically difference between the scores of papers indexed in Scopus and papers that didn’t index in Scopus (QoR = 9.77±3.03 vs. 7.00±1.73; bootstrapping p-value: 0.005; Bca 95%CI for mean difference (MD): 1.11, 5.04). Comparing the QoR between other indexing databases we did not see any significant differences (bootstrapping p-value>0.05). Furthermore, assessing the QoR in different IF quartiles we saw a statistically significant difference between them (QoR for Q1, Q2, Q3, and Q4 respectively were 11.75±2.53, 7.83±1.17, 8.36±2.73, and 6.67±0.58; bootstrapping p-value:<0.001) in which the QoR for articles that published in Q1 journals were statistically more than the other quartiles. Evaluating the QoR in different years and between IF quartiles, we saw that the QoR was increased for all quartiles except for Q4 ( Fig 2 ). Also, assessing the QoR based on being open access or not, we saw that the QoR in open access journals were significantly more (QoR in open access journals: 10.52±2.78, QoR in not open access journals: 8.11±2.89, p-value:0.007). A multivariable linear regression model was used to assess the impact of various independent variables on the Quality of Reporting (QoR) of Randomized Controlled Trial (RCT) abstracts. The independent variables included word count, indexing in databases (PubMed, Scopus, ISI), Field-Weighted Citation Impact (FWCI), publication in a gynecological journal, open access status, Impact Factor (IF), Cite Score, and H-index. The goal was to determine significant predictors of reporting quality in RCT abstracts. Our findings revealed valuable insights into the relationship between these independent variables and the QoR of abstracts. As shown in Table 3 , we observed that an increase of one unit in the word count of the abstract was associated with a significant increase in the mean QoR (beta = 0.015, p-value = 0.005). Furthermore, our analysis indicated that abstracts published in open-access journals exhibited a mean QoR approximately 2.02 units higher than those published in other types of journals (beta = 2.023, p-value = 0.023). Evaluating spin within each included paper, we found that 21 (44.7%) papers had statistically non-significant results. From these studies, 15(71.4%) had spin in both results and conclusion sections. The mean total score of spin intensity was 2.67±1.46 than wasn’t statistically different between studies with and without funding (mean difference (MD) = 0.17, P-value = 0.82). The prevalence of spin among all included articles was 38.29 (18 from 47 articles at least had spin in one of results or conclusion section). The trend of changing the spin during 2010–2023 is presented in Fig 3 . As seen in Fig 3 , the spin intensity was increasing from 2010 to 2023. Assessing the spin in subgroups of impact factor, we saw that in articles that had been published in journals with IF more than 3, the trend of spin is decreasing from 2010 to 2023 while in the other IF categories, the spin is increasing. No association was found with type of study area (obstetrics or gynecology) (p-value = 0.247), number of authors (p-value = 0.584), and being open access or not (p-value = 0.719) with spin. To evaluate the factors influencing spin in the reporting of research findings, logistic regression analysis was conducted. The independent variables considered in the analysis included the word count of the abstract, indexing status in PubMed (yes/no), indexing status in Scopus (yes/no), indexing status in ISI (yes/no), FWCI, designation as a gynecological journal (yes/no), publication in an open-access journal (yes/no), IF, Cite Score, and H-index. These variables were entered into the univariable linear regression model to assess their individual relationships with the presence of spin. However, the analysis did not reveal any statistically significant relationships between the independent variables and the presence of spin in the abstracts. Additionally, the model encountered convergence issues specifically for the variables related to indexing in PubMed, Scopus, and ISI, indicating potential limitations in the modeling approach for these factors ( Table 4 ).

This study adds to growing evidence that RCT abstracts in the gynecological literature may benefit from improvements, especially in smaller gynecologic subspecialties. Our results also suggest that spin is present in a number of RCTs with non-significant primary endpoints in the obstetrics and gynecology literature. The existence of such interpretive biases is of concern to the scientific community, as spins can influence decision-making in research, clinical, and healthcare systems. This highlights the need to advance strategies to counteract the effects of spin and promote better quality reporting in the scientific papers.

In this study, we searched all abstracts published in PubMed and Scopus databases. Available full-text articles that published from January 1st, 2010 to December 1st, 2023 were chosen. We chose this time horizon to provide researchers with enough time to implement CONSORT for abstracts since its publication in 2008. Study design: We included abstracts of primary, parallel or crossover of two-arm RCTs. We had no language restriction. Participants: Adult patients (18–45 years old) with endometriosis associated pain. We include studies if the endometriosis were diagnosed with laparoscopy or surgery. Intervention: We consider any interventions that affect the endometriosis associated pelvic pain. Outcome: Studies with the primary or secondary outcome of endometriosis associated pelvic pain. We included only studies with a statistical non-significant difference between group results for pain outcomes. We excluded the Studies with the following criteria: Animal studies non-randomized trials, protocol studies, observational or cohort studies, review articles, interim or secondary data analyses of RCTs, or short surveys. Letters, editorials, book chapters, thesis, conference papers, or news reports. Two team members (H.Sh and PM) developed search strategy and selected the articles based on the following search strategy: PubMed : ("Dysmenorrhea"[Title/Abstract] OR "painful menstruation"[Title/Abstract] OR "menstruation painful"[Title/Abstract] OR "pain menstrual"[Title/Abstract] OR ("Pain"[Title/Abstract] OR "pelvic pain"[Title/Abstract])) AND ("Endometriosis"[Title/Abstract] OR "Endometrioma"[Title/Abstract]) AND ("clinical trial"[Title/Abstract] OR "intervention study"[Title/Abstract]) AND 2010/01/01:2023/12/01[Date—Publication]. Scopus: (TITLE (dysmenorrhea) OR TITLE (“painful menstruation”) OR TITLE (“menstruation painful”) OR TITLE (“pain menstrual”) OR TITLE (pain) OR TITLE (“pelvic pain”)) AND (TITLE (endometriosis) OR TITLE (endometrioma)) AND (TITLE-ABS-KEY (“clinical trial”) OR TITLE-ABS-KEY (“intervention study”)) AND ((PUBYEAR > 2010 AND PUBYEAR < 2024) OR PUBDATETXT ( December 2023) ) In order to identify the articles, two researchers (PM and H.Sh) screened the retrieved articles. Then the duplicates were excluded. From each article, the required information (including number of authors, word count of abstracts, number of centers (single vs. multicenter), publication year, number of authors (less than 4, 4 to 7 and more than 7), and continent of origin (Asia, Europe, North America, South America and Australia) was retrieved. Furthermore, the journal name, indexing information of the journal (whether indexing in PubMed, Scopus, Embase, or DOAJ), being open access or not subject category (Specific Journal: Obstetrics & Gynecology vs. non-specific Journal), as well as the Journal metrics (impact factor (IF), IF Quartiles (Q1, Q2, Q3 and, Q4), cite score, H-index, frequency of citations in Scopus, and Field-Weighted Citation Impact (FWCI) were extracted ( S1 File ). FWCI indicates the number of citations of an article compared to similar articles. Over three years, this is the ratio of a document’s citations to the average number of citations for all similar documents. A FWCI greater than 1.00 indicates that the document’s citations are higher than expected from the average citations of similar articles. We used the extension of CONSORT statement for abstracts for assessing the reporting quality of included abstracts. If each of the items was reported adequately, we gave it a score of ‘1’, and if the description was insufficient or incomplete, we gave it ‘0’. Therefore, the overall quality of reporting score (QoR; range from 0 to 16) was calculated for each included abstract. We also reported the 11 related sub-items of CONSORT-A [ 5 ]. Spin was defined as “using specified reporting procedure, of any motivation, in order to emphasize that the medication is useful, in spite of a statistically non-significant result for the primary outcome, or to confuse the reader from statistically non-significant results,” as suggested by Boutron et al. [ 11 ]. We evaluated the spin respectively in the results and conclusions section of the abstracts. According to prior literature the spin types with the following items were assessed: The focus of the results is on within-group statistically significant analysis; The concentration of the results is on the significance of secondary endpoints; Concentration of results relates only to the primary endpoint with statistical significance, although there are multiple primary endpoints. Claims of equivalence/non-inferiority/equivalence/similarity for statistically non-significant results Claims to receive benefits with a statistically non-significant result for the primary endpoint Focus only on statistically significant results (i.e., secondary endpoints, within-group analysis, subgroup analysis, analysis of a modified population) Identify primary endpoints that are not statistically significant but recommend the use of an experimental treatment If there are multiple primary outcomes, focus only on the statistically significant primary outcome If there are multiple primary outcome time points, focus only on statistically significant time points. If the answer to any of the above questions is positive, that question is scored 1 point, otherwise it is scored 0. In each of the results and conclusion sections, we say that the section has a spin if at least one of the questions is answered affirmatively. Furthermore, in each of the results and conclusions sections, we considered the total points obtained to be equal to the spin intensity. Spin was assessed by P.M. and H.Sh. independently. In the cases of disagreement, consensus was achieved by discussion. The primary outcome must be clearly defined either in the abstract or in the full text. In the studies that the primary outcome was not clearly mentioned, we considered the declared outcome in the sample size calculations as the primary outcome. If not, we inferred a primary outcome based only on the expressed objectives of the study. In the case of secondary outcome or not identifying the primary outcome, we excluded the study from spin assessment. The main outcome of the study was the quality of the abstract reporting assessed using CONSORT-A. The overall reporting score is derived from the sum of items assessed in accordance with CONSORT-A guidelines. We also assessed the prevalence of spin in any section of the abstract as the secondary end point. Descriptive analysis was shown as frequency and percentages for categorical variables and mean, standard deviation (SD), median and, inter-quartile range (IQR) for continues variables. Assessing the associations between the study characteristics and the spin we used t-test, chi-square, Pearson correlation and, one-way ANOVA (with Bonferroni post hoc test). As the frequency of included articles was small or the distribution of the parameters was abnormal, we used the bootstrapping methods to assess the differences between groups. Bootstrapping is a with replacement sampling method. This method estimates the variations in a statistic by re-sample many times from observed data. We perform 1000 with replacement samples. Bias-corrected and accelerated (Bca) 95% confidence interval was computed. In this study, we utilized a multivariable linear regression model to examine the factors influencing the Quality of Reporting (QoR) in the abstracts of Randomized Controlled Trials (RCTs) focusing on endometriosis pelvic pain. Understanding the determinants of reporting quality in scientific literature is essential for assessing the robustness of research findings. The independent variables included in our analysis were carefully selected to capture various aspects that could potentially impact the QoR of RCT abstracts. These variables encompassed characteristics such as the word count of the abstract, indexing in prominent databases like PubMed, Scopus, and ISI, Field-Weighted Citation Impact (FWCI), publication in a gynecological journal, open access status, Impact Factor (IF), Cite Score, and H-index. Furthermore, to evaluate the strength and direction of the relationships between these independent variables and the QoR, we employed standardized coefficients, also known as Beta coefficients, in our multivariable linear regression model. Beta coefficients, or standardized coefficients, serve as a method to assess the magnitude and orientation of the association among variables that are denoted in disparate units or scales. The process of standardizing these coefficients normalizes the variables to a uniform scale, thereby simplifying the process of comparison. Assessing the factors influencing the spin existence, we used logistic regression. We consider spin existence as dependent variable and the word count of the abstract, the indexing information of the journal (whether indexing in PubMed, Scopus, and ISI), designation as a gynecological journal (yes/no), publication in an open-access journal (yes/no), and the Journal metrics (impact factor (IF), cite score, FWCI, and H-index) as the independent factors. We used Statistical Package for Social Sciences (SPSS) version 23 to analyze the data. P-value less than 0.05 is considered as statistically significant.

(DOCX) (DOCX) (SAV)