Quality evaluation of endometriosis guidelines using AGREE II

Endometriosis (endometriosis) refers to the occurrence, growth, infiltration, and repeated bleeding of endometrial tissue outside the uterus, which in turn leads to pain, infertility, nodules or masses. It is a common and frequently-occurring disease in women of childbearing age. [ 1 ] Women with endometriosis usually have a series of pelvic abdominal pain symptoms, including dysmenorrhea, difficulty in intercourse, large amount of bleeding during menstruation, non-menstrual pelvic pain, ovulation pain, difficulty in urination and chronic fatigue. [ 2 ] Endometriosis is also associated with infertility. There is a strong correlation between the severity of the disease and the impact on women’s fertility, due to impaired ovarian function, ovarian endometriosis cysts and subclinical pelvic inflammation, which may reduce the quality of oocytes and reduce the endometrial receptivity to implantation. [ 3 ] As a common chronic gynecological disease requiring long-term management, the current guidelines and consensus related to endometriosis have been up to a hundred departments, which brings great difficulties to clinicians “decision-making and patients” informed choice. Therefore, it is very important for both doctors and patients to select high-quality guidelines or consensus as clinical diagnosis and treatment guidance. Clinical practice guidelines (CPG) are designated and issued by professional academic organizations or departments based on the best evidence and combined with clinical practice to help clinicians and patients make clinical decision-making guidance, including guidelines, consensus, recommendation, etc, which can not only improve the diagnosis and treatment effect, but also make full use of medical resources. [ 4 ] In this study, the internationally recognized guideline evaluation tool-appraisal of guidelines for research and evaluation (AGREE II) was used to evaluate the quality of the included CPGs through a comprehensive retrieval of foreign published CPGs specifically for endometriosis, so as to provide further reference for clinicians. [ 5 ]

Conceptualization: Yan Lei. Data curation: Yan Lei, Xin Du. Formal analysis: Yan Lei, Dejun Chen. Funding acquisition: Hongmei Lian. Investigation: Xin Du, Dejun Chen, Yue Gao. Methodology: Yan Lei. Project administration: Yue Gao, Hongmei Lian. Software: Dejun Chen, Hongmei Lian. Supervision: Yue Gao. Validation: Yue Gao. Visualization: Yue Gao. Writing – original draft: Yan Lei. Writing – review & editing: Xin Du, Hongmei Lian.

PubMed database, Embase database, evidence-based medicine CPG database, National Institutes of Health and Clinical Medical Optimization were searched online from December 2012 to December 2020 with search terms of “endometriosis,” “guidelines,” “guidance,” “consensus.” We also searched the websites of guideline development organizations: Guidelines International Network Web site ( http://www.g-i-n.net/ ), National Institute for Health for Health and Care Excellence website ( https://www.nice.org.uk/guidance ), National Guideline Clearinghouse ( https://guidelines.gov/ ), Scottish Intercollegiate Guidelines Network ( http://www.sign.ac.uk/ ), Clinical Practice Guidelines Portal website ( https://www.clinicalguidelines.gov .au/), New Zealand Guidelines Group website ( https://www.health.govt.nz/ ), BCGuidelines website ( http://www.bcguidelines.ca/alphabetical ), AQuMed Database website ( http://www.aezq.de/aezq/publications ). In addition, we searched Google Search Engine and checked the references of all the related guidelines to include more potential guidelines. All guidelines were based on previous published studies, thus no ethical approval and patient consent are required. Inclusion criteria include: CPG for endometriosis published at home and abroad, developed or published by professional academic organizations or departments. Exclusion criteria include: old CPG issued by the same academic organization or department, lecture or expert review, review or research literature, guide interpretation, conference abstracts. Two researchers independently screened literature and extracted data according to inclusion and exclusion criteria and checked them. If disagreement arises, it shall be decided by the third researcher. Extraction of different CPG names (including guidelines, consensus and recommendations), topics, publication years, publication organization, guidelines development methods, evidence level evaluation methods and other basic information. Each included CPG was evaluated by 2 trained personnel for AGREE II. According to the “Instructions for Clinical Guidelines Research and Evaluation System II (Chinese version),” 23 entries in the following 6 fields were evaluated, including scope and purpose (3 entries), stakeholder involvement (3 entries), rigor of development (8 entries), clarity of presentation (3 entries), applicability (4 entries) and editorial independence (2 entries). Finally, a comprehensive evaluation was carried out according to the scores. Two researchers rated each item from 1 (very disagree) to 7 (very agree). Standardized scores in various areas = (actual rating-lowest possible rating)/(highest possible rating - lowest possible rating) × 100%. The higher the standardized score of a field, the more complete the reporting, reflecting the better the method of developing guidelines in the field. Combined with the scores of each field of the guide, the recommended level of the guide is divided into 3 grades, and grade A is recommended, that is, the standardized scores of the 6 fields are ≥60.00%; grade B is recommended after modification and improvement, that is, the standardized score of the field is <60.0%, and the standardized score of ≥3 fields is ≥30.0%; grade C is not recommended, that is, the number of fields with a standardized score of <30.0% in 6 fields was ≥3. Before the formal evaluation, the 2 researchers used AGREE II to perform pre-experimental scoring on the randomly selected 2 guidelines to test the reliability of the results. The scoring results were statistically analyzed by SPSS 20.0 software, and the consistency of the evaluation results was investigated by intra-group correlation coefficient (ICC). When the ICC value is 0.81 to 1.00, [ 6 ] it indicates that the consistency is high and formal experiments can be carried out.

A total of 906 relevant literature were obtained through database retrieval. According to the inclusion and exclusion criteria, 8 CPGs were finally included after excluding duplicate articles (Fig. 1 ). Literature search process and results. Among the 8 CPGs included in this study, 5 guidelines and 3 consensus. Covering 5 countries of 2 continents; published in 2013 to 2018, with 6 first editions and 2 updated editions; 7 of them were conducted evidence quality rating and recommendation intensity evaluation based on the Grading of Recommendations (Assessment, Development and Evaluation, GRADE) method. In addition to the 2016 Portuguese guidelines, the 2017 WES endometriosis classification guidelines, and the 2018 Japanese guidelines, the remaining 5 CPGs are all about the diagnosis, treatment (drugs and surgery), and pain management of endometriosis in the reproductive system. Four articles also cover the treatment of patients with endometriosis and infertility. The 2014ESHRE guideline referred to the primary prevention of endometriosis, while the 2014 German guideline defined the secondary prevention of endometriosis and made recommendations (Table 1 ). General information of CPG. − = no such item, ESHRE = European association for human reproduction and embryology, DGGG = German society for gynecological sciences, GRADE = grading of recommendations, assessment, development and evaluation, JSOG & JSE = Japanese society of obstetrics and gynecology and Japanese society of endometriosis, KSE = Korean society of endometriosis, NICE = National institutes of health and clinical optimization, WES = world endometriosis society. In this study, 2 evaluators used AGREE II to score the pre-experiment of the 2 randomly selected guidelines. The ICC value and 95% confidence interval of the pre-experiment results were 0.92 (0.82–0.97) and 0.91 (0.79–0.96), indicating that the consistency of the evaluation of the 2 researchers was good. The average standardized scores for the scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability and editorial independence of the 8 CPGs included were 1 in grade A (recommendation), 5 in grade B (recommendation after improvement) and 2 in grade C (no recommendation), respectively (Table 2 ). Quality evaluation results of AGREE II domains incorporating CPG. − = no such item, AGREE II = appraisal of guidelines for research and evaluation, CPG = clinical practice guidelines. The formation and formulation of 7 endometriosis CPGs were based on the expert consensus, and their recommendations were finally decided by the expert vote. Although a large number of relevant literature retrieval and analysis were also carried out in the process of guideline formulation, many key issues were not answered due to the lack of high-quality and new research literature. CPG, which relies on detailed literature retrieval, analysis and evidence level assessment, is a guide issued by NICE in 2017. Seven CPG statements used GRADE system as evidence level evaluation method. Six guidelines specify the number of recommendations or declarations recommended. All CPGs issued conflict of interest statements and provided references (Table 3 ). Composition of endometriosis CPG. − = no such item, CPG = clinical practice guidelines.

Endometriosis is one of the common chronic diseases in gynecology. It is currently considered as a hormone-dependent disease, and its incidence accounts for 10% of the chronic diseases in women of childbearing age. [ 15 ] The disease often leads to chronic pelvic pain, infertility and other diseases, which seriously affects the quality of life of patients. [ 16 ] Its early diagnosis is difficult, and the treatment effect is poor. [ 17 , 18 ] Therefore, high quality CPG should be followed in clinical decision making. The AGREE II system is universal and suitable for any disease area in all health care links. It can help health providers to evaluate themselves before they adopt the recommendations recommended by the guidelines. It can make the guideline developers follow a structured strict development method, help policy makers to determine which CPGs are suitable for practical applications, and also help health professionals to improve the ability to strictly evaluate the guidelines. [ 19 ] From the 8 CPGs included in this study, AGREE II evaluation results showed that 1 was A grade recommendation, 5 were B grade recommendation and 2 were C grade recommendation. The average scores in fields 1 (scope and purpose) and 4 (clarity of presentation) were ≥60%, and the average scores in fields 5 (applicability) and 6 (editorial independence) were <30%, indicating that most of the CPGs for endometriosis did not pay enough attention to these 2 fields. Main shortcomings: The average score of field 3 (rigor of development) was 50.5%, because the lack of high-quality and multi-center clinical research data led to the formulation of most CPGs based on clinical experience and consensus of experts; no detailed inclusion and exclusion criteria; there is no clear description of the method for forming recommendations; some CPGs do not even specify the strength and limitations of the evidence. The average score of filed 5 (applicability) was 31.3% because most CPGs did not mention the promotion and impediments to application; economic cost and health budget were not mentioned. The general score of field 6 (editorial independence) is not ideal. Although each CPG has a statement of interest conflict, the wording of most guidelines is not clear about the point that the views of sponsors do not affect the content of the guidelines. In particular, Japan’s CPG has barely elaborated on this field. Overall, the progress of endometriosis CPG in the past decade was limited, especially in the recommendation of diagnosis and treatment strategies. The limitations of this study are as follows. The limitations of research tools: there are no primary and secondary divisions in the 6 fields of AGREE II, which leads to the fact that some CPG levels may not be consistent with the actual. In fact, the quality of GDP cannot be evaluated only by the level, which is also clearly pointed out in the use instructions of AGREE II. Language limitation: This study was limited by language, only English and Portuguese CPG were included, and no further study was conducted on CPG in other languages. AGREE II is only an evaluation system, which is subject to the subjective influence of design developers. It is not necessarily suitable for all regions and countries, and it needs to be refined and updated. In this study, the diagnosis and treatment strategies and methods recommended by the 8 endometriosis CPGs were not significantly different, but in fact, many problems were controversial in the process of CPG production or in the diagnosis and treatment of patients with endometriosis. [ 20 ] The 2017 NICE guide has high scores in 6 fields, and is the only CPG published on the basis of literature retrieval and evidence grading assessment. It provides a detailed description of economic and social costs, promotion and impediments, and provides a detailed comparative analysis of drug treatment costs, which is worthy of promotion and reference.