Developing a set of patient-centered outcomes for routine use in endometriosis: An international Delphi study

A.N.‐B., A.T., P.S., J.B., A.F.: Steering committee. A.N.‐B., A.F.: Selection of expert panel and preparation of Delphi questionnaire. A.N.‐B.: Data acquisition. A.N.‐B., E.L.R.: Statistical analysis. A.N.‐B., E.L.R., A.F.: Interpretation of the data. A.N.‐B., E.L.R., P.S., P.V., A.F.: Drafting the article. A.F.: Study supervision.

One Clinic provided funding for the organization of the Delphi process. Neither patients nor experts were remunerated for their participation. This research/work/investigator was supported (in part) by the Division of Intramural Research of the NIH, NINDS. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

The preselected 47 PROMs and 30 CROMs constituted the starting set for Round 1 (Table  1 ). The distribution of the PROMS in the 11 subgroups was as follows: (i) Symptomatic Impact ( n  = 4); (ii) Pain ( n  = 7); (iii) Fatigue ( n  = 2); (iv) Sexual Activity ( n  = 5); (v) Gastrointestinal Symptoms ( n  = 4); (vi) Urinary Symptoms ( n  = 2); (vii) Psychological Impact ( n  = 6); (viii) Work Productivity ( n  = 2); (ix) Endometriosis Quality of Life ( n  = 2); (x) General Quality of Life ( n  = 6); (xi) Miscellaneous ( n  = 7). The distribution of the CROMs in the four subgroups is as follows groups: (i) Clinical Indicators ( n  = 15); (ii) Imaging Indicators ( n  = 6); (iii) Biological Indicators ( n  = 4); (iv) Surgical Indicators ( n  = 5). Summary of all the items analyzed, including those for which no consensus was reached. Overall, 166 experts and patient representatives were contacted to participate in the study and 67 agreed to participate. Table  2 summarizes the characteristics of those who agreed to participate. Characteristics of participants in the different rounds of the Delphi study. Note : Bold values show the total of each section. For example, total number of participants is the addition of female and male participant. The total number of participants in Europe is the addition of all participants of each of the European countries. Fifty‐one (76%) of the 67 experts who agreed to participate, participated in Round 1 (Figure  1 ). Forty‐six outcome measures were discarded after this round: 14 CROMs and 32 PROMs (Table  1 ). Open text boxes supplemented the Likert scale, providing some examples of the rationale for those CROMs that were discarded. The Biberoglu & Behrman scale was discarded because “some elements of the clinical examinations are only possible by highly trained professionals and specialists; it cannot be offered in a situation of first level screening”. The revised American Society of Reproductive Medicine (rASRM) surgical classification which is widely used to stage endometriosis was also discarded. The reasons given included “this surgical classification is not relevant for patients who are not undergoing surgery, and not all endometriosis patients undergo surgery” as well as the fact that “the classification is poor and does not allow the evaluation of deeply infiltrative endometriosis (DIE)”. Some of the PROMs that were discarded included a series of tools measuring pain (Pain Catastrophizing Scale, Pain Vigilance and Awareness Questionnaire, Pain Anxiety Symptom Scale, PAINDETECT) as they were viewed as “not fully validated and accepted” and that “they are more suited for research than routine clinical practice settings”. Furthermore, most PROMs measuring the psychological impact of endometriosis were also discarded (Beck Depression Inventory, Patient Health Questionnaire, Spielberg State Trait Anxiety Inventory, General Anxiety Disorder, Beck Anxiety Inventory). In addition, all suggested PROMs measuring fatigue were discarded because they were considered to be “more for research than clinical practice” and because “they were not specific to endometriosis”. The same occurred with the indicators measuring the impact of urinary symptoms: both indicators (the Urinary Symptom Profile and International Consultation on Incontinence Questionnaire Female Lower Urinary Tract Symptoms Modules) were considered to be used “mainly for research” and “only important for those patients who undergo bladder surgery”. Finally, most of the PROMs measuring quality of life (SF‐12, EQ‐5D, PROMIS Global Health, WHOQoL BREF, Nottingham Health Profile) were discarded because they were “mainly suitable for research purposes”. Two indicators that had not been included in the initial list were added by the participants: the Antral Follicle Count measured by transvaginal ultrasound was added since it was considered “a better indicator for the evaluation of ovarian failure”; and the Raising Awareness Tool of Endometriosis (RATE) questionnaire (developed by the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG)) was also added, since “it is a more practical alternative tool to measure sexual activity”. Forty‐four (86%) of the 51 experts who participated in Round 1 provided responses in Round 2 (Figure  1 ). In this round, 14 outcome measures were discarded (Table  1 ) – seven CROMs (the Andersch and Milsom scale, magnetic resonance imaging (MRI)‐based Enzian score, Ultrasound‐based endometriosis staging system, anti‐Müllerian hormone serum level, antral follicle count measured by transvaginal ultrasound, Enzian classification, and the Endometriosis Fertility Index); and seven PROMs (the Female Sexual Function Index, Sexual Activity Questionnaire, Gastrointestinal Quality of Life Index, Hospital Anxiety and Depression Scale, SF‐36, FertiQoL and the RATE questionnaire). Nineteen indicators constituting the final list were discussed in the expert meetings. The CROM (antral follicle count measured by transvaginal ultrasound) and PROM (RATE questionnaire) that were proposed for inclusion during Round 1 were both discarded in Round 2. Some of the CROMs that were discarded are used in many studies and the rationale for discarding them is worth highlighting. For example, the clinical indicator Andersch and Milsom was discarded because it is “an indicator based on unconfirmed assumptions” and “not the most useful scale”. The imaging indicator Ultrasound‐based Endometriosis Staging System (UBESS) was discarded because “being very operator dependent and highly trained sonographers are needed, a resource that is not always available”. Two surgical indicators were discarded: the Endometriosis Fertility Index (EFI) and the Enzian classification. The EFI was discarded because “it includes measurements that are not related to endometriosis” and “mainly measures infertility that only affects a certain percentage of patients with endometriosis”. The Enzian classification was discarded because “it is mainly used in German speaking countries and has poor external validation”. All PROMs measuring the impact of endometriosis in sexual activity, gastrointestinal symptoms and psychological impact were discarded because the proposed PROMs were not specific to endometriosis. Finally, the SF‐36, despite its use in clinical research studies in endometriosis, was discarded due to “its length, complexity and lack of specificity in endometriosis”. Such an instrument was deemed “suitable for research studies but not for routine care”. Of the 44 experts who participated in Round 2, 21 (47%) participated in one of the expert discussion sessions. During these discussions, an overview of the Round 2 results was presented and indicators that had not been selected but which had a percentage of agreement close to the threshold were discussed, as well as other indicators that participants raised for clarification or inclusion. For example, Enzian had a median score of 7 in Round 2 but the percentage of agreement of 63% was lower than the required threshold of 65%. The Enzian classification included in the Delphi phase of this study was the original version from 2003. 14 However, during the final discussion the experts suggested the recently updated version, the #Enzian classification, which can be adopted by gynecologists, surgeons, sonographers and radiologists using the same principles, independently of whether the evaluation is surgical, uses ultrasounds or MRI. 15 During the robust discussion comparing the #Enzian classification with the surgical classification of DIE, 16 the latter was discarded despite reaching a median score of 7 and a percentage agreement of 69%, due to its close resemblance to #Enzian, and the fact that the expert group considered #Enzian to be more versatile. A decision was made to create a list of complementary outcomes which were considered unsuitable for routine care in endometriosis, yet covered important dimensions. For example, all the PROMs measuring sexual activity and psychological impact were discarded after the two Delphi rounds. However, all the participants in the expert meetings agreed that these two dimensions deserved measurement in routine endometriosis care. Regarding sexual activity, both the Female Sexual Function Index (FSFI) 17 , 18 and Sexual Activity Questionnaires (SAQ) 19 , 20 are of proven validity and reliability in the field of endometriosis and are included among the complementary outcomes. Both have their drawbacks: the FSFI has a low sensitivity for endometriosis 18 and is not suitable for patients who are not sexually active; and while the SAQ take into consideration that patients may not be sexually active, they are not specific to endometriosis, and particularly for dyspareunia. 21 The panel's conclusion was that, although to date there is no suitable tool to measure the impact of endometriosis on sexual activity, the FSFI and SAQ should be included in the complementary outcomes due to the importance of measuring this commonly neglected dimension. Nevertheless, there was agreement of an urgent need to create an adapted tool to measure sexual activity. The evaluation of the psychological impact was also deemed important during discussion. Although all the proposed PROMs were considered were not validated to measure the psychological impact in endometriosis patients, until a suitable PROM is created and validated, the panel recommended adding the Hospital Anxiety and Depression Scale (HADS) to the complementary outcomes. 22 Finally, the Gastrointestinal Quality Life Index (GIQLI) was added in the list of complementary outcomes for use only with those patients for whom it is relevant, despite having been discarded in Round 2. In addition, two diaries were selected after the two Delphi rounds: the Endometriosis Symptom Diary (ESD), and Endometriosis Impact Scale (EIS). During the expert discussion, participants agreed that not only is recording symptoms daily, cumbersome for patients and may compel them to focus on their disease, but the results are challenging for clinicians to interpret. However, some considered that these kinds of PROMs could be useful within a limited timeframe for certain patients, particularly in the primary care setting. For this reason, they were included in the list of complementary outcomes. Of the 19 outcome measures selected in Round 2, 16 outcome measures (10 CROMs and six PROMs) that were judged relevant and feasible to use in routine endometriosis care were selected as the minimal set of patient‐centered outcomes (Table  2 ). Seven additional PROMs, considered to be relevant in specific clinical conditions only, were included in a complementary outcomes list, tailored to individual patients' needs (Table  3 ). Final and complementary set of patient‐centered outcomes for endometriosis care.

We developed a set of patient‐centered measures including PROMs and CROMs specifically designed for routine use in endometriosis care through a modified Delphi study with an international group of known experts and patient representatives. The final set includes: PROMs covering domains of pain‐related symptoms, symptomatic impact, work productivity, disease‐related quality of life, sexual activity, gastrointestinal symptoms and psychological impact; and CROMs covering clinical, imaging and surgical indicators. Despite patient‐to‐patient variability in symptomatology, these domains comprise the common features requested for all patients suffering from endometriosis. Collecting these outcome measures in routine practice is therefore a critical first step toward patient‐centered care. 3 Certain issues are not easily brought up during a consultation and may be consequently unaddressed by the clinician when deciding the therapeutic approach. It has been demonstrated that including patient‐centered outcome metrics in routine care and monitoring changes throughout the patient journey would provide clinicians with an opportunity to discuss their patients' expectations of a given treatment and potentially identify incompatibilities or unrealistic expectations. 3 , 11 Its implementation in routine clinical practice can complete the clinician's medical consultation and facilitate patient‐centered care by encompassing the patient's view of their disease. This is particularly relevant in endometriosis, since the patient's and physician's perception of the disease can differ significantly. 23 Overall, they can be used as shared decision‐making tools. 11 The set of patient‐centered outcomes measures presented in our study has been primarily chosen for its use in routine care. As clinical practice and research are two sides of the main coin, one may question whether the measurement set we propose can also be used in research and, in turn, those already developed for research purpose can overlap our results. Indeed, following the same method as ours, other research groups have recently published a core outcome set to standardize outcome selection, collection and reporting across future randomized controlled trials and systematic reviews evaluating potential treatments for endometriosis. 6 Our proposal differs mainly in the fact that we not only define the core dimensions that should be measured (pain, quality of life, psychologic impact) but additionally propose the use of currently validated measures for how they must be measured, ie what are the specific metrics to measure each dimension. In the Delphi process we used specific PROMs and CROMs already validated in endometriosis population, therefore guaranteeing the reliability of the measurement set we propose. Our approach toward measurement is closer to the one led by the World Endometriosis Research Foundation – the Endometriosis Phenome and Biobanking Harmonization Project (WERF EPHect), which aims to standardize the collection of a minimum of clinical information in endometriosis clinical trials using dedicated self‐assessed questionnaire. 7 , 8 , 9 , 10 Nonetheless, the principal aim of this initiative was to harmonize the various questionnaires used in distinct epidemiological or biological research projects on endometriosis and to facilitate international collaborative research. However, the questionnaires used in this approach have not been developed according to the Health‐Related patient measurement methodology (Cosmin) and therefore cannot be reliably used in the context of standard care. One of the strengths of the present study is the mix of both patient representatives and healthcare practitioners involved in the field of endometriosis. The DELPHI method has the advantage over other consensus methods that individuals can be included anonymously and without interacting directly with each other, which prevents the views of a minority from dominating the group. 24 This made it possible to select outcomes that are easy to understand and represent the patient's perspective but are also useful from a medical point of view. 3 As there is no recommended threshold for consensus studies, we chose a level of agreement based on a two‐thirds majority, which represents a commonly used level. 25 A stronger level was not chosen (eg three‐quarters, or absolute majority) as our survey did not aim to present compulsory recommendations for use (for example modifications of recommendations for good clinical practice that could require stronger agreement to be accepted), but instead to offer an approach that gave clinicians the choice of measures to use. One of the main limitations of this study is that the number and geographic diversity of patient representatives in the expert group were low. Patient associations from all over the world were contacted but few replied. This lack of response is a major challenge of efforts aimed to develop a more patient‐centered approach: patient community involvement is of upmost importance but their participation remains low. 26 Nevertheless, the ratio physician:patient representative remained stable through the different rounds, meaning that patient participation was equivalent in each round and in the final discussion meetings. Additionally, it is important to keep in mind that within the group of clinicians, the expert panel was made up of a large percentage of specialists in endometriosis. However, our work was designed to include the perspective of the standard gynecologists and general physicians. As a result, the selected set of patient‐centered measures is relevant to the common features of patients for implementation in routine care. From this list, clinicians can select according to their needs and their day‐to‐day reality. Another limitation of the study is the over‐representation of experts from European, high‐income countries because of difficulties encountered in identifying and engaging expert healthcare professionals or patient representatives from elsewhere. Thus, the outcome set may not be generalizable to or acceptable to experts and patients from low‐ and middle‐income countries where there is a large gap in both diagnosis and treatment as well as research in endometriosis between high‐income countries and low/middle‐income countries. 27 , 28 Adaptions to the outcome set may be needed for low‐ and middle‐income countries. Clinicians currently use CROMs but may be reluctant to use PROMs in routine care due to the fear of additional burden or extra workload. 3 The recent development of digital tools for PROM collection as well as advances in the development of standards for interoperability, make the idea of merging both CROMs and PROMs in a same visual interface a feasible reality. One possibility may be that these outcomes measures are displayed on the clinician dashboard during consultation. The next step of this project will be to test the implementation and evaluate the feasibility of data collection using digital tools.

We have developed a set of patient‐centered measures including PROMs and CROMs in endometriosis care through an international consensus. Considering the large variability in terms clinical setting, context and patient symptomatology in endometriosis, the selected outcomes comprise the common features for all patients suffering from endometriosis and may be implemented in routine care.

Endometriosis is a disorder which is characterized by presence of endometrium‐like tissue outside the uterus that may impact the reproductive and general health of women during the course of their life. 1 Epidemiological studies show that the worldwide prevalence of endometriosis is around 10% in women of reproductive age. 2 Multiple symptoms are associated with endometriosis, including severe dysmenorrhea, non‐menstrual pelvic pain, dyspareunia, intestinal or urinary symptoms and infertility. The heterogeneity of the symptomatology together with the multiplicity of therapeutic approaches to manage endometriosis‐associated symptoms complicate clinical management. As with other chronic pain conditions, endometriosis‐related pain often affects a woman's ability to function and her relationships, and can lead to mental health conditions such as depression. Overall, endometriosis can severely impact a patient's quality of life. 1 Incorporating in routine clinical practice a consistent approach to measure outcomes that matter to patients with endometriosis would enable providers to personalize and monitor treatment effects. Measuring outcomes (ie symptoms, quality of life) that are important to patients will be used as indicators to determine the most relevant therapeutic strategy. Outcome measures include both patient symptoms and quality of life using Patient Reported Outcomes Measurement 3 , 4 and medical observations, Clinician Reported Outcomes Measurement (CROMs). 5 Outcome measurements have been extensively used as research tools in clinical trials and have been shown to produce reliable and consistent results across different types of populations to measure treatment effect and satisfaction. 6 On the other hand, projects like the one led by the World Endometriosis Research Foundation – the Endometriosis Phenome and Biobanking Harmonization Project (WERF EPHect) – aim to standardize the collection of minimal clinical information in endometriosis research 7 , 8 , 9 , 10 and facilitate worldwide research. Importantly, in a pure research context, data collection may not in itself offer direct benefit to the individual patient, as this information is not intended for use to modify the patients' care at a particular visit. Moreover, the collection of data is not underpinned by constraints of time or human resources. On the other hand, routine care can be enhanced by using metrics to improve communication between the patient and physician and can therefore be used as shared decision‐making tools 11 and to increase patient empowerment. The use of patient‐reported metrics (ie PROMs) and the fact that they are taken into consideration by the physician will improve patients' awareness, understanding of their pathology and participation in their healthcare pathway. 3 Studies have shown that the use of PROMs in routine clinical care improves clinical outcome, enabling patients to measure, report and share the responsibility for the management of their condition. 3 The aim of the present study was therefore to identify patient‐ and clinician‐important PROMs and CROMs for use in routine endometriosis care which are relevant to any patient: those with either pain or infertility and clinical or imaging elements suggestive of endometriosis before its diagnosis; also, those with complex or severe disease. For that purpose we conducted a modified Delphi consensus process with an international multidisciplinary group of experts and patients in endometriosis including healthcare professionals (gynecologists, surgeons, radiologists, psychotherapists, pain physicians) working in centers specialized in endometriosis, general practitioners, researchers specialized in endometriosis and patient experts.

The authors have stated explicitly that there are no conflicts of interest in connection with this article.

The modified Delphi process is a common methodology which has previously been used to develop a standard set of indicators. 12 This methodology is used to reach agreement among experts on a given topic through several rounds of surveys that aim to collect the point of view of these experts. However, in the modified Delphi method, rather than continue with many rounds until agreement was reached, two survey rounds and a final open discussion are performed to reach consensus. 12 An overview of the whole process is described in Figure  1 . A two‐round modified Delphi consensus study to determine a set of patient‐centered outcome measurements for routine use in endometriosis care. A systematic review of the literature was conducted to identify the PROMs and CROMs used in observational studies and clinical trials on endometriosis. The MEDLINE/PubMed database was screened for publications in English from 1984 to January 2021 including the following MeSH (Medical Subject Headings) terms: “quality outcomes”, “quality indicators”, “quality of life”, “patient reported outcome”, “clinician reported outcomes”, “endometriosis”. Only validated (ie having undergone a validation study) PROMs and CROMs were retained. A thorough review of the identified PROMs and their potential to be used in routine endometriosis care has already been described elsewhere. 13 The PROMs and CROMs were grouped according to the type of indicator measured: four subgroups for the CROMs (clinical indicators, imaging indicators, biological indicators and surgical indicators) and 11 for the PROMs (symptomatic impact, pain, fatigue, sexual activity, gastrointestinal symptoms, urinary symptoms, psychological impact, work productivity, endometriosis quality of life, general quality of life and miscellaneous). A complete description of the PROMs and CROMs used in the survey was made available to all participants (Table  S1 ). The steering committee (AF, JB, PS, AT, ANB) selected an international and diverse panel of healthcare professionals (gynecologists, surgeons, radiologists, psychotherapists, pain physicians) working in centers specialized in endometriosis, non‐specialized gynecologists, general practitioners, researchers specialized in endometriosis and patient representatives to ensure a multidisciplinary and comprehensive view, including the patient viewpoint. The selection of participants contacted to be part in the expert panel was based on three complementary approaches: (i) some experts were selected based on the level of expertise in PROMs and CROMs in endometriosis according to the literature using MEDLINE (“quality outcomes”, “quality indicators”, “quality of life”, “patient reported outcome”, “clinician reported outcomes”, “endometriosis”) and communications on this topic in the different international congresses on endometriosis (WCE and SEUD); (ii) additional experts were added from suggestions made by the steering committee of non‐specialized gynecologists and general practitioners interested in endometriosis care; (iii) the panel was completed with experts referenced by the World Endometriosis Society. The panel experts were recruited from different geographic regions – mainly developed countries in Europe, America and Australia/Oceania – to ensure the representation of a wide range of clinical practices. Patient representatives were recruited from patient associations across the geographic regions. All associations listed in the World Endometriosis Organizations ( https://endometriosis.org/support/world‐endometriosis‐organisations‐weo/ ) were contacted by email twice. Only three national patient organizations agreed to participate: ENDOFRANCE ( https://www.endofrance.org/ ) from France, Endometriose Stichting ( https://www.endometriose.nl/ ) from the Netherlands, and Asociacion de Afectadas de Endometriosis ( https://adaec.es/ ) from Spain. The expert panel did not receive remuneration for their participation. The Qualtrics platform ( https://www.qualtrics.com/ ) was used to invite the participants to rate the PROMs and CROMs. Non‐responders were recontacted by email twice. Each participant evaluated the 47 pre‐selected PROMs and the 30 pre‐selected CROMs using a single Likert scale of 1–9 for their relevance and feasibility in assessment of women with endometriosis, where 1 meant completely disagree (not relevant or feasible for routine practice) and 9 completely agree (relevant and feasible fore in routine practice). By relevance we meant that the CROM/PROM indicators were pertinent for endometriosis care, that they measure outcomes that really matter to patients with endometriosis and that their measurement could be used to improve the care pathway at an individual level. By feasibility, we meant that the indicators could be easily measured in routine clinical practice with minimal additional time or cost. All participants were provided with a short description and literature references of each of the PROM/CROM indicators. The whole study was conducted in English. Participants were also invited to suggest additional outcomes that had not been included in the initial list. At the end of Round 1, all the indicators with a median score ≥7 and where at least 50% of the panel ratings were ≥7, were retained for Round 2. Indicators not meeting the threshold were discarded by the steering committee. 12 Round 2 was another self‐administered online survey sent to those who participated in the first round. Each participant received the results of the first round including the median rating, frequency distributions and a reminder of their personal ratings in Round 1. For Round 2 the participants were asked to consider each indicator after reviewing their prior rating as well as the panel ratings for Round 1. An indicator was retained in the final set if the median score was ≥7 and at least 65% of the panel ratings were ≥7. 12 Due to the geographic diversity and the COVID‐19 pandemic, a series of videoconference meetings scheduled on three different days – October 25, 26 and 27, 2021 – at three different times of the day enabled engagement of the international experts. The meeting dates were decided using a Doodle with different times that were created to suit to the different time zones. The chosen dates were those in which more participants declared themselves to be available. Anyone who participated in Round 2 was invited to participate in these meetings. During each meeting, the results from Round 2 were presented and indicators that were not selected but which were close to the agreement threshold (65%) were discussed, as well as any categories or indicators that participants raised for clarification or inclusion. Each meeting was chaired by two of the authors (ANB and AF). Experts attended only one of the three meetings. A descriptive analysis of the characteristics of the participants and the results of each Delphi round was conducted. Results are represented as medians (Q1, Q3, interquartile range) for continuous variables, and percentages (%) for categorical variables. The medians and the interquartile ranges of the rating were measured in each of the Delphi rounds and the percentages described the level of agreement among panelists. The analyses were performed using Microsoft EXCEL Software. The study was registered on clinicalgrials.gov (Identifier: NCT04820582 ). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee. Our study did not involve any intervention and was therefore exempt from the French regulations on biomedical research (modified version of the Law 2004–806, dated August 9, 2004). This study received ethics approval from the Comité d'Ethique de la Recherche end Obstétrique et Gynécologie (CEROG) on May 21, 2021 (number 2021‐GYN‐0409).

Table S1 Click here for additional data file.