Standardizing definitions and reporting guidelines for the infertility core outcome set: an international consensus development study†  ‡.

Randomized controlled trials (RCTs) evaluating potential treatments for infertility have reported many different outcomes ( Wilkinson et al. , 2019a ). Such variation contributes to challenges in comparing, contrasting and combining individual trials, limiting the usefulness of research to inform clinical practice ( Duffy et al. , 2019b ). The development, dissemination and implementation of a minimum data set, known as a core outcome set, will help to standardize outcome selection, collection and reporting across future infertility research. A core outcome set for infertility ( Fig. 1 ) has been developed ( Duffy et al. , 2020a ). However, there are inconsistencies in how individual core outcomes are currently defined by fertility trials. For example, definitions of live birth include a viable fetus after 24 weeks of gestation, pregnancy continuation beyond 28 weeks of gestation and delivery of a living baby ( Wilkinson et al. , 2016 ). Such variation makes it possible for researchers to selectively report their results based on statistical significance. For example, researchers can undertake multiple statistical analyses at different gestational thresholds for live birth and selectively report the most favorable result. A core outcome set for future infertility research . There are unique challenges when reporting the results of infertility research because of the multistage nature of the treatment, particularly in the context of IVF ( Wilkinson et al. , 2016 ). Multiple clinical and procedural events can occur during treatment. These events can be reported in subgroups containing only those patients who reach a certain milestone, for example, oocyte retrieval, embryo transfer and implantation. When reporting individual core outcomes, there could be many denominators available. This enables researchers to undertake multiple analyses using different denominators and selectively report results. The variation in definitions and poor reporting practices makes comparing and combining individual RCTs challenging. When these practices are common, it is likely the benefits of fertility treatments are being overestimated and the harms of treatments are being underestimated ( Duffy et al. , 2019b ). This undermines secondary research, including individual patient data (IPD) meta-analysis and network meta-analysis, and compromises clinical practice guideline development. Standardizing definitions and improving reporting for individual core outcomes creates an opportunity to develop additional consistency in future infertility trials and ensure that secondary research can be undertaken prospectively, efficiently and harmoniously ( Duffy et al. , 2017a ). No guidelines have established recommendations regarding the development of consensus definitions and reporting guidelines for individual core outcomes ( Williamson et al. , 2017 ). Outside the context of core outcome set development, the Harbin Consensus Conference Workshop has developed a standardized definition for live birth ( The Harbin Consensus Conference Workshop Group, 2014 ) and the International Committee for Monitoring Assisted Reproductive Technologies (ICMART) has standardized definitions related to infertility and ART ( Zegers-Hochschild et al. , 2017 ). Motivated by the desire to maximize the potential of infertility research to inform clinical practice, an international collaboration coordinated by the Cochrane Gynaecology and Fertility Group has brought healthcare professionals, researchers and people with infertility together to standardize definitions for the core outcome set for infertility.

Potential definitions were inventoried across four definition development initiatives, including the Brighton Collaboration ( Chen et al. , 2016 ), Harbin Consensus Conference Workshop Group, ICMART and World Health Organization (WHO) ( World Health Organization, 2018 ), 12 clinical practice guidelines ( American Urological Association, 2010a , b , c ; Jarvi et al. , 2010 ; Kroon et al. , 2011; Koch et al. , 2012 ; National Institute for Health and Care Excellence, 2013 ; Loh et al. , 2014 ; Carranza-Mamane et al. , 2015 ; Practice Committee of the American Society for Reproductive Medicine; 2017a , b ; Jungwirth et al. , 2018 ), and the standardized methods advocated by the Cochrane Gynaecology and Fertility Group for the preparation of systematic reviews evaluating potential fertility treatments. Forty-four potential definitions were discussed during the consensus development meeting. Twenty-seven participants, including 14 healthcare professionals, 7 researchers and 6 people with fertility problems, from 11 countries, participated in the consensus development meeting ( Table I ). Participant characteristics. When considering live birth, participants noted the Improving the reporting of clinical trials of infertility treatments (IMPRINT) statement recommended a gestational age threshold of 20 completed weeks. This statement was specifically developed to improve outcome reporting in infertility trials by modifying the Consolidated Standards of Reporting Trials (CONSORT) statement ( Moher et al. , 2010 ). Given this context, participants agreed the IMPRINT gestational threshold should be recommended to ensure consistency across comparable initiatives standardizing outcome reporting in RCT ( Table II ). Standardized definitions for the core outcome set for infertility. Researchers should report at which gestation the ultrasound examination was performed. Pregnancies are counted as pregnancy events, for example, a twin pregnancy is counted as one pregnancy event. Effect size estimates and 95% confidence interval should be reported for pregnancy events. The denominator should be per participant randomized. Singleton, twin and higher multiple pregnancy should be reported separately. When considering twin and higher multiple pregnancies, pregnancy loss should be explicitly accounted for. Miscarriage should be reported after a viable pregnancy has been confirmed by ultrasound. When considering stillbirth involving twins and higher multiple births they should be reported as a single event. Selective embryo or fetal reduction should be reported. Live births are counted as birth events, for example, twin live birth is counted as one live birth event. Effect size estimates and 95% confidence interval should be reported for live birth events. The denominator should be per participant randomized. Singletons, twin and higher multiple births should be reported separately. The gestational age of both live births and stillbirths should be reported. Gestational age at birth should be reported as a median and interquartile range. Reporting the mean and standard deviation in addition would support future meta-analysis. The birthweight of singletons, twins and higher multiples should be reported separately. Birthweight for each newborn infant of the multiple birth set should be reported. Birthweight should not be adjusted for gestational age. The birthweight of stillbirths should be reported. Mortality related to preterm infants should be collected up to 28 days beyond their estimated due date. If a member of a multiple birth set dies in the neonatal period this should be explicitly reported. Major congenital anomalies should be classified using a standardized taxonomy. Major congenital anomaly should be reported as an infant with at least one major congenital anomaly detected. If a major congenital anomaly is identified in a member of a multiple set this should be explicitly reported. When considering the reporting of live birth, participants recommended twin and higher multiple births should be reported as a single live birth event ( Table III ). This will ensure treatments which increase twin and higher multiple births are not favored. The participants agreed that the summary effect size estimate and 95% CI should be calculated for live birth events only, and recommended the number of participants randomized as the most appropriate denominator. In addition to reporting live birth events, singleton, twin and higher multiple births should be reported narratively. When calculating the corresponding percentages for live birth events and singleton, twin and higher multiple births, the number of participants randomized is the recommended denominator. Generic reporting table. Effect size estimates and 95% CI should only be reported for live birth event and viable pregnancy confirmed by ultrasound. The remaining data should be summarized narratively. For live birth event and viable pregnancy confirmed by ultrasound the number of participants randomized should be used as the denominator. When considering twin and higher multiple pregnancies, pregnancy loss should be explicitly accounted for within the table footnote. For gestational age at delivery reporting the mean and SD within the table footnote would support future meta-analysis. The birthweight for each newborn infant of the multiple birth set should be reported. If a member of a multiple birth set dies in the neonatal period this should be explicitly stated within the table footnote. Reported as an infant with at least one major congenital anomaly detected. If a major congenital anomaly is identified in a member of a multiple set this should be explicitly stated within the table footnote. g, grams; N, number of randomized participants; No, number of events; IQR, interquartile range. Carefully selecting an appropriate denominator will avoid common issues associated with the analyses of data arising from infertility trials, particularly for studies related to ART. These issues are discussed in detail within the discussion. Participants agreed a consensus definition, which included visualization of a heartbeat. Participants discussed the reporting of twin and higher multiple pregnancies and recommended they should be reported as a single pregnancy event. The effect size estimate and 95% CI should be calculated for pregnancy events only. Participants concluded that it was also important for singleton, twin and higher multiple pregnancy to be routinely reported. When calculating the corresponding percentages for pregnancy events and singleton, twin and higher multiple pregnancies, the number of participants randomized is the denominator which should be used. Participants discussed the importance of embedding RCT within routine clinical practice and were reluctant to insist upon mandatory urinary or serum beta-hCG testing or ultrasonographic examinations in addition to routine care. The variation in routine ultrasonographic examination between countries was discussed, for example, routine ultrasound scans are performed between 6 and 8 weeks in the USA, between 11 and 13 weeks in the UK and following 16 weeks in the Netherlands. Following the discussion, a contextual statement was recommended to ensure researchers consistently reported the gestation at which the ultrasonographic examination diagnosing viable intrauterine pregnancy was performed. Following discussion, consensus was reached to adopt the ICMART definition of ectopic pregnancy. Participants discussed the WHO’s definition for miscarriage and observed this definition was the most widely used within an international context. The definition includes a gestational age threshold of 20 completed weeks. They observed such a threshold would correlate well with the IMPRINT statement’s definition of live birth, which was previously adopted. Participants unanimously agreed to modify the ICMART definition of late fetal loss to include an estimated gestational age threshold of 20 completed weeks. Within the context of this core outcome set, participants recommended miscarriage should only be reported after a viable pregnancy has been confirmed by ultrasound. Participants discussed the variety of contextual factors including local cultural influences, legislative framework and national and international reporting requirements, which would influence the different gestational age thresholds incorporated in different definitions of stillbirth. They highlighted the importance of accounting for all pregnancy losses and the gestational age threshold for stillbirth would need to consider the threshold already agreed for miscarriage. Participants unanimously agreed to modify the ICMART definition to include a gestational age threshold of 20 completed weeks with an appropriate adjustment for birthweight. When considering stillbirth involving twins and higher multiple pregnancies, participants recommended they should be reported as a single event. Following discussion, consensus was reached to adopt the ICMART definition of termination of pregnancy. Participants noted the importance of reporting selective embryo or fetal reduction. Participants discussed the reporting of pregnancy loss and recommended ectopic pregnancies, miscarriages, stillbirths and terminations of pregnancy should be reported numerically. Percentages and effect size estimates should not be reported. When considering twin and higher multiple pregnancies, participants recommended pregnancy losses should be accounted for within the footnotes of the reporting table and summarized narratively within the study report. Following discussion, consensus was reached to adopt the ICMART definition of gestational age. Participants recommended that gestational age at delivery should be reported for both live births and stillbirths. Participants agreed gestational age at delivery should be reported as the median and interquartile range. An effect size estimate should not be reported. Participants recommended that researchers should be encouraged to report the mean and SD within the reporting table footnote to support future meta-analysis. Participants noted the measurement of birthweight as being well-characterized. Participants noted best practice recommendations, which recommend collecting birthweight within 24 h of birth and using a calibrated electronic scale with 10-g resolution. If there is limited availability of correctly calibrated electronic scales, the type of scale and its calibration should be clearly reported. Participants recommended birthweight should not be adjusted for gestational age. Participants agreed birthweight, reported as a mean and SD, should be recorded separately for singleton, twin and higher multiple infants. The birthweight for each infant of a multiple birth set should be reported. Participants noted the consistent use of the WHO definition for neonatal mortality across definition development initiatives, including ICMART, international and national guidelines and Cochrane systematic reviews. A contextual statement was agreed to ensure researchers report any mortality of preterm infants up to 28 days beyond their estimated due date. Participants agreed neonatal mortality should be reported numerically. Percentages and effect size estimates should not be reported. If a member of a multiple birth set dies in the neonatal period this should be stated within the reporting table footnote and summarized narratively within the study report. Participants discussed how congenital anomalies varied in severity, with severe anomalies impacting upon an infant’s health, development and survival. Participants reached a view that future RCT should consistently report major congenital anomalies. Participants unanimously agreed to modify the ICMART definition to include criteria to ensure only major congenital anomalies are reported. Participants stated the importance of classifying congenital anomalies using a standardized taxonomy ( DeSilva et al. , 2016 ). Participants agreed major congenital anomalies should be reported as an infant with at least one major congenital anomaly detected. If a major congenital anomaly is identified in a member of a multiple set this should be stated within the reporting table footnote and summarized narratively within the study report. Percentages and effect size estimates should not be reported. Detailed guidance regarding the collection, analysis and reporting of time to pregnancy leading to live birth was approved by the meeting participants and has been provided as Supplementary Data File S1 .

The study was prospectively registered with the Core Outcome Measures in Effectiveness Trials (COMET) initiative, registration number 1023. An international steering group, including healthcare professionals, researchers and people with fertility problems, was established to provide a perspective to inform key methodological decisions. The important work of the Harbin Consensus Conference Working Group and ICMART is complementary to this study. A protocol describing the study’s consensus methods has been published ( Duffy et al. , 2018 ). The protocol was developed with reference to the COMET initiative handbook ( Williamson et al. , 2017 ). The protocol was also informed by a systematic review of registered, progressing and completed core outcome sets relevant to women’s and newborn health (Duffy et al. , 2017b) and the experiences of steering group members involved in other core outcome set development studies ( Hirsch et al. , 2016 ; Khalil et al. , 2017 ; Webbe et al. , 2017 ; Whitehouse et al. , 2017 ; Khalil et al. , 2019 ). Potential definitions for individual core outcomes were extracted from definition development initiatives and national and international clinical practice guidelines and by reviewing the Cochrane Gynaecology and Fertility Group’s standardized guidance for infertility reviews. A systematic review was undertaken searching the COMET initiative register to identify definition development initiatives relevant to infertility research, from inception to October 2018. Clinical practice guidelines relevant to infertility were identified by searching bibliographical databases, including Embase, MEDLINE and PubMed, from inception to October 2018. The Cochrane Gynaecology and Fertility Group provided access to their editorial policy, which describes their standardized approach to the selection of outcomes and definitions across Cochrane reviews evaluating potential fertility treatments. Using a pilot-tested and standardized data extraction form, definitions were extracted verbatim from all sources. An inventory was developed by organizing potential definitions within an organizational framework ( Supplementary Fig. S1 ). Steering group members with expertise in statistics and research methodology prepared discussion points related to the analysis and reporting of the core outcome set. The inventory and discussion points were discussed during a face-to-face consensus development meeting held in Auckland, New Zealand. The consensus development conference is a formal consensus development method developed by the US National Institutes of Health and has been used to reach consensus for definitions, clinical practice recommendations and professional competencies ( Ferguson, 1996 ). The consensus method was developed to include aspects of judicial decision-making, scientific conferences and the town hall meeting. Participants hear evidence on which they will later deliberate and are able to ask questions as the evidence is presented. The chairperson is responsible for directing the discussion. The group discussion follows an informal format. Healthcare professionals, researchers and people with fertility problems who had participated in the Delphi survey, which informed the development of the core outcome set for infertility, were invited to participate ( Duffy et al. , 2020a ). The study aimed to recruit between 10 and 15 participants, as this number has yielded sufficient results and assured validity in other studies ( Murphy et al. , 1998 ). Before starting the meeting, participants provided demographic details. The group discussion followed an informal format with the chairperson providing direction. Each core outcome was discussed in turn. Potential definitions were displayed within the definition hierarchy. Each participant was asked to contribute their opinions. Participants were encouraged to suggest other potential definitions or reformulate individual definitions to improve clarity or comprehension. Although the group was encouraged to reach consensus, members were able to express minority or alternative views when consensus could not be achieved. Participants were encouraged to agree contextual statements to highlight important methodological issues which would need to be considered when reporting individual core outcomes. Participants also developed consensus guidance regarding statistical analysis and a reporting table.

Definition development initiatives, clinical practice guidelines and Cochrane reviews have defined individual core outcomes in different ways. Through formal consensus methods, 14 healthcare professionals, 7 researchers and 6 people with fertility problems, from 11 countries, have successfully developed consensus definitions for all core outcomes. Specific recommendations have been made to improve the reporting of core outcomes. This study has used formal consensus methods to develop consensus definitions for the core outcome set for infertility. The consensus development conference is a formal consensus development method developed by the US National Institutes of Health and has been used to reach consensus on a variety of topics in many different countries including, Canada, UK and Sweden. The study has engaged a range of different stakeholders, including healthcare professionals, researchers and people with fertility problems, from different countries. Such diversity should secure the generalizability of the results and increase its credibility with other researchers. The study has developed clear and concise recommendations to enable future researchers to collect core outcomes in a standardized approach and report their results in a clear and transparent manner. This study is not without limitations. There is significant uncertainty regarding the optimal methods for core outcome set development ( Duffy and McManus, 2016 ; Williamson et al. , 2017 ; Duffy et al. , 2019a ). The COMET initiative has made no formal recommendations regarding the development of consensus definitions. They advocate the use of formal consensus development method in other aspects of core outcome set development, which informed the methodological choices we made in this study. Different formal consensus methods, including the modified Delphi method and modified Nominal Group Technique, could have been used. Further methodological research is required to evaluate the most appropriate consensus methods for studies similar to ours. Consideration should be given to the representativeness of the steering group and consensus meeting participants. Many consensus meeting participants were from European countries ( n  = 12; 44%) and there was limited representation from low- and middle-income countries, which could have impacted upon the development of consensus definitions. Further research should be undertaken to evaluate virtual or blended formats to improve representation while preserving limited resources. Analyses of data arising from infertility trials, particularly for studies related to ART, are frequently undermined by the use of an inappropriate denominator ( Wilkinson et al. , 2016 ). Two main issues exist. The first is the use of a post-randomization denominator, for example, when live birth rates are calculated per embryo transferred, rather than per woman randomized. Analyses conducted on this basis do not reflect the randomized comparisons, as the groups being compared may differ with respect to their characteristics, and therefore, also with respect to their outcomes ( Hirji and Fagerland, 2009 ). The second issue relates to analyses that commit a unit of analysis error ( Vail and Gardener, 2003 ). This error occurs when proportions are calculated using an inappropriate denominator, for example, the number of oocytes or number of embryos. Unit of analysis errors commonly occurs when researchers calculate the pregnancy rate by dividing the number of gestational sacs on ultrasound by the number of embryos transferred. As the outcomes of a couple’s embryos are correlated, this approach is incorrect as standard statistical tests assume that the tested observations are independent. To avoid these important issues, it is good practice to calculate viable pregnancy confirmed by ultrasound and live birth events using the number of participants randomized as the denominator. This approach is explicitly stated within the core outcome set recommendations. Sophisticated statistical analysis methods capable of accommodating post-randomization comparisons and clustered data do exist. They could be reported in addition to the core outcome set if researchers had access to the necessarily statistical expertise. This study has developed the generic building blocks for future infertility research. A minimum data set affords the opportunity for researchers to easily populate protocols, case report forms and other data collection tools with core outcomes and consensus definitions. The generic reporting table should assist researchers to clearly report their results within journal publications and conference presentations. Implementing a standardized approach should reduce poor reporting practices, for example, incomplete reporting, selective reporting based on statistical significance and inappropriate use of denominators ( Wilkinson et al. , 2016 ). It is anticipated that research studies with limited access to methodological and statistical advice will benefit the most. Systematic implementation of this core outcome set should ensure the core outcomes are consistently defined by individual trials. Symmetrical application of standardized definitions in all trial arms is known to reduce measurement bias, including observer and verification bias ( Mansournia et al. , 2017 ). Blinding outcome assessors to the treatment allocation would further reduce bias ( Sterne et al. , 2016 ). Outcome assessors should also undertake comprehensive training. Other strategies can help to ensure consensus definitions are applied correctly and, in a manner, which is unlikely to vary, including standardized data collection tools, internal validation studies and independent adjudication panels. A freely available electronic case report form and data repository are currently being planned to standardize the collection of the core outcome set within future infertility trials (COMMIT-Collection). The Core Outcomes in Women’s and Newborn Health (CROWN) initiative, supported by over 80 specialty journals, including the Cochrane Gynaecology and Fertility Group , Fertility and Sterility and Human Reproduction , have resolved to implement the core outcome set for infertility ( Core Outcomes in Women's and Newborn Health Initiative, 2014 ). In the future participating journals will request researchers to report the definitions for individual core outcomes within published trial reports. When the consensus definition has not been used, the researchers will be asked to report this observation and its implications for their findings. Reporting will be facilitated by the recommendations made within this study. The need to combine the results of individual trials evaluating fertility treatments should be anticipated by researchers ( Wilkinson et al. , 2019b ). Implementing the core outcome set, including consensus definitions, should be considered good practice and could make a significant contribution in improving the coordination, development and delivery of fertility research within regional, national and international settings ( Devall et al. , 2020 ). Standardization will facilitate pairwise meta-analysis and more sophisticated secondary research, including IPD and network meta-analysis ( Duffy et al ., 2019b ). These approaches could provide unique insights into the effectiveness and safety of fertility treatments. The consensus definitions developed as part of this study could be incorporated into other core outcome sets to promote additional harmony across women’s health. Other core outcome sets have been developed for endometriosis, hyperemesis gravidarum and preterm birth, which share common core outcomes including live birth, neonatal mortality and major congenital anomalies ( van 't Hooft et al. , 2016 ; Jansen et al. , 2020 ; Duffy et al. , 2020b ). Core outcome set developers should be encouraged to use the consensus definitions developed as part of this study. Standardized consensus definitions are not meant to limit regional, national and international requirements to collect and report collect core outcomes using specific definitions, including live birth, stillbirth and congenital anomalies. For example, researchers undertaking research in the UK may wish to define stillbirth as occurring after 22 weeks of gestation, in line with national recommendations ( Da Silva et al ., 2016 ). Researchers wishing to collect data using other definitions in the context of their own RCT would continue to be able to do so. Selective reporting should be avoided by presenting findings for both the consensus definition and any other definition used. Researchers would need to carefully consider how these data would be collected to fulfill different definitions and reporting obligations. The ultimate objective of an infertility trial is a healthy baby who develops normally. There are significant challenges in developing an objective consensus definition regarding what constitutes a ‘healthy baby’ as contextual factors, including local practices, cultural influences and legal implications, are important considerations. Consensus was reached to define live birth based on a 20-week gestational age threshold, reflecting IMPRINT recommendations and WHO guidelines. The current limit of viability is considered to be 22 weeks of gestation; however, the threshold is constantly challenged as perinatal and neonatal medicine advances. This context was also considered and a clear threshold has been decided through a robust consensus process to facilitate clear reporting across future infertility research ( The Harbin Consensus Conference Workshop Group, 2014 ). The core outcome set should be reported by all future RCTs evaluating potential fertility treatments. This context is important when considering the consensus definition developed for pregnancy and miscarriage. Routine urinary or serum beta-hCG testing is a common feature of IVF research, however, is less likely when evaluating other interventions. To take this into account, the consensus definition for pregnancy and miscarriage includes ultrasound, which is a common component of antenatal care. An extension to the core outcome set specifically for IVF research (COMMIT-IVF) is currently being developed and includes pregnancy confirmed by urinary or serum beta-hCG testing and early miscarriage. The development of consensus definitions has provided additional focus upon the language researchers commonly use when reporting infertility research. People with fertility problems and the patient organizations involved in this study have routinely commented upon terminology. It has been often perceived as lacking a patient-centric approach including terms such as missed spontaneous abortion, induced abortion and fetal loss. Researchers should recognize the language used to report fertility research is important and holds significance to people with fertility problems. The standardization of terminology within this core outcome set has been developed to ensure precision and with consideration to good practice guidelines in partnership with people with fertility problems and the patient organizations. The COMMIT initiative has committed to undertaking further research to assess the uptake and implementation of the core outcome set for infertility (COMMIT-Implementation). Assessing the uptake of the core outcome set, including the use of consensus definitions, will be undertaken by examining registry records, published protocols and RCT. Further research is planned to examine and understand the reasons why researchers do, and do not, implement the core outcome set for infertility. By identifying perceived barriers to implementation, strategies will be developed to promote implementation of the core outcome set across future infertility research. In conclusion, ensuring that core outcomes are consistently defined across RCT evaluating potential fertility treatments will secure evidence which is more accessible and facilitate the translation of research into clinical practice. Standardized reporting should help limit poor reporting practices. Future researchers should benefit from core outcomes and consensus definitions, which can be included in protocols, case report forms and other data collection tools. The generic reporting table should assist researchers in clearly reporting their results in journal publications and conference presentations.

Click here for additional data file. Click here for additional data file.