{"paper_id":"a8f43bee-4475-4bef-b1d9-f37d4aa520e4","body_text":"Human Reproduction, Vol.33, No.12 pp. 2268 –2275, 2018\nAdvanced Access publication on October 25, 2018 doi:10.1093/humrep/dey317\nORIGINAL ARTICLE Infertility\nExternal validation of a dynamic\nprediction model for repeated\npredictions of natural conception\nover time\nR. van Eekelen1,2,*, D.J. McLernon3, M. van Wely1, M.J. Eijkemans2,\nS. Bhattacharya4, F. van der Veen 1, and N. van Geloven 5\n1Centre for Reproductive Medicine, Amsterdam UMC, Academic Medical Centre, Meibergdreef 9, 1105 AZ Amsterdam, the Netherlands\n2Department of Biostatistics and Research Support, Julius Centre, University Medical Centre Utrecht, Heidelberglaan 100, 3584 CX Utrecht,\nthe Netherlands 3Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB24 3FX Aberdeen, UK 4Institute\nof Applied Health Sciences, University of Aberdeen, AB24 3FX Aberdeen, UK 5Medical Statistics, Department of Biomedical Sciences, Leiden\nUniversity Medical Centre, Einthovenweg 20, 2333 ZC Leiden, the Netherlands\n*Correspondence address. Centre for Reproductive Medicine, Amsterdam UMC, Academic Medical Centre, Meibergdreef 9, 1105 AZ\nAmsterdam, The Netherlands. Tel: +31 20 566 7379; E-mail: r.vaneekelen@amc.uva.nl\nSubmitted on February 16, 2018; resubmitted on September 25, 2018; accepted on October 5, 2018\nSTUDY QUESTION : How well does a previously developed dynamic prediction model perform in an external, geographical validation in\nterms of predicting the chances of natural conception at various points in time?\nSUMMARY ANSWER: The dynamic prediction model performs well in an external validation on a Scottish cohort.\nWHAT IS KNOWN ALREADY : Prediction models provide information that can aid evidence-based management of unexplained subfer-\ntile couples. We developed a dynamic prediction model for natural conception (van Eekelen model) that is able to update predictions of nat-\nural conception when couples return to their clinician after a period of unsuccessful expectant management. It is not known how well this\nmodel performs in an external population.\nSTUDY DESIGN , SIZE, DURATION: A record-linked registry study including the long-term follow-up of all couples who were con-\nsidered unexplained subfertile following a fertility workup at a Scottish fertility clinic between 1998 and 2011. Couples with anovulation, uni/\nbilateral tubal occlusion, mild/severe endometriosis or impaired semen quality according to World Health Organization criteria were\nexcluded.\nPARTICIPANTS/MATERIALS, SETTING, METHODS: The endpoint was time to natural conception, leading to an ongoing preg-\nnancy (defined as reaching a gestational age of at least 12 weeks). Follow-up was censored at the start of treatment, at the change of partner\nor at the end of study (31 March 2012). The performance of the van Eekelen model was evaluated in terms of calibration and discrimination\nat various points in time. Additionally, we assessed the clinical utility of the model in terms of the range of the calculated predictions.\nMAIN RESULTS AND THE ROLE OF CHANCE : Of a total of 1203 couples with a median follow-up of 1 year and 3 months after the\nfertility workup, 398 (33%) couples conceived naturally leading to an ongoing pregnancy. Using the dynamic prediction model, the mean prob-\nability of natural conception over the course of the first year after the fertility workup was estimated at 25% (observed: 23%). After 0.5, 1 and\n1.5 years of expectant management after the completion of the fertility workup, the average probability of conceiving naturally over the next\nyear was estimated at 18% (observed: 15%), 14% (observed: 14%) and 12% (observed: 12%). Calibration plots showed good agreement\nbetween predicted chances and the observed fraction of ongoing pregnancy within risk groups. Discrimination was moderate with c statistics\nsimilar to those in the internal validation, ranging from 0.60 to 0.64. The range of predicted chances was suf ficiently wide to distinguish\nbetween couples having a good and poor prognosis with a minimum of zero at all times and a maximum of 55% over the first year after the\nworkup, which decreased to maxima of 43% after 0.5 years, 34% after 1 year and 29% after 1.5 years after the fertility workup.\n© The Author(s) 2018. Published by Oxford University Press on behalf of the European Society of Human Reproduction and Embryology. All rights reserve d.\nFor Permissions, please e-mail: journals.permissions@oup.com\n\nLIMITATIONS, REASONS FOR CAUTION : The model slightly overestimated the chances of conception by ~2 –3% points on group\nlevel in the first-year post-fertility workup and after 0.5 years of expectant management, respectively. This is likely attributable to the fact that\nthe exact dates of completion of the fertility workup for couples were missing and had to be estimated.\nWIDER IMPLICATIONS OF THE FINDINGS : The van Eekelen model is a valid and robust tool that is ready to use in clinical practice\nto counsel couples with unexplained subfertility on their individualized chances of natural conception at various points in time, notably when\ncouples return to the clinic after a period of unsuccessful expectant management.\nSTUDY FUNDING/COMPETING INTEREST(S): This work was supported by a Chief Scientist Of fice postdoctoral training fellowship\nin health services research and health of the public research (ref PDF/12/06). There are no con flicts of interest.\nKey words: natural conception / expectant management / prognosis / prediction model / dynamic prediction / retrospective cohort\nIntroduction\nApproximately 10% of all couples who wish to have a child do not con-\nceive within the first year of trying ( Gnoth et al., 2003 ; Wang et al.,\n2003). For approximately half of these couples, no clear barrier for\nconception can be found during the workup and these couples are\nconsidered unexplained subfertile (\nAboulghar et al., 2009 ; Brandes\net al., 2010 ). It is unclear whether these couples should start with\nART; first, since observational studies report that 18 –38% of unex-\nplained subfertile couples will conceive naturally in the year after the\nfertility workup ( Hunault et al., 2004 ; van der Steeg et al., 2007 ; van\nEekelen et al., 2017a ) and second, since there remains uncertainty\nregarding the effectiveness of ART for unexplained subfertile couples\n(Pandian et al., 2015 ; Tjon-Kon-Fat et al., 2016 ; Veltman-Verhulst\net al., 2016; van Eekelen et al., 2017b ).\nIn the absence of clear evidence on the management of unexplained\nsubfertile couples and when to offer ART, an enticing option is to cal-\nculate chances of natural conception and to base counselling on this\nestimated prognosis (\nvan Eekelen et al., 2017b ). Fundamental to this\napproach is to identify couples that are expected to bene fit from treat-\nment and those who are not. In clinical practice, this would imply that\ncouples with a good prognosis to conceive naturally are advised to\ncontinue to try and become pregnant by sexual intercourse, while cou-\nples with an unfavourable prognosis are advised to start ART. Several\nprediction models for natural conception have been published of\nwhich the model by Hunault et al., which calculates a prognosis of con-\nception leading to live birth over the first year after the completion of\nthe fertility workup, has been externally validated and subsequently\nimplemented in the national guidelines and clinical practice in the\nNetherlands (\nHunault et al., 2004 ; van der Steeg et al., 2007 ; Leushuis\net al., 2009; NVOG, 2010). A practical drawback of the Hunault mod-\nel is that it cannot give a prediction at later time points when couples\nwho continued expectant management after the fertility workup but\ndid not conceive, return to the clinic. This is because applying the\nHunault model at later time points leads to overestimation due to the\nselection of less fertile couples over time that is not incorporated in\nthe Hunault model (\nvan Eekelen et al., 2017b ).\nVan Eekelen et al. recently developed a dynamic prediction model\nthat accommodates the need for repeated predictions ( van Eekelen\net al., 2017a ). This model comprises the clinical factors female age,\nduration of subfertility (both at completion of the fertility workup),\npercentage of progressively motile sperm, primary or secondary sub-\nfertility and being referred to the fertility clinic by a general practitioner\nor a specialist. In addition to these factors, the model uses as an input\nthe number of menstrual cycles that have passed since the completion\nof the fertility workup, with zero cycles denoting the prediction is\nmade immediately after the workup. The output is the predicted prob-\nability to conceive naturally in the following cycle, leading to ongoing\npregnancy, which can be extended to predict over any given number\nof cycles with a maximum of 2.5 years after the workup (~28 –34\ncycles). When couples return after a period of expectant manage-\nment, the number of cycles that have passed since the workup can be\nchanged to update the predicted probability over subsequent cycles.\nThe model developed by van Eekelen et al. showed promising\nresults in the internal validation, but this in itself is insuf ficient to advise\nclinical implementation since models tend to perform better in the\ncohort they were developed on than in another cohort in which the\nmodel may be applied (\nSteyerberg, 2009).\nThe aim of this study was to externally validate the van Eekelen\nmodel on a large cohort that followed couples for natural conception\nafter registration in the fertility clinic of the Grampian region of\nScotland, UK. This is the largest contemporary cohort following cou-\nples for natural conception, aside from the Dutch cohort on which the\ndynamic model was developed.\nMaterials and Methods\nWe included couples diagnosed with unexplained subfertility residing in\nthe Grampian region of Scotland who registered with the Aberdeen\nFertility Centre (AFC) from 1998 to 2011 (\nPandey et al., 2014 ). Only\npatients from the Grampian region visiting the AFC were selected because\nthere is no other fertility clinic in the region and it was considered import-\nant to have a complete overview of a couple ’s trajectory after the fertility\nworkup, which includes treatment information. We combined the AFC\nregistration database with three other data sources using record-linkage to\nget the complete follow-up for couples from the registration at the AFC\nuntil ongoing pregnancy, treatment or end of the study, which was the 31\nMarch 2012.\nThe AFC database comprises patient characteristics and diagnostic\ninformation. Data entry in the AFC database is validated and checked by\nregular case note audits. First, we record-linked couples registered in the\nAFC database to the centre ’s Assisted Reproduction Unit database which\ncontained dates when treatment was started.\nSecond, we identi fied natural conceptions leading to an ongoing preg-\nnancy by record-linkage of the AFC database with the Aberdeen Maternity\nand Neonatal Databank, which contained gestational age, outcome and\ndelivery date of (early) pregnancies for all women residing in Aberdeen\n2269External validation of a dynamic prediction model\n\nCity District. Third, we performed record-linkage with the national\nScottish Morbidity Records Maternity database for identifying gestational\nage, outcome and delivery date of (early) pregnancies for women who\ndelivered elsewhere in Scotland.\nThe Data Management Team of the University of Aberdeen created a\nnew pseudonomized identi fier for all women by using the Community\nHealth Index identi fier. This new study-speci fic identi fier cannot be used\nto trace back to individuals and was then used by author D.J.M. to record-\nlink the databases within the Grampian Data Safe Haven environment.\nThis process was carried out according to the Standard Operating\nProcedures of the Data Management Team, University of Aberdeen. The\nresulting linked dataset was thus a combination of these four data sources.\nEthical approval was provided by the North of Scotland Research Ethics\nCommittee (reference: 12/NS/0120). Access to the Aberdeen Fertility\nClinic and the Assisted Reproduction Unit databases was approved by the\nAberdeen Fertility Databases Steering Committee. Access to the Aberdeen\nMaternity and Neonatal Databank was approved by the Aberdeen Maternity\nand Neonatal Database Steering C ommittee. Access to the Scottish\nMorbidity Records Maternity database was approved by the Privacy Advisory\nCommittee of Information Services Division Scotland.\nWe de fined unexplained subfertility as couples who tried to conceive\nfor more than 50 weeks before the fertility workup was completed and\nwho had no obvious barriers to conception in terms of uni or bilateral\ntubal occlusion, anovulation, mild or severe endometriosis according to\nthe revised American Society for Reproductive Medicine (ASRM) score\n(\nASRM, 1997 ) or impaired semen quality according to World Health\nOrganization (WHO) criteria ( WHO, 1999 , 2010). We used the gesta-\ntional age at birth or early pregnancy outcome to derive the date of con-\nception and included only pregnancies in the analysis that occurred after\nregistration of the couple at the clinic and that were ongoing, de fined as\nreaching a gestational age of at least 12 weeks. Time to conception was\ncensored at the date of start of IUI, start of IVF, when the woman returned\nto the fertility centre with a different male partner or at the end of study.\nMissing data\nThe date of completion of the fertility workup was not reported in the\nAFC database. The van Eekelen model uses this date as the starting point\nof follow-up, i.e. the time point from which onwards the model can be\nused to estimate a prognosis. The date of registration and the diagnosis\ncategory were available in the database. Judging from local protocols, we\nassumed there were 3 months in between registration and completion of\nthe fertility workup for all couples. In a sensitivity analysis, we repeated the\nvalidation study assuming 1.5 or 4.5 months between registration and\ncompletion of the fertility workup for all couples.\nMenstrual cycle length is used to determine the number of elapsed men-\nstrual cycles since the fertility workup when updating predictions using the\ndynamic prediction model. Cycle length was not recorded in the AFC\ndatabase and we therefore assumed an average cycle length of 28 days for\nall women.\nData on outcomes or at least one prognostic factor were missing for\n~4% of couples; 0.5% on pregnancy or follow-up, 0.5% on female age,\n2.3% on duration of subfertility, 0.5% on primary or secondary subfertility,\n1.9% on the percentage of progressive motile sperm and 0.5% on referral\nstatus. We had no reason to believe that couples with missing data differed\nsystematically from couples with complete data and we analysed couples\nfor which data was complete.\nAnalysis\nWe calculated the predicted probabilities of natural conception over 1\nyear for all couples in the validation cohort using the formula in the\nAppendix of the paper by van Eekelen et al. (\nvan Eekelen et al., 2017a ). To\ntest the model ’s ability to not only predict after the completion of the fer-\ntility workup but also when a couple returns after an unsuccessful period\nof expectant management, we calculated the prognosis at four time points:\ndirectly after completion of the workup, after 0.5, 1 and 1.5 years of\nexpectant management. We evaluated model performance in terms of\ncalibration, i.e. the degree of agreement between observed and predicted\nnatural conception rates, and discrimination, i.e. the ability of the dynamic\nprediction model to distinguish between couples who do conceive and\ncouples who do not conceive.\nTo assess calibration, we first explored whether the overall prediction\nof the model was correct by comparing the average predicted probability\nover a time period with the observed conception rate over that same time\nperiod. This is referred to as calibration-in-the-large and assesses whether\nthe model systematically under or overestimates the observed conception\nrate (\nSteyerberg, 2009).\nSecond, we assessed whether the effects of patient characteristics were\nestimated correctly in three ways: by visuals using calibration plots for risk\ngroups, by calibration within groups with similar patient characteristics and\nby calculating a calibration slope. For the calibration plots, we ordered the\npredicted probabilities of couples and divided them in risk groups with\nsimilar predictions ( n = 135 per risk group). We compared the mean pre-\ndicted chances within these groups with the corresponding observed frac-\ntion of ongoing pregnancy as estimated by the Kaplan –Meier method. We\nvisualized the observed fractions and predicted probabilities per risk group\nin plots and tabulated the absolute differences. In the plots, the 45 ° line\nindicates what would be a perfect agreement between the observed frac-\ntion and average predicted probability within a risk group.\nWe repeated the calibration procedure but instead of grouping based\non predicted risks, we grouped couples based on having similar patient\ncharacteristics. We again compared the mean predicted chances within\nthese groups with the corresponding observed fraction of ongoing preg-\nnancy as estimated by the Kaplan –Meier method and tabulated the results.\nTo calculate the calibration slope, we used the prognostic index (i.e. the\nsum of the multiplication between all patient characteristics and the coef fi-\ncients from the model) as an explanatory variable in a Cox model for each\nof the four evaluated time periods (\nvan Houwelingen, 2000 ). Ideally, the\ncalibration slope is unity, i.e. 1, indicating that the strength of the patient\ncharacteristics in the evaluated model perfectly matches the validation\ndata.\nThird, we used a recalibration procedure as an alternative way to assess\nthe systematic under or overestimation (calibration-in-the-large) and the\nstrength of the patient characteristics (calibration slope) in the model. We\ndid this by using the same coef ficients for the patient characteristics as\nreported by van Eekelen et al. to calculate a prognostic index but\nre-estimated the other parameters of the beta-geometric model in the val-\nidation dataset (\nBongaarts, 1975; Weinberg and Gladen, 1986). The recali-\nbration model re-estimates three parameters, which we compared to\nthose in the van Eekelen model and tested for the difference between the\ntwo using independent samples z-tests. Systematic under or overesti-\nmation was assessed by comparing the intercept and the variance para-\nmeters. The intercept parameter indicates the estimated pregnancy\nchances in the first cycle after the fertility workup and the variance param-\neter indicates how fast the estimated chances decrease over consecutive\nfailed natural cycles. Similarity in strength of the patient characteristics was\nassessed by again calculating a calibration slope parameter, which would\nideally be 1.\nWe assessed discrimination by calculating Harrel ’s c statistic at the four\ntime points, which we compared to those found at internal validation\n(\nHarrell et al., 1996).\nFinally, we explored the range of predicted probabilities at the four time\npoints to see if they facilitate meaningful prognostic strati fication of couples\n(Coppus et al., 2009).\n2270 van Eekelen et al.\n\nAll analyses were conducted in R version 3.4.3 and RStudio ( R Core\nTeam, 2013). A P-value below 0.05 was considered statistically signi ficant.\nResults\nData of 1203 couples were included (Fig. 1). The baseline characteris-\ntics of the couples are shown in Table I.\nIn total, 398 (33%) couples conceived naturally, leading to an\nongoing pregnancy. The median follow-up was 1 year and 3 months\nafter the completion of the workup (average follow-up 2 years and 6\nmonths). The observed rates of natural conception up to 2.5 years are\ndepicted in Fig.\n2 (upper panel). For couples who did not yet conceive\nafter 0.5, 1 or 1.5 years after the completion of the fertility workup,\nthe observed rates of natural conception over the following year are\ndepicted in Fig.\n2 (lower panel). The mean probability of natural con-\nception as predicted by the dynamic model over the course of the first\nyear after the fertility workup was 25%, while the observed fraction\nwas 23% (95%CI 20 –25). For couples who did not conceive after 0.5,\n1 and 1.5 years of expectant management, the mean estimated prob-\nability of conceiving over the course of the following year was esti-\nmated at 18, 14 and 12%. The observed rates were 15% (13 –18%),\n14% (11 –17%) and 12% (9 –15%) for these three time periods,\nrespectively (Fig.\n2, lower panel). Except for the second period during\nwhich the model slightly overestimated the pregnancy chances by 3%\npoints, the mean predicted probabilities fell within their respective\nconfidence limits of the observed rates, indicating good agreement\nbetween the average prediction rendered by the dynamic model and\nthe corresponding observed rate of natural conception.\nThe calibration plots for the four time periods are presented in\nFig.\n3. The dynamic prediction model was well calibrated based on the\nupward trends observed in the four plots, indicating that higher pre-\ndicted probabilities correspond to higher observed rates, and the CIs\nfrom the observed rates which all but one cover the ideal 45 ° line. The\nsecond calibration plot starting at 0.5 year after the fertility workup\nshowed a slight overestimation since all points are below the 45 ° line.\nThe absolute differences between observed fractions and predicted\nprobabilities of natural conception within risk groups are shown in\nTable\nII. This was on average 2.8% points and 9.6 at the highest.\nThe results for the calibration grouping couples by similar character-\nistics are shown in Supplementary data. Results were similar to those\nin the calibration using risk groups, with a slight overestimation in the\ntime periods right after the completion of the fertility workup and after\n0.5 year of expectant management.\nThe calibration slopes using Cox models were 0.86, 1.01, 1.01 and\n0.62 for the four time periods, respectively. None of the correspond-\ning P-values were below 0.05, indicating no statistical evidence for\nunder or overfitting.\nIn the recalibration model, the intercept and variance parameters\nwere similar to those reported by van Eekelen et al. (P = 0.69 and P =\n0.29 for the difference, respectively), indicating similar underlying\nchances of pregnancy in the first cycle after the workup and a similar\ndecrease in chances as time progresses. The slope was 0.90 ( P =\n0.37), indicating a similar strength of patient characteristics in the valid-\nation cohort and no signi ficant difference from 1.\nThe discriminative ability of the model in the validation cohort was\nmoderate and similar to that in the Dutch development cohort, ran-\nging over time from a c statistic of 0.61 (95%CI 0.57 –0.64) in the first\n5466 couples registered between \n1998 and 2011 in the Aberdeen \nFer/g415lity Clinic\nWomen excluded with diagnoses other than \nunexplained subfer/g415lity (n = 3945)\n1521 couples with unexplained \nsubfer/g415lity\n1203 couples in the final analysis \nCouples that did not provide consent for treatment data to be \nused for research (n = 10)\nCouples conceived before comple/g415on of fer/g415lity workup (n = 234)\nCouples excluded with missing outcome data (n = 8)\nCouples excluded with missing predictor values (n = 39)\nCouples excluded that were followed for less than one cycle of \nexpectant management (n = 6)\nCouples with a dura/g415on of subfer/g415lity of 50 weeks or less (n = 21)\nFigure 1 Flow chart of couples with unexplained subfertility who were considered for inclusion in the external validation.\n2271External validation of a dynamic prediction model\n\nyear, 0.62 (95% CI 0.58 –0.67) from 0.5 year, 0.63 (95% CI 0.57 –0.69)\nfrom 1 year, to 0.60 (95% CI 0.52 –0.67) for 1.5 years after the com-\npletion of the fertility workup, all for conceiving in the following year.\nThe c statistics were around 0.61 for all four time periods and seemed\nstable over time.\nThe range of predictions varied between 0% and 55% over the\ncourse of the first year after the fertility workup. After 0.5, 1 and 1.5\nyears of expectant management the ranges narrowed to 0 –43%,\n0–34% and 0 –29% respectively, all over the course of the following\nyear, facilitating a distinction between couples with a good or poor\nprognosis.\nSensitivity analyses\nResults from the two sensitivity analyses are reported online as\nSupplementary data. Theanalysis where we assumed 1.5 months between\nregistration and completion of the fertility workup showed a very good\nperformance of the dynamic prediction model (Supplementary Table SI,\nSupplementary Figs S1 and S2). The analysis assuming 4.5 months between\nregistration and completion of the fertility workup showed similar results\nto the primary analysis but with slightly more overestimation of chances by\nthe model (Supplementary Table SII, Supplementary Figs S3 and S4).\nDiscussion\nWe conducted an external, geographical validation of the van Eekelen\nmodel that can be used for repeated predictions of natural conception\nwhen couples return to the clinic after unsuccessful expectant manage-\nment. The model performed well in a Scottish cohort of couples with\nunexplained subfertility that visited a fertility clinic and the model is\nexpected to be generalizable to other fertility centres and countries\nwhere the procedure of managing unexplained subfertile couples is\ncomparable to the Netherlands and the UK. In addition, the predicted\nprobabilities varied suf ficiently to aid in distinguishing between couples\nwith a good and poor prognosis in terms of natural conception.\nThe data from the AFC was of high quality, registering every unex-\nplained subfertile couple in the Grampian region. All natural concep-\ntions leading to ongoing pregnancy, including after miscarriages and\nother early pregnancy outcomes, were found using data linkage with\nmaternity records. Indications for the fertility workup and de finitions\nof censoring and prognostic characteristics in the Scottish cohort were\nvery similar to the Dutch cohort, aiding comparability (\nvan Eekelen\net al., 2017a).\nThe model was well calibrated, which we consider of higher import-\nance than discrimination since the c statistics can be expected to be\nmoderate due to the limited range of predicted chances in fertility\n(Mol et al., 2005 ; Cook, 2007). This restricts the maximum possible c\nstatistics, even if a model was to produce perfect predictions.\nRecalibration, in which one or more parameters of the prediction\nmodel are updated to accommodate better predictions in a different\ncountry or clinical setting, was not necessary since the recalibration\nmodel showed similar values for all parameters as observed in the\ndevelopment cohort.\nThe main limitation to our study was missing data in terms of dates\nof completion of the fertility workup and menstrual cycle lengths.\nMenstrual cycle length was not considered very in fluential since the\nestimations of the number of cycles per individual are reasonable\napproximations due to the narrow range of possible cycle lengths in\nour selection of unexplained subfertile couples, but we did have to\nmake strong assumptions about the date of completion of the fertility\nworkup. We assumed 3 months between registration and completion\nof the fertility workup, which resulted in ongoing pregnancies before 3\nmonths after registration being excluded. The ‘starting’ moment of\nfollow-up thus differed from the Dutch development cohort since in\nthe latter, the date of last tubal test was used as the end of the\nworkup. Some Dutch clinics did not conduct a visual test of tubal\npatency, i.e. laparoscopy or hysterosalpingography after a negative\nresult for the chlamydia antibody test. In those Dutch clinics, the\nworkup was thus considered as complete earlier after registration\ncompared to the AFC where visual tests of tubal patency are a part of\nthe standard protocol. This may have led to the observed slight over-\nestimation in the first year after the fertility workup and after 0.5 year\nof expectant management but, despite these differences, the dynamic\nmodel was still able to estimate a prognosis that was reasonably accur-\nate on cohort and risk group level. The results from the sensitivity ana-\nlysis assuming 1.5 months between registration and completion of the\nfertility workup were very good because the resulting population more\nclosely resembled that of the Dutch development cohort in which the\nsame average duration was observed between registration and the\nworkup completion. Accordingly, in the analysis assuming 4.5 months\nbetween registration and completion of the fertility workup, the per-\nformance of the dynamic model was poorer because the populations\ndiffered more due to additional selection that occurred.\nThe dynamic model is able to reassess the chance of natural con-\nception after any given period of expectant management from the\ncompletion of the fertility workup onwards. For example, a couple\nwith 1-year secondary subfertility is referred by a general practitioner\nto the fertility clinic of which the woman is 33 years old at the comple-\ntion of the fertility workup and the man has 40% progressive motile\nsperm. Applying our model gives a predicted 38% chance of natural\nconception over the first year after the workup and they might be\nadvised expectant management. When the couple returns to the clinic\nafter 10 unsuccessful months/cycles, reapplying the model yields 25%\nchance over the following year, which is a realistic decrease given they\nhave tried for an additional 10 months. This could be a reason to con-\nsider starting treatment.\n........................................................................................\nTable I Baseline characteristics at completion of the\nfertility workup.\nn = 1203 Mean or\nn\n5th–95th\nPercentile or\n%\nFemale age, in years 33.3 25 –41\nDuration of subfertility,\nin years\n2.7 1.3 –5.6\nPrimary female subfertility 697 58%\nPercentage of progressive motile\nsperm\n51 24 –76\nReferral by secondary care 84 7%\n2272 van Eekelen et al.\n\nBoth the Hunault model and the dynamic model performed well in\nexternal validations, indicating that the added value of the dynamic\nmodel lies in the ability to update predictions at later time points ( van\nEekelen et al., 2017a ). This provides clinicians and patients with infor-\nmation regarding their prognosis of natural conception not only right\nafter the completion of the fertility workup but also when the couple\nreturns after an additional, unsuccessful period of expectant manage-\nment, thus aiding in making clinical decisions at multiple time points\nthroughout a couple’s trajectory. The ability to update predictions also\naids in studies which include the prognosis of natural conception as an\nin- or exclusion criterion, since the prognosis of couples who return\nafter unsuccessful expectant management can be updated accurately,\nleading to the desired homogeneity of the study sample (\nvan den\nBoogaard et al., 2014 ). The dynamic model is flexible and can be used\nto predict over any desired number of menstrual cycles, for instance\nwhen the couple is interested in time periods shorter or longer than 1\nyear. In short, the dynamic model has a wider clinical applicability than\nthe Hunault model and should be the model of choice.\nConclusion\nThe van Eekelen model is a valid and robust tool that is ready to use in\nclinical practice to counsel couples with unexplained subfertility on\ntheir individualized chances of natural conception at various points in\ntime, notably when couples return to the clinic after a period of unsuc-\ncessful expectant management.\nFigure 2 Cumulative chances of natural conception leading to ongoing pregnancy. Cumulative chances after the completion of fertility workup\n(upper panel) and updated chances of natural conception over the course of 1 year at the completion of the fertility workup or 0.5, 1 and 1.5 years\nthereafter (lower panel) in the validation cohort. Percentages are Kaplan –Meier estimates of the observed fraction of natural conception leading to\nongoing pregnancy.\n2273External validation of a dynamic prediction model\n\nSupplementary data\nSupplementary data are available at Human Reproduction online.\nAcknowledgements\nThe authors would like to thank Prof. Egbert te Velde for all of his\nefforts regarding development of the dynamic prediction model and\nthe current validation study. The authors acknowledge the data man-\nagement support of the Grampian Data Safe Haven (DaSH) and the\nassociated financial support of NHS Research Scotland, through NHS\nGrampian investment in the Grampian DaSH. For more information,\nFigure 3 Calibration of the predictions of the dynamic prediction model: predicted vs observed 1-year natural conception rates at fourfixed time points.\n..........................................................................................\nTable II Calibration of the dynamic prediction model by\nrisk groups.\nMean\ndifference\nMax\ndifference\nNumber\nof risk groups\nAfter completion of workup 3.2 9.6 9\nAfter 0.5-year EM 3.0 4.7 7\nAfter 1-year EM 2.1 3.5 5\nAfter 1.5-year EM 2.7 4.5 4\nTotal 2.8 9.6 25\nData are the mean and maximum of the absolute differences (in percentage points)\nbetween predicted and observed 1-year natural conception rates per risk group of n\n= 135, stratified by the elapsed period of expectant management (EM).\n2274 van Eekelen et al.\n\nvisit the DaSH website http://www.abdn.ac.uk/iahs/facilities/grampian-\ndata-safe-haven.php. The authors would like to thank all the staff at\nAberdeen Fertility Clinic for their help with database queries.\nAuthors’ roles\nNvG, MDJ, BS, FvdV, MvW and MJE conceived the study. MDJ per-\nformed the data linkage, storage in the Safe Haven and cleaned the\ndata. RvE, NvG and MJE designed the statistical analysis plan. RvE, MDJ\nand NvG analysed the data. RvE drafted the manuscript. All authors\ncontributed critical revision to the paper and approved the final\nmanuscript.\nFunding\nChief Scientist Office postdoctoral training fellowship in health services\nresearch and health of the public research (ref PDF/12/06). The views\nexpressed here are those of the authors and not necessarily those of\nthe Chief Scientist Office. The funder did not have any role in the study\ndesign; the collection, analysis and interpretation of data; the writing of\nthe report nor the decision to submit the paper for publication.\nConflict of interest\nNone declared.\nReferences\nAboulghar M, Baird D, Collin J, Evers J, Fauser B, Lambalk C, Other A.\nIntrauterine insemination. Hum Reprod Update 2009;15:265–277.\nASRM. Revised American Society for Reproductive Medicine classi fication\nof endometriosis: 1996. Fertil Steril 1997;67:817–821.\nBongaarts J. A method for the estimation of fecundability. Demography\n1975;12:645–660.\nBrandes M, Hamilton CJ, de Bruin JP, Nelen WL, Kremer JA. The relative\ncontribution of IVF to the total ongoing pregnancy rate in a subfertile\ncohort. Hum Reprod 2010;25:118–126.\nCook NR. Use and misuse of the receiver operating characteristic curve in\nrisk prediction. Circulation 2007;115:928–935.\nCoppus SF, van der Veen F, Opmeer BC, Mol BW, Bossuyt PM. Evaluating\nprediction models in reproductive medicine. Hum Reprod 2009;24:\n1774–1778.\nGnoth C, Godehardt D, Godehardt E, Frank-Herrmann P, Freundl G.\nTime to pregnancy: results of the German prospective study and impact\non the management of infertility. Hum Reprod 2003;18:1959–1966.\nHarrell FE Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in\ndeveloping models, evaluating assumptions and adequacy, and measur-\ning and reducing errors. Stat Med 1996;15:361–387.\nHunault CC, Habbema JD, Eijkemans MJ, Collins JA, Evers JL, te Velde ER.\nTwo new prediction rules for spontaneous pregnancy leading to live\nbirth among subfertile couples, based on the synthesis of three previous\nmodels. Hum Reprod 2004;19:2019–2026.\nLeushuis E, van der Steeg JW, Steures P, Bossuyt PM, Eijkemans MJ, van\nder Veen F, Mol BW, Hompes PG. Prediction models in reproductive\nmedicine: a critical appraisal. Hum Reprod Update 2009;15:537–552.\nMol BW, Coppus SF, Van der Veen F, Bossuyt P. ROC-curves are mislead-\ning; calibration is not! Fertil Steril 2005;84:253–254.\nNVOG, Dutch Society for Obstetrics and Gynaecology. Guideline on: subferti-\nlity (2010).\nhttp://bit.ly/1UhuYMV. (5 February 2017, date last accessed).\nPandey S, McLernon DJ, Scotland G, Mollison J, Wordsworth S,\nBhattacharya S. Cost of fertility treatment and live birth outcome in\nwomen of different ages and BMI. Hum Reprod 2014;29:2199–2211.\nPandian Z, Gibreel A, Bhattacharya S. In vitro fertilisation for unexplained\nsubfertility. Cochrane Database Syst Rev 2015;2:Cd003357.\nR Core Team. R: A Language and Environment for Statistical Computing .\nVienna, Austria: R Foundation for Statistical Computing, 2013.\nhttp://\nwww.R-project.org/.\nSteyerberg E. Clinical Prediction Models: A Practical Approach to Development,\nValidation and Updating. New York, USA: Springer, 2009.\nTjon-Kon-Fat RI, Bensdorp AJ, Scholten I, Repping S, van Wely M, Mol\nBW, van der Veen F. IUI and IVF for unexplained subfertility: where did\nwe go wrong? Hum Reprod 2016;31:2665–2667.\nvan den Boogaard NM, Bensdorp AJ, Oude Rengerink K, Barnhart K,\nBhattacharya S, Custers IM, Coutifaris C, Goverde AJ, Guzick DS,\nHughes EC et al . Prognostic pro files and the effectiveness of assisted\nconception: secondary analyses of individual patient data. Hum Reprod\nUpdate 2014;20:141–151.\nvan der Steeg JW, Steures P, Eijkemans MJ, Habbema JD, Hompes PG,\nBroekmans FJ, van Dessel HJ, Bossuyt PM, van der Veen F, Mol BW.\nPregnancy is predictable: a large-scale prospective external validation of\nthe prediction of spontaneous pregnancy in subfertile couples. Hum\nReprod 2007;22:536–542.\nvan Eekelen R, Scholten I, Tjon-Kon-Fat RI, van der Steeg JW, Steures P,\nHompes P, van Wely M, van der Veen F, Mol BW, Eijkemans MJ et al .\nNatural conception: repeated predictions over time. Hum Reprod\n2017a;32:346–353.\nvan Eekelen R, van Geloven N, van Wely M, McLernon DJ, Eijkemans MJ,\nRepping S, Steyerberg EW, Mol BW, Bhattacharya S, van der Veen F.\nConstructing the crystal ball: how to get reliable prognostic information\nfor the management of subfertile couples. Hum Reprod 2017b;32:\n2153–2158.\nvan Houwelingen HC. Validation, calibration, revision and combination of\nprognostic survival models. Stat Med 2000;19:3401–3415.\nVeltman-Verhulst SM, Hughes E, Ayeleke RO, Cohlen BJ. Intra-uterine\ninsemination for unexplained subfertility. Cochrane Database Syst Rev\n2016;2:Cd001838.\nWang X, Chen C, Wang L, Chen D, Guang W, French J. Conception, early\npregnancy loss, and time to clinical pregnancy: a population-based pro-\nspective study. Fertil Steril 2003;79:577–584.\nWeinberg CR, Gladen BC. The beta-geometric distribution applied to\ncomparative fecundability studies. Biometrics 1986;42:547–560.\nWHO. World Health Organization. Laboratory Manual for the Examination of\nHuman Semen and Sperm-Cervical Mucus Interaction , 4th edn. Cambridge:\nCambridge University Press, 1999.\nWHO. World Health Organisation. Laboratory Manual for the Examination\nand Processing of Human Semen , 5th edn. Geneva: World Health\nOrganization, 2010.\n2275External validation of a dynamic prediction model","source_license":"CC0","license_restricted":false}