Using electronic health record data to identify incident uterine fibroids and endometriosis within a large, urban academic medical center: a validation study

Electronic health records (EHRs) present opportunities to study uterine fibroids and endometriosis within diverse populations. When using EHR data, it is important to validate outcome classification via diagnosis codes. We performed a validation study of 3 approaches ([1] International Classification of Diseases-10 (ICD-10) code alone, [2] ICD-10 code + diagnostic procedure, and [3] ICD-10 code + all diagnostic information) to identify incident uterine fibroids and endometriosis patients among n = 750 NYU Langone Health 2016-2023. Chart review was used to determine the true diagnosis status. When using a binary classification system (incident vs nonincident patient), Approaches 2 and 3 had higher positive predictive values (PPVs) for uterine fibroids (0.86 and 0.87 vs 0.78) and for endometriosis (0.70 and 0.73 vs 0.66), but Approach 1 outperformed the other 2 in negative predictive values (NPVs) for both outcomes. When using a 3-level classification system (incident vs prevalent vs disease-free patients), PPV for prevalent patients was low for all approaches, while PPV/NPV of disease-free patients was generally above 0.8. Using ICD-10 codes alone yielded higher NPVs but resulted in lower PPVs compared with the other approaches. Continued validation of uterine fibroids/endometriosis EHR studies is warranted to increase research into these understudied gynecologic conditions.

uterine fibroids, endometriosis, electronic health records, validation

Uterine fibroids and endometriosis are common gynecologic conditions estimated to affect 1 in 4 and 1 in 12 US menstruators. Despite their high prevalences, both are understudied, with only a handful of research cohorts dedicated to the study of these conditions.1-4 Inherent challenges to diagnosing these conditions have created barriers to research. Endometriosis, defined as growth of endometrial tissue outside of the uterus, is difficult to diagnose. While visualization with imaging techniques may allow for a presumptive clinical diagnosis, a definitive clinical gold standard diagnosis requires surgical pathology.5,6 Uterine fibroids, which are benign growths that arise in the uterus, can be diagnosed via common imaging techniques; however their symptoms can be difficult to distinguish from those of endometriosis and adenomyosis, all of which are associated with chronic pelvic pain.7-10 To meet the criteria of clinical gold standard and differential diagnosis, cohorts focused on these conditions are often highly selected populations of patients who receive surgical intervention or are recruited from inpatient healthcare encounters.11-14 However, conditioning eligibility on such a high standard of healthcare engagement may not be representative of the greater underlying population of patients impacted by these conditions, particularly patients with limited healthcare access.15-17 Using electronic health record (EHR) databases to study uterine fibroids and endometriosis presents a promising avenue to conduct large and diverse population health studies of these conditions. Several studies have validated methods to identify uterine fibroid and endometriosis patients from EHR data.1,18-27 They typically take 1 of 2 approaches to identify patients with these outcomes: accept any ICD code for the outcome as a positive indication of the condition (Approach 1)1,20,23-26 or require an ICD code to be accompanied by a relevant diagnostic procedure within some time window (Approach 2).18,21,22,27 Each method has respective benefits and drawbacks. Approach 1 includes as many patients as possible in the diagnosis definition but also might include false positives who received the ICD code for a referral or a suspected diagnosis. Approach 2 is less likely to include false positives, but more likely to exclude patients whose diagnosis was confirmed at a separate health system without linked EHR data. A compromise between these 2 EHR approaches to identify diagnoses would be valuable to the study of these conditions. Therefore, we created a novel approach using additional structured EHR data not incorporated into prior validation methods (Approach 3). To improve the validity of EHR-based identification of uterine fibroids and endometriosis, we aimed to identify incident diagnoses of these conditions in a sample of patients from NYU Langone Health (NYULH), a large academic medical system that serves a diverse population across the greater New York City metropolitan area. Our primary objective was to compare the ability of Approach 1, Approach 2, and our novel Approach 3 to correctly identify incident uterine fibroid and endometriosis patients, using chart review as the standard for true diagnosis status, in a binary classification system (incident vs nonincident patients, the latter includes prevalent and disease-free patients). We repeated the primary objective in an exploratory subanalysis using a 3-level classification system (incident vs prevalent vs disease-free patients). In a secondary analysis, we also assessed the accuracy of using the date of the earliest ICD-10 code to define the true diagnosis date for an incident diagnosis.

Study population The NYULH patients were eligible for inclusion in the cohort if, as of August 1, 2016, they (1) were age 16-50 years, (2) were assigned female at birth with a uterus and at least 1 ovary, (3) had no prior uterine fibroids or endometriosis ICD-10 codes in their EHR going back as far as 2012, and (4) had 2 consecutive years of relevant care after their entry into the cohort. Based on input from clinical advisors, relevant care was defined as encounters with providers in departments that could reasonably lead to screening, diagnosis, or referral for a uterine fibroids or endometriosis diagnosis (primary care, internal medicine, Ob/Gyn, radiology, family practice, and/or urology). This definition was designed to capture patients who regularly seek care at NYULH, while excluding temporary patients (eg, emergency care, nonrelevant specialty care). Patients were excluded if they did not meet any of the eligibility criteria and/or had any disorders of sexual development or primary amenorrhea. Data were extracted on February 27, 2024 as Epic Clarity tables via SQL, then processed in RStudio to derive relevant variables.28 Electronic health record approaches to identify incident patients We aimed to identify incident uterine fibroids and/or endometriosis diagnoses between August 1, 2016 and August 1, 2023. A literature review on existing methods using EHR data was conducted and synthesized. We then sought clinical expertise to apply these methods to NYULH data and drafted Approach 3. Coauthors M.C. and S.S. tested the approach on a subset of 30 charts of pathology-confirmed patients, then made revisions based on the results. Data from the EHR used to determine diagnosis status included the presence of ICD-10 diagnostic codes, diagnosis type (eg, medical history, current problem list, encounter diagnosis, billing code), descriptive name of the diagnosis code (eg, localization of condition, method of diagnosis), presence of related diagnostic imaging or surgical ICD-10 or CPT procedures codes, time between related procedures and diagnosis codes, and continuity of relevant care at NYULH. Using the above information, we defined confirmation criteria for a predicted incident patient in Approach 3 and compare these criteria to the other 2 approaches in Table 1. Finally, we defined potential false-negative diagnoses as a patient with no ICD-10 codes but at least 2 related uterine fibroid/endometriosis symptom codes in their charts.23,29-32 All codes used to identify relevant diagnoses and procedures are listed in Table S1. Table 1. | Approach a | Criteria | Description | Rationale | || |---|---|---|---|---|---| | 1 | 2 | 3 | ||| | X | X | X | Diagnosis code | Presence of an ICD-10 diagnosis code for uterine fibroids and endometriosis | A diagnosis code is indicative of a diagnosis of that condition. | | X | X | Diagnostic procedure confirmation | A diagnosis code is assigned in the 180 days following a diagnostic procedure. | If a clinician assigns a diagnosis code within 6 months following a diagnostic procedure, the procedure was confirmatory of the diagnosis. | | | X | Diagnosis location described | The location of the fibroids/endometriosis (ie, “intramural leiomyoma,” “deep infiltrating endometriosis”) is present in the description of the diagnosis code. | Any diagnosis that specifies the location had a diagnostic procedure done to confirm the diagnosis. | || | X | Diagnosis confirmation method described | The confirmation method of the diagnosis (ie, “diagnosed via laparoscopy”) is present in the description of the diagnosis code. | Any diagnosis that specifies the confirmation method is a true diagnosis. | || | X | Pregnancy-related [uterine fibroids only] | Diagnosis code is related to pregnancy (eg, “leiomyoma complicating pregnancy”). | Uterine fibroids are often found incidentally during pregnancy during prenatal imaging. These patients are often asymptomatic. | || | X | Diagnosis after discontinuity of care | First diagnosis code is assigned after discontinuity of care (2 years without relevant care at NYULH). | Patient received a diagnosis for the condition outside NYULH but then returned to NYULH with the new diagnosis. | Abbreviations: ICD, International Classification of Diseases; NYULH, NYU Langone Health. An X indicates that a given criterion is used to define incident diagnoses for that approach. We used 2 systems to classify patients. The first was a binary classification system: incident vs nonincident. For Approach 1, patients with an ICD-10 code for either condition were classified as incident and all other patients were nonincident. For Approach 2, patients with an ICD-10 code for either condition and a related diagnosis procedure in the preceding 180 days was classified as incident, and all other patients were nonincident. For Approach 3, patients with an ICD-10 code for either condition and who met at least 1 of the other Table 1 criteria were classified as incident, and all other patients were nonincident. While ICD-10 codes were used to identify and exclude prevalent patients from our analysis, there is still the potential for prevalent patients to exist in our data and to be included in the nonincident group (ie, patients with ICD-10 codes within our observation window but no other confirmatory diagnostic information). In an exploratory analysis, we used a 3-level classification system, which separates the nonincident patients into predicted prevalent (ie, unconfirmed patients) and disease-free patients. For Approaches 2 and 3, patients were classified as prevalent if they had an ICD-10 code but did not have the other confirmatory information for an incident case (eg, no diagnostic procedure confirmation or Table 1 criteria, respectively). We assumed if a patient had a diagnosis code but no other diagnostic confirmation, that diagnosis represented a prevalent diagnosis that was just then being added to their record. As Approach 1 only used ICD-10 codes to identify incident diagnoses, there was no way to separate prevalent and disease-free patients, so all nonincident patients were classified as disease-free. In the preliminary chart review, we observed that patients with only 1 billing ICD-10 code for endometriosis in their charts and more than 1 ICD-10 code for adenomyosis were predominantly adenomyosis patients and endometriosis disease-free. Therefore, we decided to classify these patients as nonincident/disease-free for Approach 3. Chart review for true diagnosis status Chart review was performed to determine true diagnosis status. Rather than randomly select charts to review, which would risk not yielding enough uterine fibroid and endometriosis patients, we used the San Diego Approach to Variable Validation.33 This method selects an a priori critical lower bound for the positive predictive value (PPV) and negative predictive value (NPV) of the chart review validation, which in turn determines the sample size needed to review for each predicted group.33 According to this method, with an expected positive PPV and NPV of 0.80 for incident patients and a critical lower bound of 0.70 (estimates from the prior validation literature), we needed at least 60 patients per group to validate the approaches.33 Therefore, we selected 150 predicted incident uterine fibroid patients, 150 predicted incident endometriosis patients, 75 unconfirmed uterine fibroid patients, 75 unconfirmed endometriosis patients, and 150 predicted nonincident patients. Additionally, we reviewed n = 75 charts for patients who were both predicted incident uterine fibroids and endometriosis patients, and n = 75 charts each for patients who were potential false negatives for endometriosis and uterine fibroids. The goal of the chart review was to establish if a patient’s EHR contained evidence of a true diagnosis for either condition and, if so, the earliest date of a true diagnosis recorded in the chart. We defined a true diagnosis as any encounter where the reviewing clinician positively indicated the presence of uterine fibroids or endometriosis (in the clinical notes, problem list, or medical history) or an imaging, surgery, or pathology report that had positive findings for either condition. We acknowledge that this definition of a true diagnosis does not align with the clinical gold standard for endometriosis, but rather indicates true presumptive clinical diagnosis. We felt it necessary to include any positive clinical indication of endometriosis to avoid selectively excluding patients who may choose not to have confirmatory surgery after positive imaging.23 If the earliest instance of either of these clinical indicators in the charts was prior to the start of the cohort observation period (August 1, 2016), then the patient was defined as a true prevalent patient. If the earliest instance was after the start of the cohort observation period, then the patient was defined as a true incident patient. A true disease-free patient was defined as not meeting any of the definitions of a true uterine fibroid or endometriosis diagnosis upon chart review. Finally, we compared the earliest instance of a true diagnosis (herein, true diagnosis date) with the earliest ICD-10 code among correctly identified incident patients. Chart review was conducted by M.C., G.B.M., R.L., and I.R., who abstracted data into an NYULH REDCap project. Each reviewer was trained on a random sample of n = 30 charts. After the first training set, clarifications were made and then tested on another training set of n = 10 charts. Subsequently, sufficient consistency across reviewers was reached and each reviewer was randomly assigned n = 185-195 charts. All charts were reviewed by 1 reviewer after the initial training period, but complex charts were marked for secondary review at the primary reviewer’s discretion. If holistic review of the patient’s chart could not determine their diagnosis status for either condition, they were excluded from the validation analysis. Patient characteristics We collected select characteristics to describe the included patients including, age, self-reported race/ethnicity, and insurance type (private/public). Race and ethnicity were self-reported in patient charts and then combined into a single variable. Insurance type was extracted from the first encounter for patients within our observation window and was grouped into private or public (federal and state) insurance type. Statistical analysis To validate the 3 approaches, we compared the accuracy of the predicted diagnosis status to the true diagnosis status, based on chart review. The PPVs and NPVs were calculated using the following formulas34: where is the number of true positives, is the number of false positives, is the number of true negatives, and is the number of false negatives. For the 3-level classification, we also calculated overall PPV/NPV by averaging the PPV/NPV across the 3 diagnosis classifications.34 To estimate 95% CI, we used the following formula33: where is the SE of the estimate, is the estimate for which we were calculating a 95% CI (PPV or NPV), and is the total number of predicted patients in the respective classification group. For all correctly identified incident patients, we compared the true diagnosis date to the earliest ICD-10 code date and calculated the percent that were within 60 days of each other. We then estimated the impact any discrepancy would have on calculating incident patient person-time (eg, for a time-to-event analysis). For 3 specialty diagnosis groups (predicted incident patient for both conditions and potential false negatives for each), we described the final chart review determination of their diagnosis status. We performed 3 sensitivity analyses related to chart review status. As the true clinical gold standard for uterine fibroids is diagnostic imaging or pathology and for endometriosis is diagnostic pathology, we conducted a sensitivity analysis using these standards to define true diagnoses in the chart review. Additionally, while we excluded the inconclusive chart reviews from our main validation, it was important to explore what impact these exclusions may have had on our validation. Therefore, we conducted a sensitivity analysis for the 2 most extreme cases for these patients, where all inconclusive charts were classified as either (1) true incident patients or (2) true nonincident patients, and described the impact of these 2 scenarios on algorithm performance for each approach.

The eligible cohort consisted of n = 74 478 patients. There were n = 6800 and n = 1672 patients who had ICD-10 codes for uterine fibroids and endometriosis, respectively, and n = 638 patients who had ICD-10 codes for both. The selection process for validation of the EHR approach is visualized in Figure 1. After classifying patients’ predicted diagnosis status using Approach 3, we randomly selected patients for chart review. Of the n = 375 charts selected for each outcome, n = 43 and n = 50 patients for uterine fibroids and endometriosis chart reviews could not be validated, respectively, either because (1) there was no reliable diagnosis date, meaning it was unclear if the patient had a prevalent or incident diagnosis, or (2) it was unclear if the diagnosis was clinically confirmed (eg, imaging report noted “likely endometrioma”). These patients were excluded as we could not confidently assign them incident vs prevalent vs disease-free status. Therefore, the final validation sample size was n = 332 patients for uterine fibroids and n = 325 patients for endometriosis. Select patient characteristics by Approach 3-predicted diagnosis status are found in Table 2. Table 2. | Variable | Total (n = 750) | Incident | Prevalent | Disease-free (n = 150) | Potential false negative (n = 75) | ||| |---|---|---|---|---|---|---|---|---| | Uterine fibroids (n = 150) | Endometriosis (n = 150) | Both (n = 75) | Uterine fibroids (n = 75) | Endometriosis (n = 75) | |||| | Age | |||||||| | ≤20 years | 16 (2.1%) | 0 (0.0%) | 1 (0.7%) | 0 (0.0%) | 1 (1.3%) | 0 (0.0%) | 9 (6.0%) | 5 (6.7%) | | >20-30 years | 151 (20.1%) | 12 (8.0%) | 39 (26.0%) | 2 (2.7%) | 9 (12.0%) | 15 (20.0%) | 52 (34.7%) | 22 (29.3%) | | >30-40 years | 222 (29.6%) | 40 (26.7%) | 48 (32.0%) | 20 (26.7%) | 18 (24.0%) | 32 (42.7%) | 49 (32.7%) | 15 (20.0%) | | >40-50 years | 361 (48.1%) | 98 (65.3%) | 62 (41.3%) | 53 (70.7%) | 47 (62.7%) | 28 (37.3%) | 40 (26.7%) | 33 (44.0%) | | Race/ethnicity | |||||||| | Missing | 85 (11.3%) | 18 (12.0%) | 12 (8.0%) | 6 (8.0%) | 7 (9.3%) | 16 (21.3%) | 20 (13.3%) | 6 (8.0%) | | Hispanic/Latino | 156 (20.8%) | 33 (22.0%) | 36 (24.0%) | 24 (32.0%) | 13 (17.3%) | 6 (8.0%) | 28 (18.7%) | 16 (21.3%) | | NH White | 334 (44.5%) | 56 (37.3%) | 78 (52.0%) | 22 (29.3%) | 22 (29.3%) | 42 (56.0%) | 74 (49.3%) | 40 (53.3%) | | NH Black | 118 (15.7%) | 30 (20.0%) | 11 (7.3%) | 16 (21.3%) | 31 (41.3%) | 9 (12.0%) | 13 (8.7%) | 8 (10.7%) | | NH Asian | 48 (6.4%) | 11 (7.3%) | 10 (6.7%) | 7 (9.3%) | 1 (1.3%) | 2 (2.7%) | 12 (8.0%) | 5 (6.7%) | | Other race/ethnicity | 9 (1.2%) | 2 (1.3%) | 3 (2.0%) | 0 (0.0%) | 1 (1.3%) | 0 (0.0%) | 3 (2.0%) | 0 (0.0%) | | Insurance type | |||||||| | Missing | 46 (6.1%) | 12 (8.0%) | 9 (6.0%) | 8 (10.7%) | 6 (8.0%) | 2 (2.7%) | 4 (2.7%) | 5 (6.7%) | | Private | 500 (66.7%) | 91 (60.7%) | 101 (67.3%) | 43 (57.3%) | 54 (72.0%) | 53 (70.7%) | 110 (73.3%) | 48 (64.0%) | | Public | 204 (27.2%) | 47 (31.3%) | 40 (26.7%) | 24 (32.0%) | 15 (20.0%) | 20 (26.7%) | 36 (24.0%) | 22 (29.3%) | Abbreviation: NH, non-Hispanic. Validation results First, we validated the results of the 3 approaches against the chart review determination while defining a binary diagnosis status as incident vs nonincident patient, the latter including prevalent and disease-free patients. For uterine fibroids, Approach 1 had a lower PPV (0.78 [95% CI, 0.72-0.84] than Approach 2 (0.86 [0.80-0.92]) and 3 (0.87 [0.81-0.93]), but a higher NPV (0.94 [0.90-0.98] vs 0.77 [0.71-0.83] and 0.78 [0.73-0.84]). For endometriosis, a similar pattern emerged across approaches (PPV = 0.66 [0.60-0.73] vs 0.70 [0.61-0.78] vs 0.73 [0.65-0.81]), but PPV performance was lower compared to uterine fibroids overall. Endometriosis NPVs were slightly higher, with Approach 1 outperforming Approaches 2 and 3 again (1.00 [1-1] vs 0.81 [0.76-0.86] and 0.83 [0.78-0.88]). In general, the approaches that incorporated additional structured EHR data were more likely to properly classify incident patients, while Approach 1 was more likely to classify nonincident patients as incident patients (Table 3). Table 3. | Approach | Prevalence (n out of 74 478 [%]) | Predicted diagnosis status | Chart review status | Performance metrics estimate (95% CI) | | |---|---|---|---|---|---| | Incident | Nonincident a | |||| | Uterine fibroids | ||||| | 1 | 6800 (9.1) | Incident | 147 | 42 | PPV = 0.78 (0.72–0.84) NPV = 0.94 (0.90–0.98) | | 67 678 (90.9) | Nonincidentb | 9 | 134 | || | 2 | 5377 (7.2) | Incident | 109 | 18 | PPV = 0.86 (0.80-0.92) NPV = 0.77 (0.71-0.83) | | 69 101 (92.8) | Nonincident | 47 | 158 | || | 3 | 5701 (7.7) | Incident | 112 | 17 | PPV = 0.87 (0.81–0.93) NPV = 0.78 (0.73-0.84) | | 68 777 (92.4) | Nonincident | 44 | 159 | || | Endometriosis | ||||| | 1 | 1672 (2.2) | Incident | 119 | 60 | PPV = 0.66 (0.60–0.73) NPV = 1.00 (1-1) | | 72 806 (97.8) | Nonincident | 0 | 146 | || | 2 | 1342 (1.8) | Incident | 79 | 34 | PPV = 0.70 (0.61–0.78) NPV = 0.81 (0.76-0.86) | | 73 136 (98.2) | Nonincident | 40 | 172 | || | 3 | 1322 (1.8) | Incident | 83 | 31 | PPV = 0.73 (0.65-0.81) NPV = 0.83 (0.78-0.88) | | 73 156 (98.2) | Nonincident | 36 | 175 | Abbreviations: NPV, negative predictive value; PPV, positive predictive value.Approach 1, ICD-10 code-only approach; Approach 2, ICD-10 + diagnostic procedure approach; Approach 3, ICD-10 + all diagnostic information approach; CI, 2-sided binomial exact confidence interval. Nonincident predicted disease status consisted of unconfirmed patients, assumed prevalent, and predicted disease-free patients. Nonincident chart review status consisted of true prevalent and true disease-free patients. We also investigated the 3-level classification system (incident vs prevalent vs disease-free status) (Table S2). Since the definition of a predicted incident patient did not change between the binary and 3-level classification, none of the incident PPV/NPV metrics changed. As there were no prevalent patients for Approach 1, the PPV for prevalent diagnosis group and overall performance could not be calculated. For uterine fibroids, the disease-free PPV was 0.92 [0.88-0.97] using all approaches. The PPV performances were 0.29 [0.18-0.4] and 0.32 [0.2-0.43] for prevalent uterine fibroids using Approaches 2 and 3, respectively, which led to low overall PPV for both approaches. The NPV was nearly 0.90 for all 3 diagnosis types via all approaches, except in Approaches 2 and 3, where the NPV for incident patients was around 0.80. For endometriosis, the disease-free PPV was around 1.00 via all approaches. The PPV performances were 0.20 [0.1-0.29] and 0.25 [0.14-0.36] for prevalent endometriosis using Approaches 2 and 3, respectively. The NPV was lowest for the disease-free group via all approaches (0.78-0.81) and was above 0.80 for all other groups via all approaches. In general, the approaches that incorporated additional structured EHR data more often misclassified incident patients as prevalent patients, while Approach 1 more often misclassified prevalent and disease-free patients as incident patients. Secondary analysis Among the correctly identified incident uterine fibroid and endometriosis patients by Approach 3, we compared the true diagnosis date to the earliest ICD-10 diagnosis date. The earliest ICD-10 code was entered within 60 days of the true diagnosis date for 62.2% (69/111) and 72% (59/82) of correctly identified incident uterine fibroid and endometriosis patients. The next most common scenario was a true diagnosis date earlier than the first ICD-10 code (29.7% [33/111] for uterine fibroids and 20.7% [17/82] for endometriosis). The median difference in diagnosis dates was –1.0 days (IQR: –89.5, 1.0) for uterine fibroids and 0 days (IQR: –41.0, 20.0) for endometriosis. In a time-to-event analysis where the ICD-10 code diagnosis date would serve as the event indicator to calculate person-time to event, using the earliest code would lead to a 22.6% increase in person-time for uterine fibroid patients and an 8.8% increase in person-time for endometriosis patients. Of the n = 75 charts reviewed with ICD-10 codes for both uterine fibroids and endometriosis, we were able to determine the true diagnosis status for n = 56. Of those patients, 39.3% (n = 22) patients were true incident diagnoses for both conditions, 30.4% (n = 17) were true incident uterine fibroid patients, but endometriosis disease-free, and 30.4% (n = 17) were prevalent uterine fibroid and/or prevalent endometriosis patients. No patients in this subgroup were completely disease-free. We reviewed the charts of n = 75 potential false-negative patients for each outcome and found that 80% (n = 60) were truly uterine fibroid disease-free and 100% were truly endometriosis disease free. Of those who had uterine fibroids, 60% (n = 9) were incident and 40% (n = 6) were prevalent uterine fibroid patients. Sensitivity analyses When using the clinical gold standard to define chart review status, there were reductions in PPV for uterine fibroids diagnoses for all 3 approaches, with the greatest for Approach 1, and increases in NPV for Approaches 2 and 3 (Table S3). The same pattern was observed for endometriosis diagnoses, but the PPV values had larger reductions, reaching as low as 0.27 (95% CI, 0.20-0.33) for Approach 1. The sensitivity analysis for the inconclusive chart reviews revealed that, in the most extreme case assuming that all inconclusive charts were true incident patients, Approaches 2 and 3 would have lower NPV for both conditions than estimated in the main validation (Table S3). In other words, more patients with the condition would be inappropriately classified as disease-free in this scenario. In the opposite extreme scenario (all inconclusive charts were assumed to be true nonincident patients), the largest reductions in performance were of the PPV for Approach 1 for both conditions, such that more nonincident patients would inappropriately be classified as having the condition.

The goal of this study was to compare the validity of 3 different approaches to define incident uterine fibroids and endometriosis diagnoses using EHR data among a cohort of NYULH patients. Because EHR data is collected primarily for clinical and not research purposes, attempts to use them for secondary analyses require thoughtful investigation.18,35 For identifying incident cases using a binary classification, additional diagnostic information used in Approach 2 and 3 did increase the PPV performance over Approach 1. Our approach, Approach 3, slightly outperformed Approach 2 in all respects, but CIs were largely overlapping. The performance of all 3 approaches was similar to prior literature for uterine fibroids PPVs (0.78-0.88), but lower with respect to prior endometriosis PPVs (0.80-0.85).1,20,22,36 Prior endometriosis literature validated any diagnosis, prevalent and incident, and also used EHR sources where discontinuity of data may have been less of a concern (ie, claims databases and Kaiser Permanente), both of which could explain the higher PPV and NPV estimates for endometriosis in those studies compared with ours.1,36 For endometriosis, a large proportion of predicted incident patients were disease-free at the time of chart review. This might reflect a higher probability for ICD-10 codes to be assigned for presumed endometriosis that is later ruled out and/or higher rates of underdiagnosis for endometriosis relative to uterine fibroids, both of which could explain the comparatively lower performance across all approaches.6,37,38 To explore disease incidence, causation, and diagnostic timing and trends, it is important to distinguish between incident and prevalent diagnoses. This is particularly important for uterine fibroids and endometriosis, where more epidemiologic research is needed to address the large diagnostic delays for both conditions. Prior research typically assumed that all patients with their first diagnosis code after a selected date were incident diagnoses.1,18,23,24 However, we did not assume all diagnosis codes after the cohort entry date were incident and instead attempted to distinguish between incident and prevalent diagnoses. In the exploratory 3-level classification, the assumption that unconfirmed patients were prevalent patients for Approach 2 and 3 had very low predictive performance. For both outcomes, approximately 60% of unconfirmed patients were true incident patients and would have been inappropriately excluded from the eligible cohort if classified as prevalent patients. On the other hand, in Approach 1, about 17% and 12% of predicted incident cases were prevalent cases based on chart review. Future studies should continue exploring ways to distinguish between incident and prevalent cases in EHR data, but based on our findings, we suggest conducting 1 analysis including all patients and another where potential prevalent cases are excluded. The differences in performance between the approaches have varied implications for research. Approach 1, which considers all patients with ICD-10 codes as incident patients, has a greater chance of classifying patients as incident cases, which means incident cases are least likely to be missed by Approach 1. However, this also means that for Approach 1, prevalent or disease-free patients are also more likely to be classified as incident patients. In an analysis comparing a given exposure with disease risk, this would most likely bias estimates toward the null. Approaches 2 and 3 are more likely to exclude incident patients who are unconfirmed and presume them to be disease-free or prevalent. Patients who lack the structured EHR data necessary to be a confirmed incident patient may have fewer interactions with the healthcare system or seek care from multiple institutions. Selection bias may therefore be introduced if healthcare access and behaviors are associated with the exposure of interest.39-41 While Approaches 2 and 3 performed similarly overall, since Approach 3 included additional structured information instead of only relying on imaging/procedural data like Approach 2, selection bias may be reduced in Approach 3. This is particularly important considering that patients with inconsistent follow-up may be more likely to face barriers to healthcare access due to socioeconomic or insurance circumstances. We also explored the accuracy of diagnostic timing using the first ICD-10 code in the EHR. While most ICD-10 codes were assigned within 60 days of the true diagnosis date, the degree of person-time misattributed to each outcome varied. It appeared more likely for uterine fibroid ICD-10 codes to be assigned after the true diagnosis date, while endometriosis ICD-10 codes were consistently assigned closer to the true diagnosis date. The difference in misattributed person-time by condition may relate to the differing diagnostic gold standards for the 2 conditions and the functional purposes of diagnosis codes. Clinicians assign codes for referral and billing purposes as well as tracking current problems and medical histories.42 Since an endometrioma visualized via imaging would likely prompt a laparoscopy referral, a code might be dropped in a timelier manner. A visualized uterine fibroid is a definitive clinical diagnosis, so the clinician may have less impetus to assign a diagnosis code until the patient returns for care. For patients with codes for both outcomes, we found a large portion were true incident patients for both conditions, another portion were incident for one but not the other, and the final group were prevalent for at least 1. This suggests uterine fibroid and endometriosis co-occurrence may be difficult to assess with certainty using structured EHR data. Additionally, we explored the potential for false negatives for patients without diagnostic codes but with symptom codes related to the conditions. Most were truly disease-free at the time of chart review, with fewer than 20% having uterine fibroid diagnoses, and none having an endometriosis diagnosis. It is possible that some of these patients would eventually be diagnosed with uterine fibroids and/or endometriosis, as we found evidence from clinical notes in n = 16/750 charts indicating that the patient was currently seeking a diagnosis, but at the time of chart review did not yet have one. Strengths and limitations In comparing 3 different EHR approaches, we were able to highlight the trade-offs between broad and increasingly stringent definitions of an incident patient. Our study utilized data from a diverse patient population at a single large academic medical system. While outside the scope of the study, future studies should investigate if performance of these approaches vary by patient characteristics such as age, race, ethnicity, or insurance type. Many NYULH patients seek care from multiple healthcare systems throughout the greater New York City area, so the performance of our algorithms is relevant to other healthcare ecosystems where discontinuity of care or separate EHR systems are common. We also investigated the ability to distinguish incident and prevalent diagnoses and the accuracy of diagnostic dates, neither of which has been done previously to our knowledge. The major limitation of our study was our reliance on chart review as our diagnostic gold standard. For several patients, we were unable to determine their true disease status from their charts alone due to inconclusive reports, clinical notes with no dates, unclear clinical notes, etc. Based on our sensitivity analyses, if most patients with inconclusive chart reviews were true incident diagnoses, we likely overestimated the NPV for Approaches 2 and 3. On the other hand, if they were more likely to be nonincident diagnoses, we likely overestimated the PPV for Approach 1. While the reality is likely somewhere in between these 2 sensitivity analyses, researchers who use these algorithms should keep this limitation of our validation in mind as it pertains to outcome misclassification for their study. In an ideal study, every patient would be conclusively screened for uterine fibroids and endometriosis to determine true status. As these conditions are widely under diagnosed, we might expect lower false-positive and higher false-negative numbers using this gold standard, which would suggest that the true PPV may be higher and the true NPV lower than what we estimated. However, in practice, such a study design could introduce selection bias based on which patients agree to and can afford to undergo diagnostic procedures. We also included presumed clinical diagnoses of true endometriosis patients who received imaging or surgery without pathology. When we conducted a sensitivity analysis only relying on clinical gold standard information for true diagnoses, we observed substantially lower PPV values for endometriosis for all approaches. This could reflect the limitation of only having access to one medical institution for our chart review and, therefore, potentially missing surgical pathology information from medical encounters outside of NYULH. Alternatively, this could reflect the reality that many patients receive endometriosis diagnostic codes, imaging studies, and clinical notes without undergoing surgical confirmation, a common scenario for patients who are satisfied to receive endometriosis symptom management on the basis of a presumed clinical diagnosis.6,43 In determining the most suitable reference for validating their method of EHR case diagnosis, researchers should consider the diagnostic coding practices of the medical institution, continuity of care for patients within the EHR used, and finally, which is more important for their research question: a patient’s experience of living with an endometriosis diagnosis or whether a patient meets the clinical gold standards for an endometriosis diagnosis.

It is challenging to accurately identify incident patients with chronic diseases that have opaque diagnostic journeys and heterogenous presentations from EHR data. However, EHR-based observational studies offer unprecedented research opportunities in large and potentially diverse study populations over long observation periods. While all 3 approaches reliably identified incident patients, they were less accurate at separating incident patients from prevalent and disease-free patients. Future research should continue to explore methods to improve the use of EHR data in observational research of uterine fibroids and endometriosis. Supplementary Material Acknowledgments We would like to acknowledge the collaboration and guidance from several colleagues including Dr. L. Thorpe, Dr. T. Janevic, A. Fair, C. Hong, Dr. E. Iturrate, and O. Yu. Contributor Information Mia Charifson, Department of Population Health, New York University Grossman School of Medicine, New York, NY 10016, United States. Geidily Beaton-Mata, Department of Obstetrics and Gynecology, New York University Langone Health, New York, NY 10016, United States. Robyn Lipschultz, New York University Grossman School of Medicine, New York, NY 10016, United States. India Robinson, New York University Grossman School of Medicine, New York, NY 10016, United States. Simone A Sasse, Department of Obstetrics and Gynecology, New York University Langone Health, New York, NY 10016, United States; Department of Obstetrics and Gynecology, Stanford Medicine, Palo Alto, CA 94305, United States. Hye-Chun Hur, Department of Obstetrics and Gynecology, New York University Langone Health, New York, NY 10016, United States. Shilpi-Mehta S Lee, Department of Obstetrics and Gynecology, New York University Langone Health, New York, NY 10016, United States. Erinn M Hade, Department of Population Health, New York University Grossman School of Medicine, New York, NY 10016, United States. Linda G Kahn, Department of Population Health, New York University Grossman School of Medicine, New York, NY 10016, United States; Department of Pediatrics, New York University Grossman School of Medicine, New York, NY 10016, United States. Supplementary material Supplementary material is available at the American Journal of Epidemiology online. Funding M.C. acknowledges funding from the National Research Foundation Graduate Research Fellowship Program (20-A0-00-1005789). The project described was supported by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001445, via the Clinical Translational Science Institute at New York University Grossman School of Medicine. L.G.K. acknowledges support from the National Institutes of Health/National Institute of Environmental Health Sciences (R00ES030403). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Conflict of interest None to declare. Data availability Data may be available upon request to the corresponding author.

- 1. Christ JP, Yu O, Schulze-Rath R, et al. Incidence, prevalence, and trends in endometriosis diagnosis: a United States population-based study from 2006 to 2015. Am J Obstet Gynecol. 2021;225(5):500.e1-500.e9. 10.1016/j.ajog.2021.06.067 [DOI] [PubMed] [Google Scholar] - 2. Parazzini F, Esposito G, Tozzi L, et al. Epidemiology of endometriosis and its comorbidities. Eur J Obstet Gynecol Reprod Biol. 2017;209:3-7. 10.1016/j.ejogrb.2016.04.021 [DOI] [PubMed] [Google Scholar] - 3. Marshall LM, Spiegelman D, Barbieri RL, et al. Variation in the incidence of uterine leiomyoma among premenopausal women by age and race. Obstet Gynecol. 1997;90(6):967-973. 10.1016/s0029-7844(97)00534-6 [DOI] [PubMed] [Google Scholar] - 4. Smarr MM, Avakian M, Lopez AR, et al. Broadening the environmental lens to include social and structural determinants of women's health disparities. Environ Health Perspect. 2024;132(1):015002. 10.1289/EHP12996 [DOI] [PMC free article] [PubMed] [Google Scholar] - 5. Grundström H, Kjølhede P, Berterö C, et al. “A challenge”–healthcare professionals' experiences when meeting women with symptoms that might indicate endometriosis. Sex Reprod Healthc. 2016;7:65-69. 10.1016/j.srhc.2015.11.003 [DOI] [PubMed] [Google Scholar] - 6. As-Sanie S, Black R, Giudice LC, et al. Assessing research gaps and unmet needs in endometriosis. Am J Obstet Gynecol. 2019;221(2):86-94. 10.1016/j.ajog.2019.02.033 [DOI] [PubMed] [Google Scholar] - 7. Fantasia HC, Harris AL. Overview of the diagnosis and management of uterine fibroids. Women's Healthcare. 2020;8(5):38-44. https://www.npwomenshealthcare.com/wp-content/uploads/2020/10/WHC_Oct2020_FibroidsFeature.pdf [Google Scholar] - 8. Antero MF, O'Sullivan D, Mandavilli S, et al. High prevalence of endometriosis in patients with histologically proven adenomyosis. Fertil Steril. 2017;107(3):e46. 10.1016/j.fertnstert.2017.02.088 [DOI] [Google Scholar] - 9. Nezhat C, Li A, Abed S, et al. Strong association between endometriosis and symptomatic leiomyomas. JSLS. 2016;20(3):e2016.00053. 10.4293/JSLS.2016.00053 [DOI] [PMC free article] [PubMed] [Google Scholar] - 10. Jean-Baptiste H, Tetrokalashvili M, Williams T, et al. Characteristics associated with postoperative diagnosis of adenomyosis or combined adenomyosis with fibroids. Int J Gynecol Obstet. 2013;122(2):112-114. 10.1016/j.ijgo.2013.03.009 [DOI] [PubMed] [Google Scholar] - 11. Sarria-Santamera A, Orazumbekova B, Terzic M, et al. Systematic review and meta-analysis of incidence and prevalence of endometriosis. Healthcare. 2020;9(1):29-29. 10.3390/healthcare9010029 [DOI] [PMC free article] [PubMed] [Google Scholar] - 12. Stewart EA, Cookson CL, Gandolfo RA, et al. Epidemiology of uterine fibroids: a systematic review. BJOG. 2017;124(10):1501-1512. 10.1111/1471-0528.14640 [DOI] [PubMed] [Google Scholar] - 13. Holt VL, Weiss NS. Recommendations for the design of epidemiologic studies of endometriosis. Epidemiology. 2000;11(6):654-659. 10.1097/00001648-200011000-00007 [DOI] [PubMed] [Google Scholar] - 14. Harris HR, Petrick JL, Rosenberg L. The epidemiology of uterine fibroids: where do we go from here? Fertil Steril. 2022;117(4):841-842. 10.1016/j.fertnstert.2022.01.037 [DOI] [PubMed] [Google Scholar] - 15. Li HJ, Song Y, Cho YK. Racial disparities in access to diagnostic laparoscopy for endometriosis: diagnostic delay, ED visits, and pre-operative indications. J Minim Invasive Gynecol. 2021;28(11):S134-S134. 10.1016/j.jmig.2021.09.241 [DOI] [Google Scholar] - 16. Fourquet J, Zavala DE, Missmer S, et al. Disparities in healthcare services in women with endometriosis with public vs private health insurance. Am J Obstet Gynecol. 2019;221(623):623.e1-623.e11. 10.1016/j.ajog.2019.06.020 [DOI] [PubMed] [Google Scholar] - 17. Orellana M et al. “In our community, we normalize pain”: discussions around menstruation and uterine fibroids with Black women and Latinas. BMC Womens Health. 2024;24(1):1-7. 10.1186/s12905-024-03008-z [DOI] [PMC free article] [PubMed] [Google Scholar] - 18. Feingold-Link L, Edwards TL, Jones S, et al. Enhancing uterine fibroid research through utilization of biorepositories linked to electronic medical record data. J Womens Health. 2014;23(12):1027-1032. 10.1089/jwh.2014.4978 [DOI] [PMC free article] [PubMed] [Google Scholar] - 19. Hoffman SR, Vines AI, Halladay JR, et al. Optimizing research in symptomatic uterine fibroids with development of a computable phenotype for use with electronic health records. Am J Obstet Gynecol. 2018;218(610):610.e1-610.e7. 10.1016/j.ajog.2018.02.002 [DOI] [PMC free article] [PubMed] [Google Scholar] - 20. Martín-Merino E, Wallander MA, Andersson S, et al. The reporting and diagnosis of uterine fibroids in the UK: an observational study. BMC Womens Health. 2016;16(1):1-6. 10.1186/s12905-016-0320-8 [DOI] [PMC free article] [PubMed] [Google Scholar] - 21. McKillop, M., Mamykina, L. & Elhadad, N. Designing in the dark: eliciting self-tracking dimensions for understanding enigmatic disease. Conference on Human Factors in Computing Systems–Proceedings . 2018-April(2018). - 22. Yu O, Reed SD, Schulze-Rath R, et al. Identification of incident uterine fibroids using electronic medical record data. eGEMs. 2019;7(1):5-5. 10.5334/egems.264 [DOI] [PMC free article] [PubMed] [Google Scholar] - 23. Abbas S, Ihle P, Köster I, et al. Prevalence and incidence of diagnosed endometriosis and risk of endometriosis in patients with endometriosis-related symptoms: findings from a statutory health insurance-based cohort in Germany. Eur J Obstet Gynecol Reprod Biol. 2012;160(1):79-83. 10.1016/j.ejogrb.2011.09.041 [DOI] [PubMed] [Google Scholar] - 24. Burton C, Iversen L, Bhattacharya S, et al. Pointers to earlier diagnosis of endometriosis: a nested case-control study using primary care electronic health records. Br J Gen Pract. 2017;67(665):e816-e823. 10.3399/bjgp17X693497 [DOI] [PMC free article] [PubMed] [Google Scholar] - 25. Eisenberg VH, Weil C, Chodick G, et al. Epidemiology of endometriosis: a large population-based database study from a healthcare provider with 2 million members. BJOG. 2018;125(1):55-62. 10.1111/1471-0528.14711 [DOI] [PubMed] [Google Scholar] - 26. Sherifi SK, Odahowski CL, Castillo HL. Uterine leiomyomata claim rate estimates and demographic characteristics by county. Florida, 2010–2019. Women Health. 2024;64(1):75-89. 10.1080/03630242.2023.2296524 [DOI] [PubMed] [Google Scholar] - 27. Denny KJ, Barnes WA, Amdur R, et al. Variations in endometriosis location and surgical outcomes by race. J Minim Invasive Gynecol. 2021;28(11):S159-S159. 10.1016/j.jmig.2021.09.328 [DOI] [Google Scholar] - 28. Team, R.C . R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2023. [Google Scholar] - 29. Arruda MS, Petta CA, Abrão MS, et al. Time elapsed from onset of symptoms to diagnosis of endometriosis in a cohort study of Brazilian women. Hum Reprod. 2003;18(4):756-759. 10.1093/humrep/deg136 [DOI] [PubMed] [Google Scholar] - 30. Borah BJ, Nicholson WK, Bradley L, et al. The impact of uterine leiomyomas: a national survey of affected women. Am J Obstet Gynecol. 2013;209(4):319.e1-e20. 10.1016/j.ajog.2013.07.017 [DOI] [PMC free article] [PubMed] [Google Scholar] - 31. Ghai V, Jan H, Shakir F, et al. Diagnostic delay for superficial and deep endometriosis in the United Kingdom. J Obstet Gynaecol. 2019;40(1):83-89. 10.1080/01443615.2019.1603217 [DOI] [PubMed] [Google Scholar] - 32. Pugsley Z, Ballard K. Management of endometriosis in general practice: the pathway to diagnosis. Br J Gen Pract. 2007;57(539):470-476. https://pmc.ncbi.nlm.nih.gov/articles/PMC2078174/ [PMC free article] [PubMed] [Google Scholar] - 33. Liu L, Bustamante R, Earles A, et al. A strategy for validation of variables derived from large-scale electronic health record data. J Biomed Inform. 2021;121:103879-103879. 10.1016/j.jbi.2021.103879 [DOI] [PMC free article] [PubMed] [Google Scholar] - 34. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102. 10.1136/bmj.309.6947.102 [DOI] [PMC free article] [PubMed] [Google Scholar] - 35. Penrod N, Okeh C, Velez Edwards DR, et al. Leveraging electronic health record data for endometriosis research. Front Digit Health. 2023;5:1150687-1150687. 10.3389/fdgth.2023.1150687 [DOI] [PMC free article] [PubMed] [Google Scholar] - 36. McKillop MM. Phenotyping Endometriosis from Observational Health Data. Doctorate of Philosophy Columbia Univeristy, ProQuest; 2019. [Google Scholar] - 37. Agarwal SK, Chapron C, Giudice LC, et al. Clinical diagnosis of endometriosis: a call to action. Am J Obstet Gynecol. 2019;220(354):354.e1-354.e12. 10.1016/j.ajog.2018.12.039 [DOI] [PubMed] [Google Scholar] - 38. Hadfield R, Mardon H, Barlow D, et al. Delay in the diagnosis of endometriosis: a survey of women from the USA and the UK. Hum Reprod. 1996;11(4):878-880. 10.1093/oxfordjournals.humrep.a019270 [DOI] [PubMed] [Google Scholar] - 39. Goldstein BA, Bhavsar NA, Phelan M, et al. Controlling for informed presence bias due to the number of health encounters in an electronic health record. Am J Epidemiol. 2016;184(11):847-855. 10.1093/aje/kww112 [DOI] [PMC free article] [PubMed] [Google Scholar] - 40. Phelan M, Bhavsar NA, Goldstein BA. Illustrating informed presence bias in electronic health records data: how patient interactions with a health system can impact inference. eGEMs. 2017;5(1):22-22. 10.5334/egems.243 [DOI] [PMC free article] [PubMed] [Google Scholar] - 41. Vassy JL, Ho YL, Honerlaw J, et al. Yield and bias in defining a cohort study baseline from electronic health record data. J Biomed Inform. 2018;78(54-54):59. 10.1016/j.jbi.2017.12.017 [DOI] [PMC free article] [PubMed] [Google Scholar] - 42. Scheid LM, Brown LS, Clark C, et al. Data electronically extracted from the electronic health record require validation. J Perinatol. 2019;39(3):468-474. 10.1038/s41372-018-0311-8 [DOI] [PubMed] [Google Scholar] - 43. Denny E, Mann CH. Endometriosis and the primary care consultation. Eur J Obstet Gynecol Reprod Biol. 2008;139(1):111-115. 10.1016/j.ejogrb.2007.10.006 [DOI] [PubMed] [Google Scholar] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials Data Availability Statement Data may be available upon request to the corresponding author.