Identification of Cohorts with Inflammatory Bowel Disease Amidst Fragmented Clinical Databases via Machine Learning

Identification of Cohorts with Inflammatory Bowel Disease Amidst Fragmented Clinical Databases via Machine Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Identification of Cohorts with Inflammatory Bowel Disease Amidst Fragmented Clinical Databases via Machine Learning Matthew Stammers, Stephanie Sartain, Fraser Cummings, Christopher Kipps, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6298636/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 13 Aug, 2025 Read the published version in Digestive Diseases and Sciences → Version 1 posted 8 You are reading this latest preprint version Abstract Introduction : Inflammatory bowel disease (IBD) cohort identification typically relies primarily on read/billing codes, which may miss some patients. Attempts have been made to add medication records and other datasets to improve the cohort capture. However, a complete picture cannot typically be obtained because of database fragmentation and missingness. This study used novel cohort retrieval methods to identify the total IBD cohort from a large university teaching hospital with a specialist intestinal failure unit. Methods : Between 2008 and 2023, 11 clinical databases (ICD10 codes, OPCS4 codes, clinician-entry IBD registry, IBD patient portal, prescriptions, biochemistry, flare line calls, clinic appointments, endoscopy, histopathology, and clinic letters) were identified as having the potential to help identify local IBD patients. A gold-standard validation cohort was created through a manual chart review. A regex string search for normalised IBD terms was used on the three free-text databases (endoscopy, histopathology, and clinic letters) to identify patients more likely to have IBD. The 11 databases were compared statistically to assess cardinality and Jaccard Similarity in order to derive informed estimates of the total IBD population. A penalised logistic regression (LR) classifier was trained on 70% of the data and validated against a 30% holdout set to individually identify IBD patients. Results : The gold-standard validation cohort comprised 2,800 patients: 2,180(78%) with IBD and 619(22%) non-IBD cases. The precision for IBD ranged from 0.75-1 to 0.18-1. All the databases contained unique patients that were not covered by the Casemix ICD-10 database. The Jaccard similarity estimation predicted 18,594, but this represents an overestimation. The penalised LR model (AUROC: 0.85 - Validation set) confidently identified 8,060 patients with IBD (threshold: 0.586), although at lower thresholds (0.25), the model identified 12,760 patients with a higher recall of 0.92. By combining the true-positive cases from the LR model with likely true-positive IBD clinic letters, a final estimate of 12,998 patients with IBD was obtained. True positives from ICD 10 codes combined with medication (n = 8,048) covered only 61.6% of the total local IBD population, indicating that the present methods missed up to 38.4% of IBD patients. Conclusion : Diagnostic billing codes and medication data alone cannot accurately identify complete IBD cohorts. A multimodal cross-database model can partially compensate for this deficit. To improve this situation, more robust natural language processing (NLP)-based identification mechanisms are required to improve IBD cohort identification. Inflammatory bowel disease Cohort identification Data fragmentation Algorithms Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Introduction 1.2.1 Key Messages: · What is already known on this topic : IBD patient cohorts can be identified using billing/read/clinical codes and medication data. · What this study adds : Nine additional databases containing unique IBD patients are identified, and retrieval strategies to overcome database fragmentation demonstrate that medication data and ICD-10 codes only cover ~62% of the total cohort. · How this study might affect research, practice, or policy : Significant numbers of IBD patients are missing from population and local-level cohort identification exercises. IBD prevalence is, therefore, likely systematically underestimated. More advanced cohort identification mechanisms will be required in future. 1.2.2 A Primer on Clinical Cohort Identification Clinical cohort identification challenges vary substantially in difficulty by domain, ranging from comparatively simple conditions like chronic kidney disease (CKD), diagnosable purely by the estimated glomerular filtration rate(eGFR) over time(1), to more challenging conditions such as age-related macular degeneration (AMD) and its subtypes like geographic atrophy (GA), which can take an expert up to half an hour to diagnose visually from a scan, and until recently, had only a single ICD-10 umbrella code (H35.3)(2). Ulcerative colitis (UC), Crohn’s disease (CD), inflammatory bowel disease unclassified (IBDU), and microscopic colitis are chronic inflammatory conditions collectively known as inflammatory bowel disease (IBD)(3). They are diagnosed using a combination of clinical, biochemical, genetic, radiological, endoscopic, and histopathological tests(4). The best estimates suggest that the number of patients with IBD is increasing, with over 700,000 patients currently affected in the UK(5). The cost of care for IBD is demonstrably high, with annual per-patient treatment costs of £3084 and £6146 for UC and CD, respectively(3). IBD is an interesting test case for a clinical cohort identification study because it is relatively common, with existing national registries and national cohorts(7–10), suggesting a degree of national cohort identification confidence. However, there is no definitive single diagnostic test for the condition, and misdiagnosis may be as high as 10%(11). Overdiagnosis may lead to medically induced injury, such as overtreatment with medications with significant side effects and underdiagnosis risk complications directly related to the disease. In one study, 14.7% of patients were lost to follow-up and 61% subsequently developed a disease flare(12). Population-level health studies rely on diagnostic billing codes such as ICD-10. In the context of IBD, it has been claimed that diagnostic clinical codes are up to 97% accurate in identifying IBD clinical cohorts(13,14). However, this does not fit with real-world experience or other evidence that has consistently shown billing codes to be inaccurate in various clinical contexts(15–20). In a Danish study conducted in 2020, only 51% of the single-coded CD cohort and 54% of the single-coded UC cohort were accurate(21). In another study from Scotland, the use of a capture-recapture methodology involving medication data identified 427 previously missed IBD cases(22). To address this problem, baseline natural language processing (NLP) systems in gastroenterology are at a relatively early stage(23). This foundational problem must be solved before more advanced NLP systems such as large language models can be successfully leveraged. This study highlights the complexities of identifying an IBD clinical cohort, even within a single institution, using reliable source data collected over 15 years. It also highlights the risks of data fragmentation and warns against the assumption that prior gold standards such as ICD-10 codes are sufficiently robust to be relied upon. Better mechanisms are required to reliably identify patients with IBD (and, by extension, other disease cohorts). 1.2.3 Aim This study aimed to estimate the size of a local IBD cohort across disparate fragmented databases within a single institution over the past 15 years. 1.2.4 Objectives 1. Validate a gold-standard IBD cohort within our institution. 2. Uncover database patient distributions and usefulness for IBD cohort identification. 3. Explore statistical relationships/comparisons between databases. 4. Estimate the total size of the local 15-year IBD cohort using this knowledge. Methods 1.3.1 Inclusion Criteria All adults aged >18 years at the time of their first elective non-two-week wait referral (2WW) to our trust for gastroenterology specialist care between 2007 and 2023 who did not opt out of using their clinical data for research in secondary care were included in the study. The year 2007 was selected as the start of the study because this was the year the electronic patient administration system (PAS) was installed and digitised trackable referral data began to accrue. 1.3.2 Clinical Ethics & Checklist The Wessex REC and HRA provided research ethics board approval for this study (IC-IBD -23/SC/0152) on 16/05/2023. The study followed the original transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) checklist(24), as tracked by page numbers in Supplement 1 . 1.3.3 Datasets Internal databases hosted at our institution were examined and screened to identify a suspected IBD cohort. The 11 separate databases broadly fell into four categories. 1.3.3.1 Coded Databases 1. ICD-10 Diagnosis Codes (casemix). Registered IBD clinical ICD-10 codes include: ('K50.0', 'K50.1', 'K50.8', 'K50.9', 'K50.X', 'K51.0', 'K51.1', 'K51.2', 'K51.3', 'K51.5', 'K51.8', 'K51.9', 'K51.X', 'K52.3', 'K52.9') as per the recommended RCP organisational IBD audit (25). 2. OPCS-4 Surgical Procedure Codes . OPCS-4 codes that could represent IBD surgery (G58 – G82 & H01 – H56) as per the recommended RCP organisational IBD audit (25). The definitions of these codes are provided in Supplement 2 . 1.3.3.2 Registry Databases 3. Electronic Patient Record (EPR) IBD Registry - The hospital integrated clinical support system (HICSS TM ) IBD Patient Module. In this module, patients can only be registered with a diagnosis by a gastroenterology consultant or specialist registrar/fellow. 4. IBD Patient Portal - (My Medical Record TM ): Patients can self-register for the platform but can only be added to the IBD pathway by a clinician. 1.3.3.3 Event Databases 5. Appointments: Patient Appointments with Gastroenterology. This filter only flagged patients explicitly seen by an IBD specialist as suspected of having IBD. 6. Lab Biochemistry: Faecal calprotectins were recorded in our laboratory. Only patients with levels > 50ug/L (the lab upper limit of normal) were suspected to have IBD. 7. Flare Line: Recorded calls to the nurse-led flare line. As this line also locally covers coeliac disease and other queries, only those with a recorded diagnosis of IBD on the call template were considered to have suspected IBD. 8. Cytokine Modulator Prescriptions: Any patient with a documented prescription for a cytokine modulator under Gastroenterology on the EPR was suspected to have IBD. 1.3.3.4 Free-Text Databases The screening process for these databases is explained in Free Text Normalisation & Handling 9. Gastroenterology Clinic Letters 10. Endoscopy Records 11. Gastrointestinal Histopathology Records 1.3.4 Primary & Secondary Outcome The primary outcome of interest was the estimated number of patients in the IBD cohort. The secondary outcomes of interest included precision (PPV), recall(sensitivity), and F1-score for each database and model to detect IBD diagnoses correctly against the gold-standard cohort, database cardinality, and algorithm fairness. 1.3.5 Strongly Supervised Gold-Standard Validation Cohort Derivation The validation cohort was randomly selected from a larger group of patients within at least two of the 11 validation databases listed above. Therefore, the validation cohort skews towards IBD cases, and the control cases present in the validation cohort will be more difficult for algorithms to discriminate from IBD cases than they would among the general population. A strongly supervised randomised validation cohort was selected to maximise the robustness of the validation challenge. A team of three junior doctors, led by a gastroenterology registrar (SS), performed manual chart reviews of this randomly selected cohort. Each participant was blinded to the efforts of others. They were supervised by a consultant (MS) who oversaw and re-checked each validation unblinding. 1.3.5.1 Validation Sample Size Calculation This study aimed not only to calculate the total size of the cohort but also to build a model to identify individuals using a logistic regression classifier. Therefore, rather than simply relying on only 20 events per variable (26)(EPV) to calculate the sample size, the sample size estimation method described by Pate and Riley was used(27) because this method has been validated in clinical contexts. The formula for binary predictions follows the logic explained below ( Fig 1 ): [FIGURE 1] A binary classification model with an expected Cox-Snell r-squared value of less than 0.05 was chosen because the discriminative value of each clinical dataset was expected to be low. Up to 11 predictor parameters were fed into the model, as there were 11 databases. Allowable validation shrinkage was set at 0.9 among a target population of gastroenterology referrals, where we already know that at least 16.5% will have IBD(28). Based on this calculation, the gold-standard validation cohort required to train a model must be at least 1730 patients with a corresponding validation cohort of at least 519 and a training cohort of at least 1211. A base cohort of > 50% was derived to ensure sufficient scale and power for the study. The complete Python code for this calculation is provided as an open source for transparency. 1.3.6 Free Text Normalisation & Handling All free-text documents were extracted in native format from the EPR and converted into simple strings. The Unified Medical Language System (UMLS)(29) (MRCONSO meta-thesaurus) was then used to remap IBD synonyms across all free text to create normalised terms for IBD in the following list: ["Ulcerative Colitis", "Crohn's Disease", "IBD" (includes IBD-U), "Inflammatory Bowel Disease", "Proctitis", "Collagenous Colitis", "Microscopic Colitis", "Lymphocytic Colitis"]. 1.3.6.1 Regex Natural Language Processing (NLP) Model To flag free-text documents as suggestive of IBD, a simple regex-based NLP model was utilised to match the strings according to the following five regular expressions and associated IBD-related lower case terms: · (r‘ *olitis’ (ulcerative colitis, microscopic colitis, lymphocytic colitis, microscopic colitis, pan-colitis, and inflammatory colitis) · r‘*rohn*’ (crohn’s, crohn’s disease) · r‘*octitis’ (proctitis) · r‘*flammatory bowel disease’ (inflammatory bowel disease) · r‘ibd’ (ibd, ibd-u, ibdu) 1.3.7 Statistical Analysis Missing values were imputed as 0 (not-IBD) to maximise the chances of successfully examining the effects of database gaps on cohort identification in real-world practice. This causes the logistic regression (LR) model to underestimate the total cohort size but has the benefit of reducing the false positive rate. However, the Jaccard index-based cohort estimation system is unaffected and continues to create an overestimation. This allowed the upper and lower cohort size estimates to be established quickly. Means and medians were calculated as appropriate, depending on skewness using 95%CI or 25%/75% quantiles, as appropriate. The kurtosis was also assessed. The 95% confidence intervals were computed using 1000-fold bootstrapping. 1.3.7.1 Jaccard Similarity Index The Jaccard similarity index(30) ( Fig 2. ) was used to statistically compare overlaps between database content. This is defined as the intersection size divided by the union size of the two sample sets. [FIGURE 2] Jaccard index thresholding is somewhat subjective and dependent on both the context and task. However, at a basic level, when comparing databases in this context, a level of > 0.75 would typically be considered high and a level of < 0.35 low(31,32). 1.3.7.2 Plotting and Statistics Plotting was performed using Python 3.10.10 with packages matplotlib(33), seaborn(34) and bokeh(35). Table 1 lists the performance metrics of interest. Table 1 : Performance metrics used in this study Term Description Accuracy The percentage of results that were correct among all results from the system. Calc: (TP+TN)/(TP+FP+TN+FN). Precision (PPV) Also called positive predictive value (PPV), it is the percentage of true positive results among all results that the system flagged as positive. Calc: TP/(TP+FP). Negative Predictive Value (NPV) The percentage of results that were true negative (TN) among all results that the system flagged as negative. Calc: TN/(TN+FN). Recall Also called sensitivity, it is the percentage of results flagged positive among all results that should have been obtained. Calc: TP/(TP+FN). Specificity The percentage of results that were flagged negative among all negative results. Calc: TN/(TN+FP). F1-Score In this case, the harmonic mean of PPV/precision and sensitivity/recall is unweighted. Calc: 2 × (Precision x Recall)/(Precision + Recall). Abbreviations TP = True Positive, FP = False Positive, FN = False Negative, TP = True Negative Precision (PPV) was selected as the primary outcome measure to rank the databases because it offers the most helpful measure of database performance for IBD cohort identification. 1.3.7.3 Jaccard Index Union-Size Calculation The Jaccard index can be used to calculate the size of the intersection (overlapping elements) and remaining union (non-overlapping elements) between the two databases. When combined with the known precision for IBD, the total IBD cohort size can be estimated (assuming that the precision is the same for both the intersection and union, and that complex interactions do not exist within databases). This method is mathematically and clinically useful only for calculating an upper estimate as soon as more than two databases are analysed. With this caveat in place, the inference protocol is described as follows: 1.3.7.4 Cohort Size Inference Protocol 1) Start with the primary database (ICD10 codes) and multiply the unique patients in this dataset by the precision of this dataset to obtain a base ‘Combined’ predicted IBD set. 2) Sort other databases by precision (descending). 3) Iterate over these databases in order as follows: a. Pick the following highest-precision dataset that has not yet been integrated. b. Calculate the Jaccard index between the current ‘Combined’ and the next highest precision set. c. Use the recalculated Jaccard similarity index between ‘Combined’ and the following dataset to estimate the unique patients contributed by that dataset (i.e., those only present on the new dataset’s side of the union). d. The unique patients in that dataset are multiplied only by the precision of the dataset to estimate the incremental true positives. e. Add that unique set of patients to the ‘Combined’ set. f. Repeat the process until no more datasets remain. As elegant as this process may seem, it has significant weaknesses. Primarily, it assumes that precision, assessed against the gold standard, is equally weighted between patients at the intersection and those only in the union. This assumption causes the method to overestimate total cohort size. The code for this algorithm is provided as a fully open source at this URL to maximise transparency and replicability (https://github.com/MattStammers/IBD_Cohort_Size_Estimation_IC-IBD_Study_Part_1). 1.3.8 Multivariate Modelling Machine learning (ML) logistic regression (LR)(36) classifiers were constructed using 11 available databases. Demographic features such as age, sex, ethnicity, and IMD decile were excluded from the feature set using predefined patterns. Features were standardised using z-scores (mean = 0, standard deviation = 1) before model fitting. To improve the algorithm performance, the L2 (ridge) penalty was used alongside the regularised least absolute shrinkage and selection operator L1 (lasso) penalty in a 50:50 elastic net mix to evaluate the features that could improve the prediction. The lasso shrinks parameters according to their variance, reducing overfitting and enabling automatic variable selection(39), while the ridge provides stability. The optimal degree of regularisation was determined by identifying a tuning parameter (lambda) λ using nested cross-validation (as described below) with a stochastic average gradient augmented (SAGA) solver in light of the sparsity of the underlying data (primarily due to negative imputation). To avoid overfitting and to reduce the number of false-positive predictors, λ was selected to provide a model with an area under the receiver operating characteristic curve (AUC) and one standard error below the best model. All analyses used pandas, fairlearn, numpy, seaborn, matplotlib, seaborn, and scikit-learn packages in VS Code TM and Python 3.10.10 with poetry to manage virtual environments. The code was version-controlled using Git and made available open-source online to maximise replicability and transparency. 1.3.8.1.1 Cross Validation & Calibration To evaluate the model's predictive performance for new cases of the same underlying population (internal validation), nested cross-validation (10-fold for the inner loop; 10-fold/100 repeats for the outer loop) was performed. Platt scaling was used to improve the calibration (37) because the calibration distribution was approximately sigmoid in shape. Discrimination was assessed using the AUC and Brier scores(38). All steps (feature selection, scaling, and threshold selection) were incorporated into the model development and selection process to avoid data leakage that would otherwise result in optimistic performance measures(155). Type 2a validation was performed on the holdout set(39). Measures of discrimination (precision, recall, harmonic F1-score, Brier score) and calibration were assessed. Calibration was evaluated using three methods. 1. A standard calibration curve plotting mean predicted probabilities against observed proportions in bins. 2. A locally estimated scatterplot smoothing (LOESS) calibration curve was fitted to the predicted probabilities and observed outcomes. 3. A logistic regression calibration plot fitting a logistic regression curve to the same data. 1.3.8.1.2 Bias Identification/Error Analysis The model's potential for bias was also examined by conducting a stratified analysis of its performance across different demographic groups (race, age, sex, and index of multiple deprivation (IMD)) and comparing the AUC for these subgroups. Results 1.4.1 Total Study Cohort Between 2007 and 2023, 52,332 non-two-week wait referrals were made for 37,947 individual patients. The gold-standard validation cohort consisted of 2,800 patients: 2,180(78%) with IBD and 619(22%) non-IBD cases. The randomly seeded validation subset (30%) contained 841 patients, 664(79%) with IBD and 176(21%) without IBD. The distribution of these patients in each database and temporally by year of the first referral are shown in Fig 3 . [FIGURE 3] 1.4.1.1 Cohort Demographics Table 2 shows the demographic characteristics of the entire cohort. The skewness and kurtosis for age were 0.017 and -1.076, respectively, and those for IMD were -0.111 and -1.181, respectively. Table 2: Full Cohort Demographics Feature Mean Median Age at Point of Referral 51.79yrs (95%CI:51.59-51.92) 52.22yrs (IQR:32.4) Sex (Female) 60.27% Ethnicity (White) 85.04% IMD Decile 5.91 (95%CI:5.88-5.94) 6 (IQR:4) Urgent Referrals 21.34% Local Referrals From Southampton Catchment 83.01% Table 2 describes the cohort demographics of patients included in the study 1.4.2 Coding, Event and Registry-Based Predictions By examining the relationships between each dataset and the gold-standard validation cohort, the baseline precision and recall were established for each database, as shown in Table 3 . Table 3: Coding, Registry & Event-Based Predictions Database Coverage Accuracy Precision Recall Specificity NPV F1 Score Coding ICD10 Codes 802 (95.48%) 0.92 (0.9 - 0.94) 0.96 (0.94 - 0.98) 0.93 (0.91 - 0.95) 0.86 (0.81 - 0.91) 0.78 (0.72 - 0.84) 0.95 (0.93 - 0.96) OPCS4 Codes 274 (32.6%) 0.32 (0.26 - 0.38) 0.91 (0.81 - 0.98) 0.18 (0.12 - 0.23) 0.92 (0.84 - 0.99) 0.21 (0.15 - 0.26) 0.29 (0.22 - 0.37) Registries Patient Portal 428 (50.95%) 0.99 (0.98 - 1.0) 0.99 (0.98 - 1.0) 1.0 (1.0 - 1.0) 0.0 (0.0 - 0.0) 0 (0.0 - 0.0) 1.0 (0.99 - 1.0) EPR IBD Registry 571 (67.98%) 0.99 (0.98 - 0.99) 0.99 (0.98 - 0.99) 1.0 (1.0 - 1.0) 0.0 (0.0 - 0.0) 0 (0.0 - 0.0) 0.99 (0.99 - 1.0) Events Cytokine Modulator Prescriptions 198 (23.57%) 1.0 (1.0 - 1.0) 1.0 (1.0 - 1.0) 1.0 (1.0 - 1.0) 0.0 (0.0 - 0.0) 0.0 (0.0 - 0.0) 1.0 (1.0 - 1.0) Flare Calls 505 (60.12%) 0.89 (0.86 - 0.91) 0.89 (0.86 - 0.91) 1.0 (1.0 - 1.0) 0.0 (0.0 - 0.0) 0 (0.0 - 0.0) 0.94 (0.92 - 0.96) IBD Clinic Appointments 664 (79.05%) 0.38 (0.34 - 0.42) 0.82 (0.77 - 0.87) 0.31 (0.27 - 0.35) 0.71 (0.62 - 0.79) 0.19 (0.15 - 0.23) 0.45 (0.4 - 0.49) Calprotectin > 50 533 (63.45%) 0.32 (0.28 - 0.36) 0.82 (0.75 - 0.9) 0.19 (0.15 - 0.22) 0.84 (0.77 - 0.91) 0.2 (0.17 - 0.24) 0.31 (0.26 - 0.35) Table 3: Baseline ground truth as established using the validation cohort comparing coverage, precision and recall for each dataset. The F1 score is given for each dataset. 1.4.3 Simple String Regex Search Model The string regression search model is the most straightforward natural language processing (NLP) based cohort identification model. It was used as a proxy for the likelihood of IBD among free-text documents. Table 4 . Table 4: String Search Model Comparison Database Algo Coverage Accuracy Precision Recall Specificity NPV F1 Score Endoscopy Records Simple Regex 738 (87.9%) 0.72 (0.69 - 0.75) 0.95 (0.93 - 0.97) 0.68 (0.64 - 0.72) 0.86 (0.8 - 0.91) 0.41 (0.36 - 0.47) 0.79 (0.76 - 0.82) Clinical Letters Simple Regex 794 (94.5%) 0.83 (0.8 - 0.86) 0.83 (0.8 - 0.86) 0.99 (0.98 - 1.0) 0.24 (0.18 - 0.31) 0.89 (0.78 - 0.97) 0.9 (0.89 - 0.92) Histopathology Records Simple Regex 506 (60.24%) 0.7 (0.66 - 0.74) 0.75 (0.71 - 0.79) 0.89 (0.86 - 0.92) 0.16 (0.1 - 0.22) 0.34 (0.23 - 0.46) 0.81 (0.79 - 0.84) Table 4 describes the results of the regex string search models across the clinical, endoscopy and histopathology records. 1.4.4 Database Cardinality Cardinality measures the uniqueness or distinctiveness of elements within a database table. Given the sheer number of intersecting sets in this study, the results were best visualised using UpSet plots ( Fig 4 ). These are superior to Venn diagrams for visualising data sets with more than three intersecting sets in a matrix. [FIGURE 4] Significant overlaps between suspected IBD cases in databases are an exception rather than a rule. Of the 8,212 unique patients with at least one ICD-10 code for IBD, only 476 (5.8%) were found in ten or more clinical databases. Similarly, 7,570 individuals with suspected IBD were identified in a single database. 1.4.5 Jaccard Similarity Indices Fig 5. shows the Jaccard indices(30) across all 11 databases before and after the application of IBD prediction criteria. [FIGURE 5] Jaccard indices were high for the complete clinical coding database, clinical letters (0.88), and endoscopy records (0.76). The same was true for endoscopy and clinical letters (0.74). Moderately high indices existed between coding and appointments (0.55), OPCS4 codes and ICD10 coding (0.38), endoscopy (0.39), and clinic letters (0.37). Moderately high indices existed between patient portal registration and registration within EPR (0.58), flare calls (0.36), calprotectin testing (0.37), and cytokine modulator usage (0.38). The same was observed between endoscopies, appointments (0.48), and GI histopathology (0.43), while appointments intersected with clinical letters (0.53) and histopathology (0.45). However, these indices were substantially altered in the suspected IBD cohort, with no high Jaccard indices. IBD ICD10 codes intersected moderately with the registry databases (0.4-0.49), clinic letters (0.54), and endoscopies (0.45). The EPR registry intersects with endoscopy (0.47) and clinical letters (0.51), whereas the patient portal intersects with flare line calls (0.38), cytokine modulator prescriptions (0.37), clinical letters (0.45), calprotectins >50 (0.42), IBD specialist appointments (0.37), and endoscopy (0.41). Endoscopy overlapped with clinical letters suggesting IBD (0.43). IBD specialist appointments also overlapped with clinical letters suggestive of IBD (0.36). The remainder of the database intersections are low, ranging from (0.19-0.35), except for IBD OPCS4 codes, where the range is even lower (0.04-0.07). 1.4.6 Cohort Size Estimation by Recursive Jaccard Similarity Database Inference The full results of the inference process are described in Table 5 . Table 5: Full Recursive Jaccard Similarity Cohort Size Estimation Database Flagged Positive Cases Jaccard with Combined Intersection Unique Precision Incremental TPs Cumulative TPs ICD10 Codes 8337 8337 0.96 8004 8004 Cytokine Modulator Prescriptions 1762 0.205 1718 44 1.00 44 8048 Patient Portal 3643 0.408 3483 160 0.99 158 8206 EPR IBD Registry 4312 0.501 4288 24 0.99 24 8230 Endoscopy Records 4327 0.447 3982 345 0.95 328 8557 OPCS4 Codes 1190 0.052 501 689 0.91 627 9184 Flare Calls 7705 0.353 4519 3186 0.89 2836 12020 Clinical Letters 14984 0.461 8761 6223 0.83 5165 17185 IBD Clinic Appointments 5520 0.237 4707 813 0.82 667 17852 Calprotectin >50 4000 0.184 3708 292 0.82 239 18091 Histopathology Records 6070 0.257 5352 718 0.7 503 18594 All Integrated 20831 18594 Table 5: Describes the results of the recursive cohort size estimation strategy pursued according to the defined protocol. TP’s – True Positives. The largest unaccounted-for group emerged from clinic letters (n=5,165), followed by flare calls (n=2,836). These factors alone accounted for 7,889 additional patients with uncoded IBD. 1.4.7 Cohort Size Estimation by Penalised Logistic Regression The following estimates of the total IBD population size per Table 6 were obtained by applying thresholding to the penalised logistic regression model. Table 6: IBD LR Predictions by Threshold Threshold Precision (95%CI) Recall (95%CI) Accuracy (95%CI) Predicted IBD Total (95%CI) Actual Predicted IBD Total (95%CI) 0.25 0.83 ( 0.82 - 0.84 ) 0.92 ( 0.91 - 0.93) 0.79 ( 0.78 - 0.81) 15373 ( 15180 - 15554) 12760 (12464 - 13040) 0.31 0.84 ( 0.83 - 0.86 ) 0.92 ( 0.91 - 0.93 ) 0.8 ( 0.79 - 0.82 ) 13507 ( 13320 - 13680) 11346 ( 11119 - 11596) 0.38 0.85 ( 0.83 - 0.86) 0.92 ( 0.91 - 0.93 ) 0.81 ( 0.79 - 0.82 ) 11687 ( 11517 - 11868 ) 9934 ( 9687 - 10125) 0.44 0.85 ( 0.84 - 0.87) 0.92 ( 0.9 - 0.93 ) 0.81 ( 0.79 - 0.82 ) 10318 ( 10167 - 10490 ) 8770 ( 8567 - 8986) 0.5 0.85 ( 0.84 - 0.87 ) 0.91 ( 0.9 - 0.92 ) 0.81 ( 0.79 - 0.82) 9619 ( 9457 - 9795) 8176 ( 7990 - 8398) 0.56 0.86 ( 0.84 - 0.87 ) 0.91 ( 0.9 - 0.92 ) 0.81 ( 0.8 - 0.83) 9432 ( 926 -9593 ) 8112 ( 7890 - 8290) 0.62 0.87 ( 0.85 - 0.88 ) 0.91 ( 0.9 - 0.92) 0.82 ( 0.81 - 0.83 ) 9299 ( 9147 - 9458 ) 8090 ( 7869 - 8257 ) 0.69 0.88 ( 0.86 - 0.89 ) 0.91 ( 0.9 -0.92) 0.83 ( 0.81 - 0.84) 9057 ( 8890 - 9228 ) 7970 ( 7745 - 8124 ) 0.75 0.89 ( 0.87 - 0.9 ) 0.9 ( 0.89 - 0.91 ) 0.83 ( 0.82 - 0.84 ) 8567 ( 8416 - 8719) 7625 ( 7415 - 7767) Table 6: Describes the full results of thresholding the LR model at different levels. The optimal threshold was 0.586, detecting 8,060 true positive IBD patients with a global AUROC of 0.85 and a PR-AUC of 0.95 against the validation set. By adding only the true positives from a single database (clinic letters) to this total (n=4,938), a final estimated total of 12,998 true-positive IBD patients was reached. 1.4.7.1 Calibration & Bias Calibration remains an issue for this model. Figure 6 demonstrates that the model tends to underpredict IBD patients at lower predicted probabilities but overpredicts at >0.6, despite Platt scaling, which improved the Brier score from 0.0515 to 0.0461. [FIGURE 6] No significant bias was detected in this model for levels of deprivation or sex. However, the model performed significantly better among Caucasians (AUC-0.91) than among Asians (AUC-0.83) or Africans (AUC-0.76). Additionally, performance decreased in older age groups, declining from 0.93 in the 20-39 age range to 0.85 for 50-59 year olds. Discussion Accurate population-level data have substantial implications for policy formation, departmental resourcing, and the avoidance of discrimination in patient care, research, and service improvement. This study highlights significant flaws in relying on billing/read codes and medication data alone to identify clinical cohorts at both local and population levels. Only 8048 (61.6%) patients were identifiable from billing codes and medication data combined from an actual local IBD patient cohort of at least 12,998 individuals. The major strengths of this work include the real-world nature of the study, a robust approach to validation, the variety of databases investigated, and the simplicity of the methodology, aiding the ease of replicability in other settings without the need for advanced data science capability. This study revealed significant flaws in current IBD clinical cohort identification assumptions. Further cohort capture can be achieved by adding additional databases if sensible defragmentation attempts are made. Weaknesses include the study's single-center nature, meaning that prevalence cannot be accurately calculated, combined with the fact that actual positive cases are unlikely to be equally distributed between intersections and unions, thus reducing the accuracy. As the patients in the validation cohort were randomly drawn from a higher-probability IBD sample (present in at least two databases at the outset), they did not adequately represent the cohort fringes where only a single database node value existed. This means that the LR model underpredicts these cases because the classifier is not a good fit for them. This single-node problem cannot be overcome by adding additional features (such as age, ethnicity, and IMD) to the model. Such attempts typically compounded the problem without substantially ameliorating the model biases or calibration issues. Because two-week wait referrals were excluded from the cohort, the total cohort is likely to be even more significant than that reported here. Finally, some of the clinical information (e.g., clinical letters scanned as images) was unavailable for analysis in this study, suggesting that even more patients who may be available for retrieval were optical character recognition in place. Clinical letters are the most important contributors to patients outside the ICD-10 codes. Montoto et al. ( 39 ) (2022) claimed to achieve 0.88 precision and 0.98 recall for diagnosing Crohn’s disease within a large Spanish multicenter cohort. However, the free-text precision of the simple regex algorithm we derived here was only 0.83 and the recall 0.99 for detecting positive IBD cases, which needs to be improved in subsequent work. However, because they used ICD codes alone to generate their gold-standard validation cohorts, if our results were replicated in that study, at least 37.5% of the actual IBD cases would have been missing from their cohort at the outset and 4% of their true-positive cases would have, in fact, not had IBD. This is the first time that the fully complicated impacts of database fragmentation have been documented for IBD cohorts, building on the work of others( 21 , 22 ). This study primarily examined the extent of the problem. However, the principles of 1) exposing many different databases, 2) validating a gold standard cohort, and 3) using ML to identify a complete cohort are transferable to most other clinical domains and diseases. To make this process more scalable, novel methodologies to standardise datasets, positively identify patients, and compare databases across a graph of tables are required. Success in these endeavors will positively impact clinical research, population health, and frontline clinical care by highlighting the true IBD clinical cohorts of local teams. Conclusion Using historical billing/reading codes and medication data alone in the context of IBD can substantially underestimate the total local IBD clinical cohort size. Further research on methodologies to overcome this problem is required, particularly for retrieving IBD cases from free-text clinical records. Declarations Contributors: MS performed all analyses and final data preparation. SS led and supervised the derivation of the gold-standard cohort, which MS then double-checked. MS drafted the first version of the manuscript. MG, RN, CM, CK, FC, SS, and JB provided critical feedback regarding the manuscript. MS is the primary guarantor for the review and the corresponding author. Acknowledgements: To make this project possible, we acknowledge the SETT data and AI team (in particular Mr Cai Davis and Dr Michael George), the CIRU/AXIS team (Dr. Ashley Heinson) – bioinformatician who checked the statistics, and three clinical junior doctors who helped with the IBD manual chart review: Dr Perez Pablo, Dr Maryam Al-Ezairej, and Dr Mahmoud Abosamra, as well as Dr Richard Felwick who helped feed back on some early iterations of the project. None of them were involved in the analysis, and the corresponding author assumes responsibility for all data handling and analytics. Funding: This work was indirectly supported by the Research Leaders' Program through funding provided to MS by the Southampton Academy of Research (SoAR) and University Hospital, Southampton. The protocol was independently developed. Competing Interests: RN received an educational grant from Pentax Medical. MS and MG attended the fully funded Dr. Falk Symposium on AI in Gastroenterology. CK is SRO for the Wessex SDE and FC has received grants/consultancy Fees/speaker Fees from Jassen, Celltrion, Biogen, Samsung, Sandoz, MSD, Abbvie, Pfizer, Hospiral and Gilead and is also clinical lead for the UK IBD Registry. Patient consent for publication: Not applicable Data Availability: All data generated or analysed during this study is unavailable to protect patient privacy as the patients involved did not consent to sharing their data. Code Availability: All codes used in the analytics for this project are made available open source on GitHub at https://github.com/MattStammers/IBD_Cohort_Size_Estimation_IC-IBD_Study_Part_1 Model Availability: The LR model developed in this project is relatively custom to our local system; therefore, it has not been open-sourced but can be shared on request. Patient and Public Involvement: An IBD patient from our local IBD patient panel helped develop the ethics application and study protocol. Human Ethics and Consent to Participate Declarations: The Wessex REC and HRA provided research ethics board approval for this study (IC-IBD -23/SC/0152) on 16/05/2023 (https://www.hra.nhs.uk/planning-and-improving-research/application-summaries/research-summaries/ic-ibd-ibd-cohort-identification-study/). References Chen TK, Knicely DH, Grams ME. Chronic Kidney Disease Diagnosis and Management: A Review. JAMA. 2019;322(13):1294–304. Park JG, Chen XD, Clontz M, Begaj T, Runner MM, Wolfe JD. Coding of Geographic Atrophy and Exudative Age-related Macular Degeneration. Ophthalmol Retina. 2023;7(7):644–5. Baumgart DC, Sandborn WJ. Inflammatory bowel disease: clinical aspects and established and evolving therapies. The Lancet. 2007;369(9573):1641–57. Nikolaus S, Schreiber S. Diagnostics of Inflammatory Bowel Disease. Gastroenterology. 2007;133(5):1670–89. Burisch J, Jess T, Martinato M, Lakatos PL. The burden of inflammatory bowel disease in Europe. J Crohns Colitis. 2013;7(4):322–37. Ghosh N, Premchand P. A UK cost of care model for inflammatory bowel disease. Frontline Gastroenterol. 2015;6(3):169–74. Guevara M, Salamanca-Fernández E, Miqueleiz E, Gavrila D, Amiano P, Bonet C, et al. Inflammatory Potential of the Diet and Incidence of Crohn’s Disease and Ulcerative Colitis in the EPIC-Spain Cohort. Nutrients. 2021;13(7):2201. Vavricka SR, Brun L, Ballabeni P, Pittet V, Vavricka BMP, Zeitz J, et al. Frequency and Risk Factors for Extraintestinal Manifestations in the Swiss Inflammatory Bowel Disease Cohort. Off J Am Coll Gastroenterol ACG. 2011;106(1):110. Molodecky NA, Panaccione R, Ghosh S, Barkema HW, Kaplan GG, on Behalf of the Alberta Inflammatory Bowel Disease Consortium. Challenges associated with identifying the environmental determinants of the inflammatory bowel diseases. Inflamm Bowel Dis. 2011;17(8):1792–9. Charpentier C, Salleron J, Savoye G, Fumery M, Merle V, Laberenne JE, et al. Natural history of elderly-onset inflammatory bowel disease: a population-based cohort study. Gut. 2014;63(3):423–32. Carlson AM, Dennis C, Singh I, Altaher W, Cleary J, Gohil H, et al. S1267 Investigating Misdiagnosis Rates of Inflammatory Bowel Disease: A Retrospective Single-Center Review. Off J Am Coll Gastroenterol ACG. 2024;119(10S):S902. Woo DH, Kim KO, Kang MK, Lee SH, Jang BI, Kim TN. Predictors and clinical outcomes of follow-up loss in patients with inflammatory bowel disease: Follow up loss in inflammatory bowel disease. J Gastroenterol Hepatol. 2018;33(11):1834–8. Ananthakrishnan AN, Cai T, Savova G, Cheng SC, Chen P, Perez RG, et al. Improving Case Definition of Crohnʼs Disease and Ulcerative Colitis in Electronic Medical Records Using Natural Language Processing: A Novel Informatics Approach. Inflamm Bowel Dis. 2013;19(7):1411–20. Stapley SA, Rubin GP, Alsina D, Shephard EA, Rutter MD, Hamilton WT. Clinical features of bowel disease in patients aged < 50 years in primary care: a large case-control study. Br J Gen Pract. 2017;67(658):e336–44. Bhangu A, Nepogodiev D, Taylor C, Durkin N, Patel R. Accuracy of clinical coding from 1210 appendicectomies in a British district general hospital. Int J Surg. 2012;10(3):144–7. Naran S, Hudovsky A, Antscherl J, Howells S, Nouraei SAR. Audit of accuracy of clinical coding in oral surgery. Br J Oral Maxillofac Surg. 2014;52(8):735–9. Nouraei SAR, Hudovsky A, Frampton AE, Mufti U, White NB, Wathen CG, et al. A Study of Clinical Coding Accuracy in Surgery: Implications for the Use of Administrative Big Data for Outcomes Management. Ann Surg. 2015;261(6):1096–107. Nouraei SAR, O’Hanlon S, Butler CR, Hadovsky A, Donald E, Benjamin E, et al. A multidisciplinary audit of clinical coding accuracy in otolaryngology: financial, managerial and clinical governance considerations under payment-by-results. Clin Otolaryngol. 2009;34(1):43–51. Horsky J, Drucker EA, Ramelson HZ. Accuracy and Completeness of Clinical Coding Using ICD-10 for Ambulatory Visits. AMIA Annu Symp Proc AMIA Symp. 2017;2017:912–20. Campbell S, Giadresco K. Computer-assisted clinical coding: A narrative review of the literature on its benefits, limitations, implementation and impact on clinical coding professionals. Health Inf Manag J. 2020;49(1):5–18. Rye C, Rubin KH, Moller FT, Julsgaard M, Jess T, Andersen V. Positive Predictive Value of Diagnostic Codes for Inflammatory Bowel Disease in the Danish National Patient Registry Among Individuals 50 + Years, Using Patient Records as Reference Standard. Clin Epidemiol. 2021;Volume 13:335–44. Jones GR, Lyons M, Plevris N, Jenkinson PW, Bisset C, Burgess C, et al. IBD prevalence in Lothian, Scotland, derived by capture–recapture methodology. Gut. 2019;68(11):1953–60. Stammers M, Ramgopal B, Owusu Nimako A, Vyas A, Nouraei R, Metcalf C, et al. A foundation systematic review of natural language processing applied to gastroenterology & hepatology. BMC Gastroenterol. 2025;25(1):58. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Intern Med. 2015;162(1):55–63. RCP London [Internet]. 2015 [cited 2023 Nov 24]. IBD organisational audit. Available from: https://www.rcplondon.ac.uk/projects/ibd-organisational-audit Austin PC, Steyerberg EW. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat Methods Med Res. 2017;26(2):796–808. Pate A, Riley RD, Collins GS, van Smeden M, Van Calster B, Ensor J, et al. Minimum sample size for developing a multivariable prediction model using multinomial logistic regression. Stat Methods Med Res. 2023;32(3):555–71. Sarkar S, Livingstone R, Borca F, Stammers M, Gwiggner M. PTH-32 Development of a novel electronic referral grading & triage system. Gut. 2021;70(Suppl 4):A186–7. Humphreys BL, Lindberg DA. The UMLS project: making the conceptual connection between users and the information they need. Bull Med Libr Assoc. 1993;81(2):170–7. Real R, Vargas JM. The Probabilistic Basis of Jaccard’s Index of Similarity. Syst Biol. 1996;45(3):380–5. Dharavath R, Singh AK. Entity Resolution-Based Jaccard Similarity Coefficient for Heterogeneous Distributed Databases. In: Satapathy SC, Raju KS, Mandal JK, Bhateja V, editors. Proceedings of the Second International Conference on Computer and Communication Technologies. New Delhi: Springer India; 2016. p. 497–507. Fletcher S, Islam MZ. Comparing sets of patterns with the Jaccard index. Australas J Inf Syst [Internet]. 2018 Mar 7 [cited 2024 Dec 18];22. Available from: https://ajis.aaisnet.org/index.php/ajis/article/view/1538 Hunter JD. Matplotlib: A 2D Graphics Environment. Comput Sci Eng. 2007;9(03):90–5. Waskom M. seaborn: statistical data visualization. J Open Source Softw. 2021;6(60):3021. Jolly K. Hands-On Data Visualization with Bokeh: Interactive web plotting for Python using Bokeh. Packt Publishing Ltd; 2018. 168 p. Jr DWH, Lemeshow S, Sturdivant RX. Applied Logistic Regression. John Wiley & Sons; 2013. 528 p. Platt J. Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods. In 1999 [cited 2025 Feb 1]. Available from: https://www.semanticscholar.org/paper/Probabilistic-Outputs-for-Support-vector-Machines-Platt/42e 5ed832d4310ce4378c44d05570439df28a393 Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010;63(8):938–9. Montoto C, Gisbert JP, Guerra I, Plaza R, Pajares Villarroya R, Moreno Almazán L, et al. Evaluation of Natural Language Processing for the Identification of Crohn Disease-Related Variables in Spanish Electronic Health Records: A Validation Study for the PREMONITION-CD Project. JMIR Med Inform. 2022;10(2):e30345. Additional Declarations Competing interest reported. RN received an educational grant from Pentax Medical. MS and MG attended the fully funded Dr. Falk Symposium on AI in Gastroenterology. CK is SRO for the Wessex SDE and FC has received grants/consultancy Fees/speaker Fees from Jassen, Celltrion, Biogen, Samsung, Sandoz, MSD, Abbvie, Pfizer, Hospiral and Gilead and is also clinical lead for the UK IBD Registry. Supplementary Files Supplement1TRIPODChecklist.pdf Supplement2OPCS4codesofrelevance.pdf Cite Share Download PDF Status: Published Journal Publication published 13 Aug, 2025 Read the published version in Digestive Diseases and Sciences → Version 1 posted Editorial decision: Revision requested 25 Jun, 2025 Reviews received at journal 09 Apr, 2025 Reviewers agreed at journal 28 Mar, 2025 Reviewers agreed at journal 26 Mar, 2025 Reviewers invited by journal 26 Mar, 2025 Editor assigned by journal 25 Mar, 2025 Submission checks completed at journal 25 Mar, 2025 First submitted to journal 24 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6298636","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":440572587,"identity":"94834aa7-5d10-4e6e-ada3-ab2bc7537776","order_by":0,"name":"Matthew Stammers","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6ElEQVRIiWNgGAWjYHAD5gNInAM4FEEAYwOEZktgYEggTQuPAXFadBvYnz/4mbNNzrz/zOcPH3/Y5DGwH37AzHMGtxazAzyGjb3bbhvL3MjdJjkjIa2YgSfNgJnnBl4tjA28224nzpDg3cbMk3A4sYEhh4GZ5wM+LewPG/9uu10/g//M489gLfxvCGlhMGwG2pIgATRcGqxFAmQLPocd5jGcLbvttuEMiTQzyRlpaYltEs8MDs7B5/3j7Q8+vt12W16C//DjDx9sbBL7+ZMfPnhzDLcWBmZ0ATYGQhE5CkbBKBgFo4AgAAAqB1OYgV679gAAAABJRU5ErkJggg==","orcid":"","institution":"University of Southampton","correspondingAuthor":true,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Stammers","suffix":""},{"id":440572588,"identity":"b9596225-c96e-4d83-bd30-e0a5d57090cd","order_by":1,"name":"Stephanie Sartain","email":"","orcid":"","institution":"University Hospital Southampton","correspondingAuthor":false,"prefix":"","firstName":"Stephanie","middleName":"","lastName":"Sartain","suffix":""},{"id":440572589,"identity":"9709b54e-fcaf-4299-9948-ef0328ad1ba0","order_by":2,"name":"Fraser Cummings","email":"","orcid":"","institution":"University Hospital Southampton","correspondingAuthor":false,"prefix":"","firstName":"Fraser","middleName":"","lastName":"Cummings","suffix":""},{"id":440572590,"identity":"9f3b78bb-a7f5-4f09-9c20-28aea7fb0ad0","order_by":3,"name":"Christopher Kipps","email":"","orcid":"","institution":"University Hospital Southampton","correspondingAuthor":false,"prefix":"","firstName":"Christopher","middleName":"","lastName":"Kipps","suffix":""},{"id":440572591,"identity":"64ee430f-ec73-4b4d-be88-9edec463eb6a","order_by":4,"name":"Reza Nouraei","email":"","orcid":"","institution":"University of Southampton","correspondingAuthor":false,"prefix":"","firstName":"Reza","middleName":"","lastName":"Nouraei","suffix":""},{"id":440572592,"identity":"3db4327a-f3c4-4e27-a307-ef4a3f755772","order_by":5,"name":"Markus Gwiggner","email":"","orcid":"","institution":"University Hospital Southampton","correspondingAuthor":false,"prefix":"","firstName":"Markus","middleName":"","lastName":"Gwiggner","suffix":""},{"id":440572593,"identity":"c4268add-718a-488e-be00-82905e295a85","order_by":6,"name":"Cheryl Metcalf","email":"","orcid":"","institution":"University of Southampton","correspondingAuthor":false,"prefix":"","firstName":"Cheryl","middleName":"","lastName":"Metcalf","suffix":""},{"id":440572594,"identity":"b9cba6bd-9fb9-4ee7-84e0-a9b881f60a81","order_by":7,"name":"James Batchelor","email":"","orcid":"","institution":"University of Southampton","correspondingAuthor":false,"prefix":"","firstName":"James","middleName":"","lastName":"Batchelor","suffix":""}],"badges":[],"createdAt":"2025-03-24 23:23:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6298636/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6298636/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10620-025-09323-1","type":"published","date":"2025-08-13T15:57:15+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":80709832,"identity":"9f02dcab-2050-46e7-ac0e-514295a3aeac","added_by":"auto","created_at":"2025-04-16 08:57:07","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":49623,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003ePate \u0026amp; Riley Binary Prediction Sample Size Estimation Formula\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eExplains the sample size calculation formula for binary classification models as developed by Pate\u0026amp;Riley\u003c/em\u003e(27)\u003cem\u003e where N is the required sample size, K is the number of candidate predictors, S is the desired shrinkage factor, and R\u003c/em\u003e\u003csup\u003e\u003cem\u003e2 \u003c/em\u003e\u003c/sup\u003e\u003cem\u003eis the expected Cox-Snell R\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e\u003cem\u003e. Additionally, the formula adjusts for the outcome prevalence (p) as displayed, where K1 is the effective sample size (derived from the initial equation without prevalence adjustment), and p is the overall prevalence of the outcome.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure1PateRileySampleSizeCalculationFormula.png","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/2194370aa71af292d8fda8f4.png"},{"id":80707789,"identity":"ae717607-d21b-4467-a89e-cde77d81938e","added_by":"auto","created_at":"2025-04-16 08:49:07","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":3643,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eJaccard similarity index calculation formula\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eShows the calculation for the Jaccard similarity index\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure2JaccardSimilarityIndexCalculationFormula.png","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/cb52da04f4dc3069d7f70548.png"},{"id":80707797,"identity":"cdc659b9-d1ae-474a-8549-a97057b21e67","added_by":"auto","created_at":"2025-04-16 08:49:07","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":117555,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eStudy Population CONSORT and Temporal Distribution\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFigure A describes the distribution of patients in the study by database and database type. Figure B describes the distribution of patients temporally by year of first referral to Gastroenterology, demonstrating the distribution of the validation cohort across the study period with some notable fluctuations, such as 2019, 2020 and the two first years the PAS was initiated at the trust (2007-2008).\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure3CONSORTAndReferralsbyyearwithvalidations.png","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/1c6c0b301a94677dbf8a91ef.png"},{"id":80712112,"identity":"3727291c-6419-4877-ba1b-45b6b0143b84","added_by":"auto","created_at":"2025-04-16 09:13:07","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":1373270,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eDatabase Cardinality Comparison (UpSet Plots)\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eTwo UpSet plots are displayed above. Figure A describes database cardinality among the total cohort, and Figure B describes database cardinality among the patients suspected of having IBD. Both figures demonstrate high overall database cardinality, with only 385 (1%) of patients having records in all 11 databases.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure4UpsetCardinalityPlots.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/d008eca213de28ca1ec89e18.jpg"},{"id":80707814,"identity":"3b95be4f-62f1-4cb3-ae8e-bcfb5c5f99f7","added_by":"auto","created_at":"2025-04-16 08:49:07","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":5163658,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eHeatmap of Jaccard similarity indices across databases\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eDatabase Correlations by Jaccard Similarity.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure5DatabaseJaccardSimilarityHeatmaps.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/538648a1c26668a731df0c8b.jpg"},{"id":80711015,"identity":"a1d7f583-f233-4069-b157-ead2debc0ca9","added_by":"auto","created_at":"2025-04-16 09:05:07","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":42091,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eModel Calibration Curve\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eDemonstrates the calibration curve for the LR model. Using the existing database co-variates it is not possible to perfectly calibrate the model.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure6CalibrationPlot.png","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/8b0fb0d2a1c58aea9219b2d8.png"},{"id":89310527,"identity":"c9d4fbd7-c2f4-4ba5-923c-f29bd2ea2ed4","added_by":"auto","created_at":"2025-08-18 16:06:58","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":8908970,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/6eac9c65-37ef-4f55-924a-6f337651f88d.pdf"},{"id":80709834,"identity":"7435a99f-c2bb-4404-a40f-cd2dd54b0b89","added_by":"auto","created_at":"2025-04-16 08:57:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":38869,"visible":true,"origin":"","legend":"","description":"","filename":"Supplement1TRIPODChecklist.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/50f150f4c674a8ed8b6d50ec.pdf"},{"id":80709833,"identity":"a09354b5-dcaf-4fea-81ec-f74e3616a046","added_by":"auto","created_at":"2025-04-16 08:57:07","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":89979,"visible":true,"origin":"","legend":"","description":"","filename":"Supplement2OPCS4codesofrelevance.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6298636/v1/80e3e692d7feb7edc0a36fe9.pdf"}],"financialInterests":"Competing interest reported. RN received an educational grant from Pentax Medical. MS and MG attended the fully funded Dr. Falk Symposium on AI in Gastroenterology. CK is SRO for the Wessex SDE and FC has received grants/consultancy Fees/speaker Fees from Jassen, Celltrion, Biogen, Samsung, Sandoz, MSD, Abbvie, Pfizer, Hospiral and Gilead and is also clinical lead for the UK IBD Registry.","formattedTitle":"Identification of Cohorts with Inflammatory Bowel Disease Amidst Fragmented Clinical Databases via Machine Learning","fulltext":[{"header":"Introduction","content":"\u003ch3\u003e1.2.1\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Key Messages:\u003c/h3\u003e\n\u003cp\u003e· \u003cstrong\u003eWhat is already known on this topic\u003c/strong\u003e: \u003cem\u003eIBD patient cohorts can be identified using billing/read/clinical codes and medication data.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003eWhat this study adds\u003c/strong\u003e: \u003cem\u003eNine additional databases containing unique IBD patients are identified, and retrieval strategies to overcome database fragmentation demonstrate that medication data and ICD-10 codes only cover ~62% of the total cohort.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e· \u003cstrong\u003eHow this study might affect research, practice, or policy\u003c/strong\u003e: \u003cem\u003eSignificant numbers of IBD patients are missing from population and local-level cohort identification exercises. IBD prevalence is, therefore, likely systematically underestimated. More advanced cohort identification mechanisms will be required in future.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003ch3 id=\"_Toc185539703\"\u003e1.2.2\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;A Primer on Clinical Cohort Identification\u003c/h3\u003e\n\u003cp\u003eClinical cohort identification challenges vary substantially in difficulty by domain, ranging from comparatively simple conditions like chronic kidney disease (CKD), diagnosable purely by the estimated glomerular filtration rate(eGFR) over time(1), to more challenging conditions such as age-related macular degeneration (AMD) and its subtypes like geographic atrophy (GA), which can take an expert up to half an hour to diagnose visually from a scan, and until recently, had only a single ICD-10 umbrella code (H35.3)(2).\u003c/p\u003e\n\u003cp\u003eUlcerative colitis (UC), Crohn’s disease (CD), inflammatory bowel disease unclassified (IBDU), and microscopic colitis are chronic inflammatory conditions collectively known as inflammatory bowel disease (IBD)(3). They are diagnosed using a combination of clinical, biochemical, genetic, radiological, endoscopic, and histopathological tests(4). The best estimates suggest that the number of patients with IBD is increasing, with over 700,000 patients currently affected in the UK(5). The cost of care for IBD is demonstrably high, with annual per-patient treatment costs of £3084 and £6146 for UC and CD, respectively(3).\u003c/p\u003e\n\u003cp\u003eIBD is an interesting test case for a clinical cohort identification study because it is relatively common, with existing national registries and national cohorts(7–10), suggesting a degree of national cohort identification confidence. However, there is no definitive single diagnostic test for the condition, and misdiagnosis may be as high as 10%(11). Overdiagnosis may lead to medically induced injury, such as overtreatment with medications with significant side effects and underdiagnosis risk complications directly related to the disease. In one study, 14.7% of patients were lost to follow-up and 61% subsequently developed a disease flare(12).\u003c/p\u003e\n\u003cp\u003ePopulation-level health studies rely on diagnostic billing codes such as ICD-10. In the context of IBD, it has been claimed that diagnostic clinical codes are up to 97% accurate in identifying IBD clinical cohorts(13,14). However, this does not fit with real-world experience or other evidence that has consistently shown billing codes to be inaccurate in various clinical contexts(15–20). In a Danish study conducted in 2020, only 51% of the single-coded CD cohort and 54% of the single-coded UC cohort were accurate(21). In another study from Scotland, the use of a capture-recapture methodology involving medication data identified 427 previously missed IBD cases(22). To address this problem, baseline natural language processing (NLP) systems in gastroenterology are at a relatively early stage(23). This foundational problem must be solved before more advanced NLP systems such as large language models can be successfully leveraged.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis study highlights the complexities of identifying an IBD clinical cohort, even within a single institution, using reliable source data collected over 15 years. It also highlights the risks of data fragmentation and warns against the assumption that prior gold standards such as ICD-10 codes are sufficiently robust to be relied upon. Better mechanisms are required to reliably identify patients with IBD (and, by extension, other disease cohorts).\u0026nbsp;\u003c/p\u003e\n\u003ch3 id=\"_Toc185539704\"\u003e1.2.3\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Aim\u003c/h3\u003e\n\u003cp\u003eThis study aimed to estimate the size of a local IBD cohort across disparate fragmented databases within a single institution over the past 15 years.\u0026nbsp;\u003c/p\u003e\n\u003ch3 id=\"_Toc185539705\"\u003e1.2.4\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Objectives\u003c/h3\u003e\n\u003cp\u003e1.\u0026nbsp; \u0026nbsp;\u0026nbsp;Validate a gold-standard IBD cohort within our institution.\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp; \u0026nbsp;\u0026nbsp;Uncover database patient distributions and usefulness for IBD cohort identification.\u003c/p\u003e\n\u003cp\u003e3.\u0026nbsp; \u0026nbsp;\u0026nbsp;Explore statistical relationships/comparisons between databases.\u003c/p\u003e\n\u003cp\u003e4.\u0026nbsp; \u0026nbsp;\u0026nbsp;Estimate the total size of the local 15-year IBD cohort using this knowledge.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch3\u003e1.3.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Inclusion Criteria\u003c/h3\u003e\n\u003cp\u003eAll adults aged \u0026gt;18 years at the time of their first elective non-two-week wait referral (2WW) to our trust for gastroenterology specialist care between 2007 and 2023 who did not opt out of using their clinical data for research in secondary care were included in the study. The year 2007 was selected as the start of the study because this was the year the electronic patient administration system (PAS) was installed and digitised trackable referral data began to accrue.\u003c/p\u003e\n\u003ch3 id=\"_Toc185539708\"\u003e1.3.2 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Clinical Ethics \u0026amp; Checklist\u003c/h3\u003e\n\u003cp\u003eThe Wessex REC and HRA provided research ethics board approval for this study (IC-IBD -23/SC/0152) \u0026nbsp;on 16/05/2023. The study followed the original transparent reporting of a multivariable prediction model for individual prognosis or diagnosis\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e(TRIPOD) checklist(24), as tracked by page numbers in \u003cstrong\u003eSupplement 1\u003c/strong\u003e.\u003c/p\u003e\n\u003ch3 id=\"_Toc185539709\"\u003e1.3.3 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Datasets\u003c/h3\u003e\n\u003cp\u003eInternal databases hosted at our institution were examined and screened to identify a suspected IBD cohort. The 11 separate databases broadly fell into four categories.\u003c/p\u003e\n\u003ch4 id=\"_Toc185539710\"\u003e1.3.3.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Coded Databases\u003c/h4\u003e\n\u003cp\u003e1. \u003cstrong\u003eICD-10 Diagnosis Codes\u003c/strong\u003e (casemix). Registered IBD clinical ICD-10 codes include: (\u0026apos;K50.0\u0026apos;, \u0026apos;K50.1\u0026apos;, \u0026apos;K50.8\u0026apos;, \u0026apos;K50.9\u0026apos;, \u0026apos;K50.X\u0026apos;, \u0026apos;K51.0\u0026apos;, \u0026apos;K51.1\u0026apos;, \u0026apos;K51.2\u0026apos;, \u0026apos;K51.3\u0026apos;, \u0026apos;K51.5\u0026apos;, \u0026apos;K51.8\u0026apos;, \u0026apos;K51.9\u0026apos;, \u0026apos;K51.X\u0026apos;, \u0026apos;K52.3\u0026apos;, \u0026apos;K52.9\u0026apos;) as per the recommended RCP organisational IBD audit (25).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003eOPCS-4 Surgical Procedure Codes\u003c/strong\u003e. OPCS-4 codes that could represent IBD surgery (G58 \u0026ndash; G82 \u0026amp; H01 \u0026ndash; H56) as per the recommended RCP organisational IBD audit (25). The definitions of these codes are provided in \u003cstrong\u003eSupplement 2\u003c/strong\u003e.\u0026nbsp;\u003c/p\u003e\n\u003ch4 id=\"_Toc185539711\"\u003e1.3.3.2 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Registry Databases\u003c/h4\u003e\n\u003cp\u003e3. \u003cstrong\u003eElectronic Patient Record (EPR) IBD Registry\u0026nbsp;\u003c/strong\u003e- The hospital integrated clinical support system (HICSS\u003csup\u003eTM\u003c/sup\u003e) IBD Patient Module. In this module, patients can only be registered with a diagnosis by a gastroenterology consultant or specialist registrar/fellow.\u003c/p\u003e\n\u003cp\u003e4. \u003cstrong\u003eIBD Patient Portal -\u0026nbsp;\u003c/strong\u003e(My Medical Record\u003csup\u003eTM\u003c/sup\u003e): Patients can self-register for the platform but can only be added to the IBD pathway by a clinician.\u003c/p\u003e\n\u003ch4 id=\"_Toc185539712\"\u003e1.3.3.3 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Event Databases\u003c/h4\u003e\n\u003cp\u003e5. \u003cstrong\u003eAppointments:\u003c/strong\u003e Patient Appointments with Gastroenterology. This filter only flagged patients explicitly seen by an IBD specialist as suspected of having IBD.\u003c/p\u003e\n\u003cp\u003e6. \u003cstrong\u003eLab Biochemistry:\u003c/strong\u003e Faecal calprotectins were recorded in our laboratory. Only patients with levels \u0026gt; 50ug/L (the lab upper limit of normal) were suspected to have IBD.\u003c/p\u003e\n\u003cp\u003e7. \u003cstrong\u003eFlare Line:\u003c/strong\u003e Recorded calls to the nurse-led flare line. As this line also locally covers coeliac disease and other queries, only those with a recorded diagnosis of IBD on the call template were considered to have suspected IBD.\u003c/p\u003e\n\u003cp\u003e8. \u003cstrong\u003eCytokine Modulator Prescriptions:\u003c/strong\u003e Any patient with a documented prescription for a cytokine modulator under Gastroenterology on the EPR was suspected to have IBD.\u0026nbsp;\u003c/p\u003e\n\u003ch4 id=\"_Toc185539713\"\u003e1.3.3.4 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Free-Text Databases\u003c/h4\u003e\n\u003cp\u003eThe screening process for these databases is explained in Free Text Normalisation \u0026amp; Handling\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e9. \u0026nbsp; \u0026nbsp;Gastroenterology Clinic Letters\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e10. \u0026nbsp;Endoscopy Records\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e11. \u0026nbsp;Gastrointestinal Histopathology Records\u003c/strong\u003e\u003c/p\u003e\n\u003ch3 id=\"_Toc185539714\"\u003e1.3.4 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Primary \u0026amp; Secondary Outcome\u003c/h3\u003e\n\u003cp\u003eThe primary outcome of interest was the estimated number of patients in the IBD cohort.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe secondary outcomes of interest included precision (PPV), recall(sensitivity), and F1-score for each database and model to detect IBD diagnoses correctly against the gold-standard cohort, database cardinality, and algorithm fairness.\u003c/p\u003e\n\u003ch3\u003e1.3.5 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Strongly Supervised Gold-Standard Validation Cohort Derivation\u003c/h3\u003e\n\u003cp\u003eThe validation cohort was randomly selected from a larger group of patients within at least two of the 11 validation databases listed above. Therefore, the validation cohort skews towards IBD cases, and the control cases present in the validation cohort will be more difficult for algorithms to discriminate from IBD cases than they would among the general population. A strongly supervised randomised validation cohort was selected to maximise the robustness of the validation challenge.\u003c/p\u003e\n\u003cp\u003eA team of three junior doctors, led by a gastroenterology registrar (SS), performed manual chart reviews of this randomly selected cohort. Each participant was blinded to the efforts of others. They were supervised by a consultant (MS) who oversaw and re-checked each validation unblinding.\u003c/p\u003e\n\u003ch4 id=\"_Toc185539716\"\u003e1.3.5.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Validation Sample Size Calculation\u003c/h4\u003e\n\u003cp\u003eThis study aimed not only to calculate the total size of the cohort but also to build a model to identify individuals using a logistic regression classifier. \u0026nbsp;Therefore, rather than simply relying on only 20 events per variable (26)(EPV) to calculate the sample size, the sample size estimation method described by Pate and Riley was used(27) because this method has been validated in clinical contexts.\u003c/p\u003e\n\u003cp\u003eThe formula for binary predictions follows the logic explained below (\u003cstrong\u003eFig 1\u003c/strong\u003e):\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 1]\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA binary classification model with an expected Cox-Snell r-squared value of less than 0.05 was chosen because the discriminative value of each clinical dataset was expected to be low. Up to 11 predictor parameters were fed into the model, as there were 11 databases. Allowable validation shrinkage was set at 0.9 among a target population of gastroenterology referrals, where we already know that at least 16.5% will have IBD(28).\u003c/p\u003e\n\u003cp\u003eBased on this calculation, the gold-standard validation cohort required to train a model must be at least 1730 patients with a corresponding validation cohort of at least 519 and a training cohort of at least 1211. A base cohort of \u0026gt; 50% was derived to ensure sufficient scale and power for the study. The complete Python code for this calculation is provided as an open source for transparency.\u003c/p\u003e\n\u003ch3\u003e1.3.6 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Free Text Normalisation \u0026amp; Handling\u003c/h3\u003e\n\u003cp\u003eAll free-text documents were extracted in native format from the EPR and converted into simple strings. The Unified Medical Language System (UMLS)(29) (MRCONSO meta-thesaurus) was then used to remap IBD synonyms across all free text to create normalised terms for IBD in the following list: [\u0026quot;Ulcerative Colitis\u0026quot;, \u0026quot;Crohn\u0026apos;s Disease\u0026quot;, \u0026quot;IBD\u0026quot; (includes IBD-U), \u0026quot;Inflammatory Bowel Disease\u0026quot;, \u0026quot;Proctitis\u0026quot;, \u0026nbsp;\u0026quot;Collagenous Colitis\u0026quot;, \u0026quot;Microscopic Colitis\u0026quot;, \u0026quot;Lymphocytic Colitis\u0026quot;].\u003c/p\u003e\n\u003ch4\u003e1.3.6.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Regex Natural Language Processing (NLP) Model\u003c/h4\u003e\n\u003cp\u003eTo flag free-text documents as suggestive of IBD, a simple regex-based NLP model was utilised to match the strings according to the following five regular expressions and associated IBD-related lower case terms:\u003c/p\u003e\n\u003cp\u003e\u0026middot; (r\u0026lsquo;\u003cstrong\u003e*olitis\u0026rsquo;\u0026nbsp;\u003c/strong\u003e(ulcerative colitis, microscopic colitis, lymphocytic colitis, microscopic colitis, pan-colitis, and inflammatory colitis)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003er\u0026lsquo;*rohn*\u0026rsquo;\u0026nbsp;\u003c/strong\u003e(crohn\u0026rsquo;s, crohn\u0026rsquo;s disease)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003er\u0026lsquo;*octitis\u0026rsquo;\u0026nbsp;\u003c/strong\u003e(proctitis)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003er\u0026lsquo;*flammatory bowel disease\u0026rsquo;\u0026nbsp;\u003c/strong\u003e(inflammatory bowel disease)\u003c/p\u003e\n\u003cp\u003e\u0026middot; \u003cstrong\u003er\u0026lsquo;ibd\u0026rsquo;\u0026nbsp;\u003c/strong\u003e(ibd, ibd-u, ibdu)\u003c/p\u003e\n\u003ch3 id=\"_Toc185539719\"\u003e1.3.7 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Statistical Analysis\u003c/h3\u003e\n\u003cp\u003eMissing values were imputed as 0 (not-IBD) to maximise the chances of successfully examining the effects of database gaps on cohort identification in real-world practice. This causes the logistic regression (LR) model to underestimate the total cohort size but has the benefit of reducing the false positive rate.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eHowever, the Jaccard index-based cohort estimation system is unaffected and continues to create an overestimation. This allowed the upper and lower cohort size estimates to be established quickly. Means and medians were calculated as appropriate, depending on skewness using 95%CI or 25%/75% quantiles, as appropriate. The kurtosis was also assessed. The 95% confidence intervals were computed using 1000-fold bootstrapping.\u0026nbsp;\u003c/p\u003e\n\u003ch4 id=\"_Toc185539720\"\u003e1.3.7.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Jaccard Similarity Index\u003c/h4\u003e\n\u003cp\u003eThe Jaccard similarity index(30)\u0026nbsp;(\u003cstrong\u003eFig 2.\u003c/strong\u003e) was used to statistically compare overlaps between database content. This is defined as the intersection size divided by the union size of the two sample sets.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 2]\u003c/strong\u003e\u003c/p\u003e\n\n\u003cp\u003eJaccard index thresholding is somewhat subjective and dependent on both the context and task. However, at a basic level, when comparing databases in this context, a level of \u0026gt; 0.75 would typically be considered high and a level of \u0026lt; 0.35 low(31,32).\u003c/p\u003e\n\u003ch4\u003e1.3.7.2 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Plotting and Statistics\u003c/h4\u003e\n\u003cp\u003ePlotting was performed using Python 3.10.10 with packages matplotlib(33), seaborn(34) and bokeh(35). \u003cstrong\u003e\u003cem\u003eTable\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003cstrong\u003e\u003cem\u003e1\u003c/em\u003e\u003c/strong\u003e lists the performance metrics of interest.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eTable\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003cstrong\u003e\u003cem\u003e1\u003c/em\u003e\u003c/strong\u003e\u003cstrong\u003e\u003cem\u003e:\u003c/em\u003e\u003c/strong\u003e\u003cem\u003e\u0026nbsp;Performance metrics used in this study\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eTerm\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eDescription\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eThe percentage of results that were correct among all results from the system. Calc: (TP+TN)/(TP+FP+TN+FN).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePrecision (PPV)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eAlso called positive predictive value (PPV), it is the percentage of true positive results among all results that the system flagged as positive. Calc: TP/(TP+FP).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eNegative Predictive Value (NPV)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eThe percentage of results that were true negative (TN) among all results that the system flagged as negative. Calc: TN/(TN+FN).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eRecall\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eAlso called sensitivity, it is the percentage of results flagged positive among all results that should have been obtained. Calc: TP/(TP+FN).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSpecificity\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eThe percentage of results that were flagged negative among all negative results. Calc: TN/(TN+FP).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1-Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 459px;\"\u003e\n \u003cp\u003eIn this case, the harmonic mean of PPV/precision and sensitivity/recall is unweighted. Calc: 2 \u0026times; (Precision x Recall)/(Precision + Recall).\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eAbbreviations \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; TP = True Positive, FP = False Positive, FN = False Negative, TP = True Negative\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003ePrecision (PPV) was selected as the primary outcome measure to rank the databases because it offers the most helpful measure of database performance for IBD cohort identification.\u0026nbsp;\u003c/p\u003e\n\u003ch4 id=\"_Toc185539722\"\u003e1.3.7.3 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Jaccard Index Union-Size Calculation\u003c/h4\u003e\n\u003cp\u003eThe Jaccard index can be used to calculate the size of the intersection (overlapping elements) and remaining union (non-overlapping elements) between the two databases. When combined with the known precision for IBD, the total IBD cohort size can be estimated (assuming that the precision is the same for both the intersection and union, and that complex interactions do not exist within databases). This method is mathematically and clinically useful only for calculating an upper estimate as soon as more than two databases are analysed. With this caveat in place, the inference protocol is described as follows:\u003c/p\u003e\n\u003ch4 id=\"_Toc185539724\"\u003e1.3.7.4 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Cohort Size Inference Protocol\u003c/h4\u003e\n\u003cp\u003e1) \u0026nbsp; \u0026nbsp;Start with the primary database (ICD10 codes) and multiply the unique patients in this dataset by the precision of this dataset to obtain a base \u0026lsquo;Combined\u0026rsquo; predicted IBD set.\u003c/p\u003e\n\u003cp\u003e2) \u0026nbsp; \u0026nbsp;Sort other databases by precision (descending).\u003c/p\u003e\n\u003cp\u003e3) \u0026nbsp; \u0026nbsp;Iterate over these databases in order as follows:\u003c/p\u003e\n\u003cp\u003ea. \u0026nbsp; \u0026nbsp;Pick the following highest-precision dataset that has not yet been integrated.\u003c/p\u003e\n\u003cp\u003eb. \u0026nbsp; \u0026nbsp;Calculate the Jaccard index between the current \u0026lsquo;Combined\u0026rsquo; and the next highest precision set.\u003c/p\u003e\n\u003cp\u003ec. \u0026nbsp; \u0026nbsp; Use the recalculated Jaccard similarity index between \u0026lsquo;Combined\u0026rsquo; and the following dataset to estimate the unique patients contributed by that dataset (i.e., those only present on the new dataset\u0026rsquo;s side of the union).\u003c/p\u003e\n\u003cp\u003ed. \u0026nbsp; \u0026nbsp;The unique patients in that dataset are multiplied only by the precision of the dataset to estimate the incremental true positives.\u003c/p\u003e\n\u003cp\u003ee. \u0026nbsp; \u0026nbsp;Add that unique set of patients to the \u0026lsquo;Combined\u0026rsquo; set.\u003c/p\u003e\n\u003cp\u003ef. \u0026nbsp; \u0026nbsp; Repeat the process until no more datasets remain. \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAs elegant as this process may seem, it has significant weaknesses. Primarily, it assumes that precision, assessed against the gold standard, is equally weighted between patients at the intersection and those only in the union. This assumption causes the method to overestimate total cohort size. The code for this algorithm is provided as a fully open source at this URL to maximise transparency and replicability (https://github.com/MattStammers/IBD_Cohort_Size_Estimation_IC-IBD_Study_Part_1).\u0026nbsp;\u003c/p\u003e\n\u003ch3 id=\"_Toc185539756\"\u003e1.3.8 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Multivariate Modelling\u003c/h3\u003e\n\u003cp\u003eMachine learning (ML) logistic regression (LR)(36) classifiers\u0026nbsp;were constructed using 11 available databases.\u0026nbsp;Demographic features such as age, sex, ethnicity, and IMD decile were excluded from the feature set using predefined patterns. Features were standardised using z-scores (mean = 0, standard deviation = 1) before model fitting. To improve the algorithm performance, the L2 (ridge) penalty was used alongside the regularised least absolute shrinkage and selection operator L1 (lasso) penalty in a 50:50 elastic net mix to evaluate the features that could improve the prediction. The lasso shrinks parameters according to their variance, reducing overfitting and enabling automatic variable selection(39), while the ridge provides stability. The optimal degree of regularisation was determined by identifying a tuning parameter (lambda) \u0026lambda; using nested cross-validation (as described below) with a stochastic average gradient augmented (SAGA) solver in light of the sparsity of the underlying data (primarily due to negative imputation). To avoid overfitting and to reduce the number of false-positive predictors, \u0026lambda; was selected to provide a model with an area under the receiver operating characteristic curve (AUC) and one standard error below the best model.\u003c/p\u003e\n\u003cp\u003eAll analyses used pandas, fairlearn, numpy, seaborn, matplotlib, seaborn, and scikit-learn packages in VS Code\u003csup\u003eTM\u003c/sup\u003e and Python 3.10.10 with poetry to manage virtual environments. The code was version-controlled using Git and made available open-source online to maximise replicability and transparency.\u003c/p\u003e\n\u003cp\u003e1.3.8.1.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Cross Validation \u0026amp; Calibration\u003c/p\u003e\n\u003cp\u003eTo evaluate the model\u0026apos;s predictive performance for new cases of the same underlying population (internal validation), nested cross-validation (10-fold for the inner loop; 10-fold/100 repeats for the outer loop) was performed. Platt scaling was used to improve the calibration\u0026nbsp;(37)\u0026nbsp;because the calibration distribution was approximately sigmoid in shape. Discrimination was assessed using the AUC and Brier scores(38). All steps (feature selection, scaling, and threshold selection) were incorporated into the model development and selection process to avoid data leakage that would otherwise result in optimistic performance measures(155). Type 2a validation was performed on the holdout set(39).\u003c/p\u003e\n\u003cp\u003eMeasures of discrimination (precision, recall, harmonic F1-score, Brier score) and calibration were assessed. Calibration was evaluated using three methods.\u003c/p\u003e\n\u003cp\u003e1. \u0026nbsp; \u0026nbsp;A standard calibration curve plotting mean predicted probabilities against observed proportions in bins.\u003c/p\u003e\n\u003cp\u003e2. \u0026nbsp; \u0026nbsp;A locally estimated scatterplot smoothing (LOESS) calibration curve was fitted to the predicted probabilities and observed outcomes.\u003c/p\u003e\n\u003cp\u003e3. \u0026nbsp; \u0026nbsp;A logistic regression calibration plot fitting a logistic regression curve to the same data.\u003c/p\u003e\n\u003cp\u003e1.3.8.1.2 Bias Identification/Error Analysis\u003c/p\u003e\n\u003cp\u003eThe model\u0026apos;s potential for bias was also examined by conducting a stratified analysis of its performance across different demographic groups (race, age, sex, and index of multiple deprivation (IMD)) and comparing the AUC for these subgroups.\u0026nbsp;\u003c/p\u003e"},{"header":"Results","content":"\u003ch3\u003e1.4.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Total Study Cohort\u003c/h3\u003e\n\u003cp\u003eBetween 2007 and 2023, 52,332 non-two-week wait referrals were made for 37,947 individual patients. The gold-standard validation cohort consisted of 2,800 patients: 2,180(78%) with IBD and 619(22%) non-IBD cases. The randomly seeded validation subset (30%) contained 841 patients, 664(79%) with IBD and 176(21%) without IBD.\u003c/p\u003e\n\u003cp\u003eThe distribution of these patients in each database and temporally by year of the first referral are shown in \u003cstrong\u003eFig 3\u003c/strong\u003e.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 3]\u003c/strong\u003e\u003c/p\u003e\n\u003ch4\u003e1.4.1.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Cohort Demographics\u003c/h4\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2\u003c/strong\u003e shows the demographic characteristics of the entire cohort. The skewness and kurtosis for age were 0.017 and -1.076, respectively, and those for IMD were -0.111 and -1.181, respectively.\u0026nbsp;\u003c/p\u003e\n\u003cp id=\"_Toc185539857\"\u003e\u003cstrong\u003e\u003cem\u003eTable 2:\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003cem\u003eFull Cohort Demographics\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"633\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eFeature\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eMean\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eMedian\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge at Point of Referral\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e51.79yrs \u003cem\u003e(95%CI:51.59-51.92)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e52.22yrs \u003cem\u003e(IQR:32.4)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSex (Female)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e60.27%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEthnicity (White)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e85.04%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eIMD Decile\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e5.91 \u003cem\u003e(95%CI:5.88-5.94)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e6 \u003cem\u003e(IQR:4)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eUrgent Referrals\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e21.34%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 125px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLocal Referrals From Southampton Catchment\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 290px;\"\u003e\n \u003cp\u003e83.01%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eTable 2 describes the cohort demographics of patients included in the study\u003c/em\u003e\u003c/p\u003e\n\u003ch3 id=\"_Toc185539728\"\u003e1.4.2 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Coding, Event and Registry-Based Predictions\u003c/h3\u003e\n\u003cp\u003eBy examining the relationships between each dataset and the gold-standard validation cohort, the baseline precision and recall were established for each database, as shown in \u003cstrong\u003eTable 3\u003c/strong\u003e.\u003c/p\u003e\n\u003cp id=\"_Toc185539858\"\u003e\u003cstrong\u003e\u003cem\u003eTable 3:\u003c/em\u003e\u003c/strong\u003e\u003cem\u003e\u0026nbsp;Coding, Registry \u0026amp; Event-Based Predictions\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" title=\"Full Table of Results\" summary=\"This table details the full table of results for the databases and simple models compared to detect the IBD cohort. \" width=\"585\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eDatabase\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCoverage\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAccuracy\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePrecision\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eRecall\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSpecificity\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eNPV\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eF1 Score\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCoding\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eICD10 Codes\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e802 (95.48%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.92 (0.9 - 0.94)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.96 (0.94 - 0.98)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.93 (0.91 - 0.95)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.86 (0.81 - 0.91)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.78 (0.72 - 0.84)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.95 (0.93 - 0.96)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eOPCS4 Codes\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e274 \u0026nbsp; \u0026nbsp; \u0026nbsp; (32.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.32 (0.26 - 0.38)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.91 (0.81 - 0.98)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.18 (0.12 - 0.23)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.92 \u0026nbsp;(0.84 - 0.99)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.21 \u0026nbsp;(0.15 - 0.26)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.29 (0.22 - 0.37)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eRegistries\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePatient Portal\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e428 (50.95%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.99 (0.98 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.99 (0.98 - 1.0)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (0.99 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEPR IBD Registry\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e571 (67.98%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.99 (0.98 - 0.99)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.99 (0.98 - 0.99)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.99 (0.99 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEvents\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCytokine Modulator Prescriptions\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e198 (23.57%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eFlare Calls\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e505 (60.12%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.89 (0.86 - 0.91)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.89 (0.86 - 0.91)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e1.0 (1.0 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0 (0.0 - 0.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.94 (0.92 - 0.96)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eIBD Clinic Appointments\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e664 (79.05%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.38 (0.34 - 0.42)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.82 (0.77 - 0.87)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.31 (0.27 - 0.35)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.71 (0.62 - 0.79)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.19 (0.15 - 0.23)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.45 (0.4 - 0.49)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCalprotectin \u0026gt; 50\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 78px;\"\u003e\n \u003cp\u003e533 (63.45%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.32 (0.28 - 0.36)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.82 (0.75 - 0.9)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.19 (0.15 - 0.22)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.84 (0.77 - 0.91)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.2 (0.17 - 0.24)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 65px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.31 (0.26 - 0.35)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eTable 3: Baseline ground truth as established using the validation cohort comparing coverage, precision and recall for each dataset. The F1 score is given for each dataset.\u003c/em\u003e\u003c/p\u003e\n\u003ch3 id=\"_Toc185539729\"\u003e1.4.3 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Simple String Regex Search Model\u003c/h3\u003e\n\u003cp\u003eThe string regression search model is the most straightforward natural language processing (NLP) based cohort identification model. It was used as a proxy for the likelihood of IBD among free-text documents. \u003cstrong\u003eTable 4\u003c/strong\u003e.\u003c/p\u003e\n\u003cp id=\"_Toc185539859\"\u003e\u003cstrong\u003e\u003cem\u003eTable 4:\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003cem\u003eString Search Model Comparison\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" title=\"Full Table of Results\" summary=\"This table details the full table of results for the databases and simple models compared to detect the IBD cohort. \" width=\"585\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 91px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eDatabase\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 48px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAlgo\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCoverage\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 69px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAccuracy\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 60px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePrecision\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eRecall\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSpecificity\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eNPV\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eF1 Score\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 91px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEndoscopy Records\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 48px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSimple Regex\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e738 (87.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 69px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.72 (0.69 - 0.75)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 60px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.95 (0.93 - 0.97)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.68 (0.64 - 0.72)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.86 (0.8 - 0.91)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.41 (0.36 - 0.47)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.79 (0.76 - 0.82)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 91px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eClinical Letters\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 48px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSimple Regex\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e794 (94.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 69px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.83 (0.8 - 0.86)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 60px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.83 (0.8 - 0.86)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.99 (0.98 - 1.0)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.24 (0.18 - 0.31)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.89 (0.78 - 0.97)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.9 (0.89 - 0.92)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 91px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eHistopathology Records\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 48px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eSimple Regex\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 70px;\"\u003e\n \u003cp\u003e506 (60.24%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 69px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.7 (0.66 - 0.74)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 60px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.75 (0.71 - 0.79)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.89 (0.86 - 0.92)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 57px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.16 (0.1 - 0.22)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.34 (0.23 - 0.46)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e\u003cem\u003e0.81 \u0026nbsp; \u0026nbsp; (0.79 - 0.84)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eTable 4 describes the results of the regex string search models across the clinical, endoscopy and histopathology records.\u003c/em\u003e\u003c/p\u003e\n\u003ch3 id=\"_Toc185539730\"\u003e1.4.4 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Database Cardinality\u003c/h3\u003e\n\u003cp\u003eCardinality measures the uniqueness or distinctiveness of elements within a database table. Given the sheer number of intersecting sets in this study, the results were best visualised using UpSet plots (\u003cstrong\u003eFig 4\u003c/strong\u003e). These are superior to Venn diagrams for visualising data sets with more than three intersecting sets in a matrix.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 4]\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSignificant overlaps between suspected IBD cases in databases are an exception rather than a rule. Of the 8,212 unique patients with at least one ICD-10 code for IBD, only 476 (5.8%) were found in ten or more clinical databases. Similarly, 7,570 individuals with suspected IBD were identified in a single database.\u003c/p\u003e\n\u003ch3\u003e1.4.5 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Jaccard Similarity Indices\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eFig 5.\u0026nbsp;\u003c/strong\u003eshows the Jaccard indices(30) across all 11 databases before and after the application of IBD prediction criteria.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 5]\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eJaccard indices were high for the complete clinical coding database, clinical letters (0.88), and endoscopy records (0.76). The same was true for endoscopy and clinical letters (0.74). Moderately high indices existed between coding and appointments (0.55), OPCS4 codes and ICD10 coding (0.38), endoscopy (0.39), and clinic letters (0.37). Moderately high indices existed between patient portal registration and registration within EPR (0.58), flare calls (0.36), calprotectin testing (0.37), and cytokine modulator usage (0.38). The same was observed between endoscopies, appointments (0.48), and GI histopathology (0.43), while appointments intersected with clinical letters (0.53) and histopathology (0.45).\u003c/p\u003e\n\u003cp\u003eHowever, these indices were substantially altered in the suspected IBD cohort, with no high Jaccard indices. IBD ICD10 codes intersected moderately with the registry databases (0.4-0.49), clinic letters (0.54), and endoscopies (0.45). The EPR registry intersects with endoscopy (0.47) and clinical letters (0.51), whereas the patient portal intersects with flare line calls (0.38), cytokine modulator prescriptions (0.37), clinical letters (0.45), calprotectins \u0026gt;50 (0.42), IBD specialist appointments (0.37), and endoscopy (0.41). Endoscopy overlapped with clinical letters suggesting IBD (0.43). IBD specialist appointments also overlapped with clinical letters suggestive of IBD (0.36). The remainder of the database intersections are low, ranging from (0.19-0.35), except for IBD OPCS4 codes, where the range is even lower (0.04-0.07).\u003c/p\u003e\n\u003ch3 id=\"_Toc185539732\"\u003e1.4.6 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Cohort Size Estimation by Recursive Jaccard Similarity Database Inference\u003c/h3\u003e\n\u003cp\u003eThe full results of the inference process are described in \u003cstrong\u003e\u003cem\u003eTable 5\u003c/em\u003e\u003c/strong\u003e.\u003c/p\u003e\n\u003cp id=\"_Toc185539860\"\u003e\u003cstrong\u003e\u003cem\u003eTable 5:\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003cem\u003eFull Recursive Jaccard Similarity Cohort Size Estimation\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eDatabase\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eFlagged Positive Cases\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eJaccard with Combined\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eIntersection\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eUnique\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePrecision\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eIncremental TPs\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCumulative TPs\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eICD10 Codes\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8337\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8337\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.96\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8004\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8004\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCytokine Modulator Prescriptions\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e1762\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.205\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e1718\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e44\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e1.00\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e44\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8048\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePatient Portal\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e3643\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.408\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e3483\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e160\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.99\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e158\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8206\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEPR IBD Registry\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4312\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.501\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4288\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e24\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.99\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e24\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8230\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eEndoscopy Records\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4327\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.447\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e3982\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e345\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.95\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e328\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8557\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eOPCS4 Codes\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e1190\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.052\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e501\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e689\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.91\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e627\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e9184\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eFlare Calls\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e7705\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.353\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4519\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e3186\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.89\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e2836\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e12020\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eClinical Letters\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e14984\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.461\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e8761\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e6223\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.83\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e5165\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e17185\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eIBD Clinic Appointments\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e5520\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.237\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4707\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e813\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.82\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e667\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e17852\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eCalprotectin \u0026gt;50\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e4000\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.184\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e3708\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e292\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.82\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e239\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e18091\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eHistopathology Records\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e6070\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.257\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e5352\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e718\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e0.7\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e503\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e18594\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAll Integrated\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e20831\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e18594\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eTable 5: Describes the results of the recursive cohort size estimation strategy pursued according to the defined protocol. TP\u0026rsquo;s \u0026ndash; True Positives.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe largest unaccounted-for group emerged from clinic letters (n=5,165), followed by flare calls (n=2,836). These factors alone accounted for 7,889 additional patients with uncoded IBD.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e1.4.7 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Cohort Size Estimation by Penalised Logistic Regression\u003c/h3\u003e\n\u003cp\u003eThe following estimates of the total IBD population size per \u003cstrong\u003eTable 6\u003c/strong\u003e were obtained by applying thresholding to the penalised logistic regression model.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 6:\u0026nbsp;\u003c/strong\u003eIBD LR Predictions by Threshold\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eThreshold\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePrecision (95%CI)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eRecall (95%CI)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eAccuracy (95%CI)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003ePredicted IBD Total (95%CI)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eActual Predicted IBD Total (95%CI)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.25\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.83 (\u003c/em\u003e\u003cem\u003e0.82 - 0.84\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.92 (\u003c/em\u003e\u003cem\u003e0.91 - 0.93)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.79 (\u003c/em\u003e\u003cem\u003e0.78 - 0.81)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e15373 (\u003c/em\u003e\u003cem\u003e15180 - 15554)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e12760\u0026nbsp;\u003c/em\u003e\u003cem\u003e(12464 - 13040)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.31\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.84 (\u003c/em\u003e\u003cem\u003e0.83 - 0.86\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.92 (\u003c/em\u003e\u003cem\u003e0.91 - 0.93\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.8 (\u003c/em\u003e\u003cem\u003e0.79 - 0.82\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e13507 (\u003c/em\u003e\u003cem\u003e13320 - 13680)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e11346 (\u003c/em\u003e\u003cem\u003e11119 - 11596)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.38\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.85 (\u003c/em\u003e\u003cem\u003e0.83 - 0.86)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.92 (\u003c/em\u003e\u003cem\u003e0.91 - 0.93\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.81 (\u003c/em\u003e\u003cem\u003e0.79 - 0.82\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e11687 (\u003c/em\u003e\u003cem\u003e11517 - 11868\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e9934 (\u003c/em\u003e\u003cem\u003e9687 - 10125)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.44\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.85 (\u003c/em\u003e\u003cem\u003e0.84 - 0.87)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.92 (\u003c/em\u003e\u003cem\u003e0.9 - 0.93\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.81 (\u003c/em\u003e\u003cem\u003e0.79 - 0.82\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e10318 (\u003c/em\u003e\u003cem\u003e10167 - 10490\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e8770 (\u003c/em\u003e\u003cem\u003e8567 - 8986)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.5\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.85 (\u003c/em\u003e\u003cem\u003e0.84 - 0.87\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.91 (\u003c/em\u003e\u003cem\u003e0.9 - 0.92\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.81 (\u003c/em\u003e\u003cem\u003e0.79 - 0.82)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e9619 (\u003c/em\u003e\u003cem\u003e9457 - 9795)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e8176 (\u003c/em\u003e\u003cem\u003e7990 - 8398)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.56\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.86 (\u003c/em\u003e\u003cem\u003e0.84 - 0.87\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.91 (\u003c/em\u003e\u003cem\u003e0.9 - 0.92\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.81 (\u003c/em\u003e\u003cem\u003e0.8 - 0.83)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e9432 (\u003c/em\u003e\u003cem\u003e926 -9593\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e8112 (\u003c/em\u003e\u003cem\u003e7890 - 8290)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.62\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.87 (\u003c/em\u003e\u003cem\u003e0.85 - 0.88\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.91 (\u003c/em\u003e\u003cem\u003e0.9 - 0.92)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.82 (\u003c/em\u003e\u003cem\u003e0.81 - 0.83\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e9299 (\u003c/em\u003e\u003cem\u003e9147 - 9458\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e8090 (\u003c/em\u003e\u003cem\u003e7869 - 8257\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.69\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.88 (\u003c/em\u003e\u003cem\u003e0.86 - 0.89\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.91 (\u003c/em\u003e\u003cem\u003e0.9 -0.92)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.83 (\u003c/em\u003e\u003cem\u003e0.81 - 0.84)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e9057 (\u003c/em\u003e\u003cem\u003e8890 - 9228\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e7970 (\u003c/em\u003e\u003cem\u003e7745 - 8124\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003e0.75\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.89 (\u003c/em\u003e\u003cem\u003e0.87 - 0.9\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.9 (\u003c/em\u003e\u003cem\u003e0.89 - 0.91\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e0.83 (\u003c/em\u003e\u003cem\u003e0.82 - 0.84\u003c/em\u003e\u003cem\u003e)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e8567 (\u003c/em\u003e\u003cem\u003e8416 - 8719)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cem\u003e7625 (\u003c/em\u003e\u003cem\u003e7415 - 7767)\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eTable 6: Describes the full results of thresholding the LR model at different levels.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe optimal threshold was 0.586, detecting 8,060 true positive IBD patients with a global AUROC of 0.85 and a PR-AUC of 0.95 against the validation set. By adding only the true positives from a single database (clinic letters) to this total (n=4,938), a final estimated total of \u003cstrong\u003e12,998\u0026nbsp;\u003c/strong\u003etrue-positive IBD patients was reached.\u003c/p\u003e\n\u003ch4\u003e1.4.7.1 \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Calibration \u0026amp; Bias\u003c/h4\u003e\n\u003cp\u003eCalibration remains an issue for this model. \u003cstrong\u003e\u003cem\u003eFigure 6\u0026nbsp;\u003c/em\u003e\u003c/strong\u003edemonstrates that the model tends to underpredict IBD patients at lower predicted probabilities but overpredicts at \u0026gt;0.6, despite Platt scaling, which improved the Brier score from 0.0515 to 0.0461.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e[FIGURE 6]\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo significant bias was detected in this model for levels of deprivation or sex. However, the model performed significantly better among Caucasians (AUC-0.91) than among Asians (AUC-0.83) or Africans (AUC-0.76). Additionally, performance decreased in older age groups, declining from 0.93 in the 20-39 age range to 0.85 for 50-59 year olds.\u0026nbsp;\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003e Accurate population-level data have substantial implications for policy formation, departmental resourcing, and the avoidance of discrimination in patient care, research, and service improvement. This study highlights significant flaws in relying on billing/read codes and medication data alone to identify clinical cohorts at both local and population levels. Only 8048 (61.6%) patients were identifiable from billing codes and medication data combined from an actual local IBD patient cohort of at least 12,998 individuals.\u003c/p\u003e \u003cp\u003eThe major strengths of this work include the real-world nature of the study, a robust approach to validation, the variety of databases investigated, and the simplicity of the methodology, aiding the ease of replicability in other settings without the need for advanced data science capability. This study revealed significant flaws in current IBD clinical cohort identification assumptions. Further cohort capture can be achieved by adding additional databases if sensible defragmentation attempts are made.\u003c/p\u003e \u003cp\u003eWeaknesses include the study's single-center nature, meaning that prevalence cannot be accurately calculated, combined with the fact that actual positive cases are unlikely to be equally distributed between intersections and unions, thus reducing the accuracy. As the patients in the validation cohort were randomly drawn from a higher-probability IBD sample (present in at least two databases at the outset), they did not adequately represent the cohort fringes where only a single database node value existed. This means that the LR model underpredicts these cases because the classifier is not a good fit for them. This single-node problem cannot be overcome by adding additional features (such as age, ethnicity, and IMD) to the model. Such attempts typically compounded the problem without substantially ameliorating the model biases or calibration issues. Because two-week wait referrals were excluded from the cohort, the total cohort is likely to be even more significant than that reported here. Finally, some of the clinical information (e.g., clinical letters scanned as images) was unavailable for analysis in this study, suggesting that even more patients who may be available for retrieval were optical character recognition in place.\u003c/p\u003e \u003cp\u003eClinical letters are the most important contributors to patients outside the ICD-10 codes. Montoto et al. (\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e) (2022) claimed to achieve 0.88 precision and 0.98 recall for diagnosing Crohn\u0026rsquo;s disease within a large Spanish multicenter cohort. However, the free-text precision of the simple regex algorithm we derived here was only 0.83 and the recall 0.99 for detecting positive IBD cases, which needs to be improved in subsequent work. However, because they used ICD codes alone to generate their gold-standard validation cohorts, if our results were replicated in that study, at least 37.5% of the actual IBD cases would have been missing from their cohort at the outset and 4% of their true-positive cases would have, in fact, not had IBD.\u003c/p\u003e \u003cp\u003eThis is the first time that the fully complicated impacts of database fragmentation have been documented for IBD cohorts, building on the work of others(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). This study primarily examined the extent of the problem. However, the principles of 1) exposing many different databases, 2) validating a gold standard cohort, and 3) using ML to identify a complete cohort are transferable to most other clinical domains and diseases. To make this process more scalable, novel methodologies to standardise datasets, positively identify patients, and compare databases across a graph of tables are required. Success in these endeavors will positively impact clinical research, population health, and frontline clinical care by highlighting the true IBD clinical cohorts of local teams.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eUsing historical billing/reading codes and medication data alone in the context of IBD can substantially underestimate the total local IBD clinical cohort size. Further research on methodologies to overcome this problem is required, particularly for retrieving IBD cases from free-text clinical records.\u003c/p\u003e "},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eContributors:\u0026nbsp;\u003c/strong\u003eMS performed all analyses and final data preparation. SS led and supervised the derivation of the gold-standard cohort, which MS then double-checked. MS drafted the first version of the manuscript. MG, RN, CM, CK, FC, SS, and JB provided critical feedback regarding the manuscript.\u003cstrong\u003e\u0026nbsp;MS\u0026nbsp;\u003c/strong\u003eis the primary guarantor for the review and the corresponding author.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements:\u003c/strong\u003e To make this project possible, we acknowledge the SETT data and AI team (in particular Mr Cai Davis and Dr Michael George), the CIRU/AXIS team (Dr. Ashley Heinson) \u0026ndash; bioinformatician who checked the statistics, and three clinical junior doctors who helped with the IBD manual chart review: Dr Perez Pablo, Dr Maryam Al-Ezairej, and Dr Mahmoud Abosamra, as well as Dr Richard Felwick who helped feed back on some early iterations of the project. None of them were involved in the analysis, and the corresponding author assumes responsibility for all data handling and analytics.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003eThis work was indirectly supported by the Research Leaders\u0026apos; Program through funding provided to MS by the Southampton Academy of Research (SoAR) and University Hospital, Southampton. The protocol was independently developed.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests:\u003c/strong\u003e RN received an educational grant from Pentax Medical. MS and MG attended the fully funded Dr. Falk Symposium on AI in Gastroenterology. CK is SRO for the Wessex SDE and FC has received grants/consultancy Fees/speaker Fees from Jassen, Celltrion, Biogen, Samsung, Sandoz, MSD, Abbvie, Pfizer, Hospiral and Gilead and is also clinical lead for the UK IBD Registry.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePatient consent for publication:\u003c/strong\u003e Not applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability:\u0026nbsp;\u003c/strong\u003eAll data generated or analysed during this study is unavailable to protect patient privacy as the patients involved did not consent to sharing their data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode Availability:\u0026nbsp;\u003c/strong\u003eAll codes used in the analytics for this project are made available open source on GitHub at https://github.com/MattStammers/IBD_Cohort_Size_Estimation_IC-IBD_Study_Part_1 \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eModel Availability:\u0026nbsp;\u003c/strong\u003eThe LR model developed in this project is relatively custom to our local system; therefore, it has not been open-sourced but can be shared on request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePatient and Public Involvement:\u003c/strong\u003e An IBD patient from our local IBD patient panel helped develop the ethics application and study protocol.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHuman Ethics and Consent to Participate Declarations:\u0026nbsp;\u003c/strong\u003eThe Wessex REC and HRA provided research ethics board approval for this study (IC-IBD -23/SC/0152) \u0026nbsp;on 16/05/2023 (https://www.hra.nhs.uk/planning-and-improving-research/application-summaries/research-summaries/ic-ibd-ibd-cohort-identification-study/).\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eChen TK, Knicely DH, Grams ME. Chronic Kidney Disease Diagnosis and Management: A Review. JAMA. 2019;322(13):1294\u0026ndash;304.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePark JG, Chen XD, Clontz M, Begaj T, Runner MM, Wolfe JD. Coding of Geographic Atrophy and Exudative Age-related Macular Degeneration. Ophthalmol Retina. 2023;7(7):644\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBaumgart DC, Sandborn WJ. Inflammatory bowel disease: clinical aspects and established and evolving therapies. The Lancet. 2007;369(9573):1641\u0026ndash;57.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNikolaus S, Schreiber S. Diagnostics of Inflammatory Bowel Disease. Gastroenterology. 2007;133(5):1670\u0026ndash;89.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBurisch J, Jess T, Martinato M, Lakatos PL. The burden of inflammatory bowel disease in Europe. J Crohns Colitis. 2013;7(4):322\u0026ndash;37.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGhosh N, Premchand P. A UK cost of care model for inflammatory bowel disease. Frontline Gastroenterol. 2015;6(3):169\u0026ndash;74.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuevara M, Salamanca-Fern\u0026aacute;ndez E, Miqueleiz E, Gavrila D, Amiano P, Bonet C, et al. Inflammatory Potential of the Diet and Incidence of Crohn\u0026rsquo;s Disease and Ulcerative Colitis in the EPIC-Spain Cohort. Nutrients. 2021;13(7):2201.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVavricka SR, Brun L, Ballabeni P, Pittet V, Vavricka BMP, Zeitz J, et al. Frequency and Risk Factors for Extraintestinal Manifestations in the Swiss Inflammatory Bowel Disease Cohort. Off J Am Coll Gastroenterol ACG. 2011;106(1):110.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMolodecky NA, Panaccione R, Ghosh S, Barkema HW, Kaplan GG, on Behalf of the Alberta Inflammatory Bowel Disease Consortium. Challenges associated with identifying the environmental determinants of the inflammatory bowel diseases. Inflamm Bowel Dis. 2011;17(8):1792\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCharpentier C, Salleron J, Savoye G, Fumery M, Merle V, Laberenne JE, et al. Natural history of elderly-onset inflammatory bowel disease: a population-based cohort study. Gut. 2014;63(3):423\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarlson AM, Dennis C, Singh I, Altaher W, Cleary J, Gohil H, et al. S1267 Investigating Misdiagnosis Rates of Inflammatory Bowel Disease: A Retrospective Single-Center Review. Off J Am Coll Gastroenterol ACG. 2024;119(10S):S902.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWoo DH, Kim KO, Kang MK, Lee SH, Jang BI, Kim TN. Predictors and clinical outcomes of follow-up loss in patients with inflammatory bowel disease: Follow up loss in inflammatory bowel disease. J Gastroenterol Hepatol. 2018;33(11):1834\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAnanthakrishnan AN, Cai T, Savova G, Cheng SC, Chen P, Perez RG, et al. Improving Case Definition of Crohnʼs Disease and Ulcerative Colitis in Electronic Medical Records Using Natural Language Processing: A Novel Informatics Approach. Inflamm Bowel Dis. 2013;19(7):1411\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStapley SA, Rubin GP, Alsina D, Shephard EA, Rutter MD, Hamilton WT. Clinical features of bowel disease in patients aged\u0026thinsp;\u0026lt;\u0026thinsp;50 years in primary care: a large case-control study. Br J Gen Pract. 2017;67(658):e336\u0026ndash;44.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBhangu A, Nepogodiev D, Taylor C, Durkin N, Patel R. Accuracy of clinical coding from 1210 appendicectomies in a British district general hospital. Int J Surg. 2012;10(3):144\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNaran S, Hudovsky A, Antscherl J, Howells S, Nouraei SAR. Audit of accuracy of clinical coding in oral surgery. Br J Oral Maxillofac Surg. 2014;52(8):735\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNouraei SAR, Hudovsky A, Frampton AE, Mufti U, White NB, Wathen CG, et al. A Study of Clinical Coding Accuracy in Surgery: Implications for the Use of Administrative Big Data for Outcomes Management. Ann Surg. 2015;261(6):1096\u0026ndash;107.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNouraei SAR, O\u0026rsquo;Hanlon S, Butler CR, Hadovsky A, Donald E, Benjamin E, et al. A multidisciplinary audit of clinical coding accuracy in otolaryngology: financial, managerial and clinical governance considerations under payment-by-results. Clin Otolaryngol. 2009;34(1):43\u0026ndash;51.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHorsky J, Drucker EA, Ramelson HZ. Accuracy and Completeness of Clinical Coding Using ICD-10 for Ambulatory Visits. AMIA Annu Symp Proc AMIA Symp. 2017;2017:912\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCampbell S, Giadresco K. Computer-assisted clinical coding: A narrative review of the literature on its benefits, limitations, implementation and impact on clinical coding professionals. Health Inf Manag J. 2020;49(1):5\u0026ndash;18.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRye C, Rubin KH, Moller FT, Julsgaard M, Jess T, Andersen V. Positive Predictive Value of Diagnostic Codes for Inflammatory Bowel Disease in the Danish National Patient Registry Among Individuals 50\u0026thinsp;+\u0026thinsp;Years, Using Patient Records as Reference Standard. Clin Epidemiol. 2021;Volume 13:335\u0026ndash;44.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJones GR, Lyons M, Plevris N, Jenkinson PW, Bisset C, Burgess C, et al. IBD prevalence in Lothian, Scotland, derived by capture\u0026ndash;recapture methodology. Gut. 2019;68(11):1953\u0026ndash;60.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStammers M, Ramgopal B, Owusu Nimako A, Vyas A, Nouraei R, Metcalf C, et al. A foundation systematic review of natural language processing applied to gastroenterology \u0026amp; hepatology. BMC Gastroenterol. 2025;25(1):58.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCollins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Intern Med. 2015;162(1):55\u0026ndash;63.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRCP London [Internet]. 2015 [cited 2023 Nov 24]. IBD organisational audit. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.rcplondon.ac.uk/projects/ibd-organisational-audit\u003c/span\u003e\u003cspan address=\"https://www.rcplondon.ac.uk/projects/ibd-organisational-audit\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAustin PC, Steyerberg EW. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat Methods Med Res. 2017;26(2):796\u0026ndash;808.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePate A, Riley RD, Collins GS, van Smeden M, Van Calster B, Ensor J, et al. Minimum sample size for developing a multivariable prediction model using multinomial logistic regression. Stat Methods Med Res. 2023;32(3):555\u0026ndash;71.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSarkar S, Livingstone R, Borca F, Stammers M, Gwiggner M. PTH-32 Development of a novel electronic referral grading \u0026amp; triage system. Gut. 2021;70(Suppl 4):A186\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHumphreys BL, Lindberg DA. The UMLS project: making the conceptual connection between users and the information they need. Bull Med Libr Assoc. 1993;81(2):170\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReal R, Vargas JM. The Probabilistic Basis of Jaccard\u0026rsquo;s Index of Similarity. Syst Biol. 1996;45(3):380\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDharavath R, Singh AK. Entity Resolution-Based Jaccard Similarity Coefficient for Heterogeneous Distributed Databases. In: Satapathy SC, Raju KS, Mandal JK, Bhateja V, editors. Proceedings of the Second International Conference on Computer and Communication Technologies. New Delhi: Springer India; 2016. p. 497\u0026ndash;507.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFletcher S, Islam MZ. Comparing sets of patterns with the Jaccard index. Australas J Inf Syst [Internet]. 2018 Mar 7 [cited 2024 Dec 18];22. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ajis.aaisnet.org/index.php/ajis/article/view/1538\u003c/span\u003e\u003cspan address=\"https://ajis.aaisnet.org/index.php/ajis/article/view/1538\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHunter JD. Matplotlib: A 2D Graphics Environment. Comput Sci Eng. 2007;9(03):90\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWaskom M. seaborn: statistical data visualization. J Open Source Softw. 2021;6(60):3021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJolly K. Hands-On Data Visualization with Bokeh: Interactive web plotting for Python using Bokeh. Packt Publishing Ltd; 2018. 168 p.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJr DWH, Lemeshow S, Sturdivant RX. Applied Logistic Regression. John Wiley \u0026amp; Sons; 2013. 528 p.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePlatt J. Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods. In 1999 [cited 2025 Feb 1]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.semanticscholar.org/paper/Probabilistic-Outputs-for-Support-vector-Machines-Platt/42e\u003c/span\u003e\u003cspan address=\"https://www.semanticscholar.org/paper/Probabilistic-Outputs-for-Support-vector-Machines-Platt/42e\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e5ed832d4310ce4378c44d05570439df28a393\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010;63(8):938\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMontoto C, Gisbert JP, Guerra I, Plaza R, Pajares Villarroya R, Moreno Almaz\u0026aacute;n L, et al. Evaluation of Natural Language Processing for the Identification of Crohn Disease-Related Variables in Spanish Electronic Health Records: A Validation Study for the PREMONITION-CD Project. JMIR Med Inform. 2022;10(2):e30345.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"digestive-diseases-and-sciences","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ddsj","sideBox":"Learn more about [Digestive Diseases and Sciences](http://link.springer.com/journal/10620)","snPcode":"10620","submissionUrl":"https://submission.nature.com/new-submission/10620/3","title":"Digestive Diseases and Sciences","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Inflammatory bowel disease, Cohort identification, Data fragmentation, Algorithms","lastPublishedDoi":"10.21203/rs.3.rs-6298636/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6298636/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eIntroduction\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eInflammatory bowel disease (IBD) cohort identification typically relies primarily on read/billing codes, which may miss some patients. Attempts have been made to add medication records and other datasets to improve the cohort capture. However, a complete picture cannot typically be obtained because of database fragmentation and missingness.\u003c/p\u003e\n\u003cp\u003eThis study used novel cohort retrieval methods to identify the total IBD cohort from a large university teaching hospital with a specialist intestinal failure unit.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eBetween 2008 and 2023, 11 clinical databases (ICD10 codes, OPCS4 codes, clinician-entry IBD registry, IBD patient portal, prescriptions, biochemistry, flare line calls, clinic appointments, endoscopy, histopathology, and clinic letters) were identified as having the potential to help identify local IBD patients.\u003c/p\u003e\n\u003cp\u003eA gold-standard validation cohort was created through a manual chart review. A regex string search for normalised IBD terms was used on the three free-text databases (endoscopy, histopathology, and clinic letters) to identify patients more likely to have IBD. The 11 databases were compared statistically to assess cardinality and Jaccard Similarity in order to derive informed estimates of the total IBD population. A penalised logistic regression (LR) classifier was trained on 70% of the data and validated against a 30% holdout set to individually identify IBD patients.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eThe gold-standard validation cohort comprised 2,800 patients: 2,180(78%) with IBD and 619(22%) non-IBD cases. The precision for IBD ranged from 0.75-1 to 0.18-1. All the databases contained unique patients that were not covered by the Casemix ICD-10 database. The Jaccard similarity estimation predicted 18,594, but this represents an overestimation. The penalised LR model (AUROC: 0.85 - Validation set) confidently identified 8,060 patients with IBD (threshold: 0.586), although at lower thresholds (0.25), the model identified 12,760 patients with a higher recall of 0.92. By combining the true-positive cases from the LR model with likely true-positive IBD clinic letters, a final estimate of \u003cstrong\u003e12,998\u003c/strong\u003e patients with IBD was obtained. True positives from ICD 10 codes combined with medication (n = 8,048) covered only 61.6% of the total local IBD population, indicating that the present methods missed up to \u003cstrong\u003e38.4%\u003c/strong\u003e of IBD patients.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eDiagnostic billing codes and medication data alone cannot accurately identify complete IBD cohorts. A multimodal cross-database model can partially compensate for this deficit. To improve this situation, more robust natural language processing (NLP)-based identification mechanisms are required to improve IBD cohort identification.\u003c/p\u003e","manuscriptTitle":"Identification of Cohorts with Inflammatory Bowel Disease Amidst Fragmented Clinical Databases via Machine Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-16 08:49:02","doi":"10.21203/rs.3.rs-6298636/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-06-25T17:49:27+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-09T10:28:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"173173289010879468368163554993928413255","date":"2025-03-28T05:02:11+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"70603749158124997758215508901468043756","date":"2025-03-26T21:54:07+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-26T21:50:04+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-03-25T21:15:42+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-03-25T12:50:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"Digestive Diseases and Sciences","date":"2025-03-24T23:16:19+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"digestive-diseases-and-sciences","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ddsj","sideBox":"Learn more about [Digestive Diseases and Sciences](http://link.springer.com/journal/10620)","snPcode":"10620","submissionUrl":"https://submission.nature.com/new-submission/10620/3","title":"Digestive Diseases and Sciences","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"e45de560-bfbc-4f81-b007-919ca3bcc8be","owner":[],"postedDate":"April 16th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-08-18T15:59:55+00:00","versionOfRecord":{"articleIdentity":"rs-6298636","link":"https://doi.org/10.1007/s10620-025-09323-1","journal":{"identity":"digestive-diseases-and-sciences","isVorOnly":false,"title":"Digestive Diseases and Sciences"},"publishedOn":"2025-08-13 15:57:15","publishedOnDateReadable":"August 13th, 2025"},"versionCreatedAt":"2025-04-16 08:49:02","video":"","vorDoi":"10.1007/s10620-025-09323-1","vorDoiUrl":"https://doi.org/10.1007/s10620-025-09323-1","workflowStages":[]},"version":"v1","identity":"rs-6298636","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6298636","identity":"rs-6298636","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}