Improving the Accuracy and Precision of Disease Identification When Utilizing Ehr Data for Research: the Case for Hepatocellular Carcinoma

Improving the Accuracy and Precision of Disease Identification When Utilizing Ehr Data for Research: the Case for Hepatocellular Carcinoma | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Improving the Accuracy and Precision of Disease Identification When Utilizing Ehr Data for Research: the Case for Hepatocellular Carcinoma Carrie R. Wong, Yvonne N. Flores, Analissa Avila, Lina Tieu, Catherine M. Crespi, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4993106/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Oct, 2025 Read the published version in BMC Research Notes → Version 1 posted 12 You are reading this latest preprint version Abstract Objective We assessed the performance of ICD codes to identify patients with hepatocellular carcinoma (HCC) in a large academic health system and determined whether employing an algorithm using a combination of ICD codes could deliver higher accuracy and precision than single ICD codes in identifying HCC cases using electronic health record (EHR) data. Results The use of a single ICD code entry for HCC (ICD-9-CM 155.0 or ICD-10-CM C22.0) in our cohort of 1,007 established ambulatory care patients with potential HCC yielded 58% false positives (not true HCC cases) based on chart reviews. We developed an ICD code-based algorithm that prioritized positive predictive value (PPV), F-score, and accuracy to minimize false positives and negatives. The highest performing algorithm required at least 10 ICD code entries for HCC and the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies. The algorithm demonstrated high performance (PPV 97.4%, F-score 0.92, accuracy 94%), which was internally validated (PPV 92.3%, F-score 0.90, accuracy 91%) using a separate sample of potential HCC cases. Our findings support the need to assess the accuracy and precision of ICD codes before using EHR data to study HCC more broadly. Figures Figure 1 Introduction Hepatocellular carcinoma (HCC) is the most common type of liver cancer among adults and the sixth leading cause of cancer deaths in the United States( 1 ). Epidemiological trends for HCC vary based on race/ethnicity, sex, and age, and outcomes are conditional on patient and tumor characteristics and treatment options. Population-based estimates of the epidemiology and outcomes of HCC have been largely derived from Surveillance Epidemiology and End Results (SEER) or Veterans Affair (VA) administrative data( 2 , 3 ). SEER captures large patient samples using validated measures from histology and radiology reports but lacks treatment-related factors, which limits its ability to compare treatments and outcomes( 4 ). While the use of a single International Classification of Diseases (ICD) code for HCC has been previously validated in the VA with a high positive predictive value (PPV) of 86%, findings from VA administrative data have limited generalizability beyond the veteran population( 5 ). Large data from health systems that provide longitudinal care for patients with HCC are needed to obtain more precise estimates of HCC epidemiology and treatment outcomes. A study based in an academic health system examined the accuracy of ICD, Ninth Revision, Clinical Modification (ICD-9-CM) codes from an administrative database to identify patients with HCC and found that a combination of two occurrences of ICD-9-CM codes for HCC and two ICD-9-CM codes for chronic liver disease and/or cirrhosis were needed to achieve an 87% PPV in capturing true HCC cases.( 6 ) Since transitioning to the ICD, Tenth Revision, Clinical Modification (ICD-10-CM) coding system, the precision, accuracy, and validity of ICD codes in identifying HCC cases have not been thoroughly assessed using health system administrative data outside of SEER and VA sources. We sought to assess the performance of ICD-9-CM and ICD-10-CM diagnostic codes to identify patients with HCC with established ambulatory care at a large academic health system and determine whether employing an ICD code-based algorithm could deliver higher accuracy and precision in identifying HCC cases. Methods We used ICD-9-CM and ICD-10-CM codes (Table 1 in Supplement) to assemble a study cohort of adult patients with chronic liver disease between 2006 and 2022 using electronic health record (EHR) data at an academic health system (UCLA Health), which included two hospitals and over 200 medical clinics across Southern California (Figure 1). All patients had a minimum of two ambulatory care visits in primary care at least one year apart to be considered established ambulatory care patients in the health system (n=26,439). We defined the follow-up period as the time between the first ICD code entry for HCC to the most recent encounter. We first queried for patients with potential HCC, which we defined as having at least one entry of ICD-9-CM (155.0) or ICD-10-CM (C22.0) code for HCC (n=1,007). To assess the performance of a single ICD code entry for HCC, we assembled a development sample, which was a random pool of 300 patients from the potential HCC sample, for chart review by three physicians (SJS, AB, SB) using a structured abstraction form. Separate chart reviews were conducted by a transplant hepatologist (CRW) for any discrepancies. HCC diagnosis was established using a combination of imaging, histology, and clinical notes in the EHR. Abstraction of clinical data from chart reviews provided information about true HCC cases (true positive) and false HCC cases (false positive), which served as the gold standard comparison. From our chart reviews using the development sample, we identified the most frequent non-HCC malignancies, which were included in our algorithm. Since the algorithm was based on the development sample using manual chart reviews as the gold standard, we also performed a sensitivity analysis using the institution’s cancer registry as a reference. This sensitivity analysis was applied to 285 patients from the development sample diagnosed with HCC between 2006 and 2020. The sensitivity analysis excluded 15 patients because they were diagnosed with HCC outside of the registry date range (e.g. after 2020 when registry data was unavailable at the time of this study). Performance measurements, including sensitivity, specificity, PPV (precision), negative predictive value (NPV), F-score (harmonic mean of PPV and sensitivity), and accuracy (percentage of patients who were correctly classified) accompanied each algorithm iteration (Table 1). We selected the best performing algorithm with the highest PPV, F-score, and accuracy to reduce the number of false positive and negative cases. We internally validated the highest performing algorithm using a different random sample of 300 patients from the pool of potential patients with HCC (Figure 1). Table 1. Performance measurements of ICD code-based algorithm iterations Sensitivity Specificity PPV NPV F-Score Accuracy Algorithm 1: Increasing number of ICD code entries for HCC ³ 9 ICD code entries for HCC 88.1 93.7 91.0 91.6 0.90 91.3 ³ 10 ICD code entries for HCC 88.1 94.8 92.5 91.7 0.90 92.0 ³ 11 ICD code entries for HCC 85.7 94.8 92.3 90.2 0.89 91.0 Algorithm 2: Exclusion of each non-HCC malignancy with ³ 1 ICD code entry for HCC Secondary malignancy 95.2 17.8 45.6 83.8 0.62 50.3 Cholangiocarcinoma 89.7 17.2 44.0 69.8 0.59 47.7 Pancreatic cancer 100 0 42.0 0 0.59 42.0 Colorectal cancer 100 0 42.0 0 0.59 42.0 Neuroendocrine tumor 100 0 42.0 0 0.59 42.0 Algorithm 3: ³ 10 ICD code entries for HCC and exclusion of each non-HCC malignancy Secondary malignancy 83.3 95.4 92.9 88.8 0.88 90.3 Cholangiocarcinoma 77.8 98.9 98.0 86.0 0.87 90.0 Pancreatic cancer 88.1 94.8 92.5 91.7 0.90 92.0 Colorectal cancer 88.1 94.8 92.5 91.7 0.90 92.0 Neuroendocrine tumor 88.1 94.8 92.5 91.7 0.90 92.0 Algorithm 4: Increasing number of ICD code entries for HCC and sum of ICD code entries for HCC exceed sum of ICD code entries for non-HCC malignancies ³ 9 ICD code entries for HCC 88.1 97.1 95.7 91.8 0.92 93.3 ³ 10 ICD code entries for HCC 88.1 98.3 97.4 91.9 0.92 94.0 ³ 11 ICD code entries for HCC 85.7 98.3 97.3 90.5 0.91 93.0 PPV; positive predictive value, NPV; negative predictive value; HCC, hepatocellular carcinoma Performance measurements of each algorithm were obtained using the Development Sample, which was a random pool of 300 potential HCC patients with at least 1 ICD code entry for HCC. Algorithm 1 included an increasing number of ICD code entries for HCC using ICD codes 155.0 and C22.0. Performance measurements in Algorithm 2 were for exclusions of each respective non-HCC malignancy: secondary malignancy (ICD codes 197.7, C78.7), cholangiocarcinoma (ICD codes 155.1, C22.1), pancreatic cancer (ICD codes 157.9, C25.9), colorectal cancer (ICD codes 153.9, C18.9), or neuroendocrine tumor (ICD codes 209, 209.0, 209.00-209.03, 209.1, 209.10-209.17, 209.72, C7A.1, C7A.010-C7A.012, C7A.019-C7A.026, C7A.029, C7B.02). Algorithm 3 included the best performing iteration in Algorithm 1, which included at least 10 ICD code entries for HCC, and excluded each respective non-HCC malignancy, which were evaluated in Algorithm 2 (secondary malignancy, cholangiocarcinoma, pancreatic cancer, colorectal cancer, neuroendocrine tumor). Algorithm 4 included iterations from Algorithm 1, which included an increasing number of ICD code entries for HCC and required the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies (secondary malignancy, cholangiocarcinoma, pancreatic cancer, colorectal cancer, neuroendocrine tumor). Highlighted are the highest performing iterations in Algorithm 1 and Algorithm 4. Results The cohort of 26,439 established patients with ICD codes indicative of chronic liver disease included 1,007 patients with potential HCC based on a single ICD-9-CM or ICD-10-CM code entry. Chart reviews of the random sample of 300 potential HCC cases revealed that 58% of the potential cases of HCC were false positives (not true HCC cases). Of the 174 false HCC cases, 64 patients (36.8%) were found to have non-HCC malignancies, including 20 with cholangiocarcinoma, 16 with metastatic colorectal cancer, 9 with metastatic neuroendocrine tumors, 4 with metastatic pancreatic adenocarcinoma, and 15 with other metastatic malignancies. The second largest group of false positives (32.1%) included patients undergoing surveillance for HCC in the setting of chronic liver disease (n = 32) and other malignancies (n = 24) including history of renal cell carcinoma, breast cancer, and head and neck cancers. Other false HCC cases included patients who were found to have non-HCC liver lesions (n = 32; 18.4%) such as hemangiomas and hepatocellular adenomas or had diagnostics performed for other indications including abnormal liver enzymes (n = 22; 12.6%). In the development sample (n = 300), the median age (interquartile range [IQR]) was similar between the true and false HCC cases, which were 64 (57–71) and 65 (53–73) years, respectively (p = 0.69). Median follow-up was also not significantly different between the HCC cases and non-cases: 6.4 (2.6–10.3) versus 5.7 (2.2–9.4) years, respectively (p = 0.21). Notably, there were significantly more females among false HCC cases (50%) compared to true HCC cases (31%), p < 0.01. We first examined the frequency of entries for HCC using ICD codes based on prior work( 6 ), which revealed that the best iteration of Algorithm 1 included at least 10 ICD code entries for HCC (Table 1 ). We then compared the performance metrics after excluding specific non-HCC malignancies that were frequently miscoded using ICD codes for HCC (Algorithm 2). We combined the best performing iteration from Algorithm 1 with exclusion of each non-HCC malignancy (Algorithm 3). To account for all common non-HCC malignancies, we developed Algorithm 4, which included the best performing iteration from Algorithm 1 and required the sum of ICD code entries for HCC to exceed the sum of ICD code entries for the most common non-HCC malignancies, including cholangiocarcinoma, colorectal cancer, pancreatic cancer, neuroendocrine tumor, and other secondary malignancies. The best performing algorithm was an iteration of Algorithm 4 which had a of PPV of 97.4%, F-score of 0.92, and accuracy of 94% using the development sample. When we tested the algorithm using the cancer registry as the gold standard, the algorithm had similar performance metrics (PPV 94.3%, F-score 0.93, accuracy 94.4%). Our internal validation results revealed a consistently high performance with a PPV of 92.3%, F-score of 0.90, and accuracy of 91%. Discussion This study assessed the performance of multiple ICD-9-CM and ICD-10-CM for HCC to precisely and accurately identify patients with HCC and longitudinal ambulatory care in a large academic health system. In contrast to findings from the VA Corporate Data Warehouse( 3 ), a single ICD code for HCC performed poorly with a 58% false positive rate (42% accuracy). An algorithm requiring at least 10 ICD code entries for HCC in combination with the sum of HCC ICD code entries exceeding the sum of non-HCC malignancy ICD code entries identified true HCC cases with at least 90% accuracy and precision in our development and validation samples. The discrepancy between the performance metrics of a single ICD code entry in our health system compared to the VA may be related to different coding practices and different patient populations including a higher prevalence of patients undergoing specialized cancer treatments in our health system. Compared to the VA, our health system includes patients with more female-specific malignancies, such as breast cancer, which require monitoring for liver metastasis. Specialized cancer treatments offered in our health system also require frequent surveillance imaging. As a referral center for specialty care, including transplant hepatology and hepatobiliary surgery, our health system likely encounters more consultations for evaluation of suspicious liver lesions, which may be miscoded using ICD-9-CM and ICD-10-CM codes for HCC. The PPV of the algorithm increased with an increasing number of ICD code entries for HCC as similarly reported in a previous study conducted in an academic health system( 6 ). Limitations and Strengths Several limitations should be considered when interpreting our findings. First, UCLA Health includes a tertiary care center, which receives referrals for specialized care in oncology, hepatobiliary surgery, and liver transplantation. Therefore, the prevalence of HCC is expectedly high in the study’s population, which could have inflated the PPV of our algorithm. Nonetheless, a higher prevalence of non-HCC malignancies would also be expected, which was countered by including the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies in the algorithm. We also included additional performance metrics, such as the specificity and F-score, which are not as affected by disease prevalence. Second, while chart reviews were considered the gold standard to develop the best performing algorithm, human error during manual abstractions of EHR data could bias results. Third, the study cohort was defined using ICD codes for chronic liver diseases, which were similarly subject to misclassification and could have led to downstream effects on the development and performance of our algorithm. In recognition of these limitations, we tested the performance of the algorithm using cancer registry data, which identified HCC cases independently from our study team using separate criteria, as a sensitivity test which showed similar results. Conclusion In conclusion, this study demonstrated that a combination of ICD-9-CM and ICD-10-CM codes for HCC can be used via an algorithm with over 90% accuracy and precision, whereas the use of a single ICD code entry to identify HCC cases requires caution in similar large health systems. With emerging new treatments for HCC(7) and use of artificial intelligence to capture HCC characteristics and outcomes using EHR data(8), large samples of patients with accurate, precise, and valid HCC diagnoses are needed to derive population-based estimates of treatment receipt and response, compare effectiveness of different treatments, and assess for differences in treatment uptake and outcomes by patient characteristics. Future studies that leverage EHR-based data to conduct studies about HCC epidemiology and treatment outcomes are encouraged to validate the accuracy and precision of ICD codes for HCC diagnoses before generating population-based estimates. Abbreviations HCC hepatocellular carcinoma SEER Surveillance Epidemiology and End Results VA Veterans Affair ICD International Classification of Diseases PPV positive predictive value ICD-9-CM ICD, Ninth Revision, Clinical Modification ICD-10-CM ICD, Tenth Revision, Clinical Modification NPV negative predictive value EHR electronic health records Declarations The current study was approved by the Institutional Review Board at the University of California, Los Angeles. Written informed consent was not required for the use of de-identified electronic health data. Funding Jonsson Comprehensive Cancer Center, David Geffen School of Medicine, and Clinical Translational Science Institute at the University of California, Los Angeles (UL1TR001881) Author Contribution C.R.W., Y.N.F., A.A., L.T., F.P.M., D.B., B.G., and R.B. contributed to the study conception and design. Y.N.F., C.M.C., F.P.M., D.B., B.G., and R.B. acquired the data. A.A. provided primary statistical support. All authors contributed to the methods and statistical analyses. C.R.W. wrote the main manuscript text. All authors reviewed and edited the manuscript. Acknowledgement We would like to acknowledge Drs. Saroja Bangaru, Aileen Bui, and Sarah J. Stettner and the Cancer Registry Department of Performance Strategy and Analytics at the University of California, Los Angeles for their contributions. Data Availability Study data cannot be openly shared to protect the private health information of patients at UCLA Health. References Causes LC, Factors R, and Prevention - NCI [Internet]. 2022 [cited 2024 Aug 15]. https://www.cancer.gov/types/liver/what-is-liver-cancer/causes-risk-factors SEER [Internet]. [cited 2024 Apr 2]. Cancer of the Liver and Intrahepatic Bile Duct - Cancer Stat Facts. https://seer.cancer.gov/statfacts/html/livibd.html Ju MR, Karalis JD, Chansard M, Augustine MM, Mortensen E, Wang SC, et al. Variation of Hepatocellular Carcinoma Treatment Patterns and Survival Across Geographic Regions in a Veteran Population. Ann Surg Oncol. 2022;29(13):8413–20. Noone AM, Lund JL, Mariotto A, Cronin K, McNeel T, Deapen D, et al. Comparison of SEER Treatment Data With Medicare Claims. Med Care. 2016;54(9):e55–64. OVERVIEW OF VHA PATIENT VETERAN, AND, NON-VETERAN POPULATIONS, AND CHARACTERISTICS. In: National Healthcare Quality and Disparities Report: Chartbook on Healthcare for Veterans [Internet]. Agency for Healthcare Research and Quality (US); 2020 [cited 2024 Aug 22]. https://www.ncbi.nlm.nih.gov/books/NBK578553/ Goldberg DS, Lewis JD, Halpern SD, Weiner MG, Lo Re V. Validation of a coding algorithm to identify patients with hepatocellular carcinoma in an administrative database. Pharmacoepidemiol Drug Saf. 2013;22(1):103–7. Llovet JM, Kelley RK, Villanueva A, Singal AG, Pikarsky E, Roayaie S, et al. Hepatocellular carcinoma. Nat Rev Dis Primers. 2021;7(1):1–28. Ge J, Li M, Delk MB, Lai JC. A Comparison of a Large Language Model vs Manual Chart Review for the Extraction of Data Elements From the Electronic Health Record. Gastroenterology. 2024;166(4):707–e7093. Additional Declarations No competing interests reported. Supplementary Files SupplementaryMaterial.docx Cite Share Download PDF Status: Published Journal Publication published 01 Oct, 2025 Read the published version in BMC Research Notes → Version 1 posted Editorial decision: Revision requested 14 Jul, 2025 Reviews received at journal 13 Jul, 2025 Reviews received at journal 06 Jul, 2025 Reviewers agreed at journal 05 Jul, 2025 Reviewers agreed at journal 05 Jul, 2025 Reviewers agreed at journal 05 Jul, 2025 Reviewers agreed at journal 03 Jul, 2025 Reviewers invited by journal 14 Oct, 2024 Editor invited by journal 06 Sep, 2024 Editor assigned by journal 02 Sep, 2024 Submission checks completed at journal 02 Sep, 2024 First submitted to journal 28 Aug, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4993106","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":357977888,"identity":"8e36e1e0-c429-4286-b759-84e03e1dad33","order_by":0,"name":"Carrie R. Wong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA60lEQVRIiWNgGAWjYPACGyBOgDD5QAQPYS1pEC0HgBQbkVoOk6CFf/YZw8cFv85H87PnHnz8geGwPBv7AcYHb9twa5E4l2NsPLPvdu7MnnfJBgcYDhu28SQwG87Fo8WAh3ebNG/P7dwNN3LMJIBaEtgYEtikeQlrOZe7/0aO+Q+wFv4H7L8JauH5cSB3g0SOGQNYi0QCGzM+LRJn+D8b8zYk584488ZY4oxBumGbxMNmyTnncGvh72FLfMzzxy63vz3H8ENFhbU8P3/ywQ9vynBrAQNGuDMMwNwGAupB4A8RakbBKBgFo2DkAgCOMU42SbeBZQAAAABJRU5ErkJggg==","orcid":"","institution":"Vatche and Tamar Manoukian Division of Digestive Diseases, Department of Medicine, University of California, Los Angeles","correspondingAuthor":true,"prefix":"","firstName":"Carrie","middleName":"R.","lastName":"Wong","suffix":""},{"id":357977889,"identity":"f5b6c02a-cad6-487b-b357-510050760838","order_by":1,"name":"Yvonne N. Flores","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Yvonne","middleName":"N.","lastName":"Flores","suffix":""},{"id":357977890,"identity":"ab524334-9980-4d28-8528-56842e9a5760","order_by":2,"name":"Analissa Avila","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Analissa","middleName":"","lastName":"Avila","suffix":""},{"id":357977891,"identity":"f8e63575-da66-4b12-a68a-3e1ece8eebb8","order_by":3,"name":"Lina Tieu","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Lina","middleName":"","lastName":"Tieu","suffix":""},{"id":357977892,"identity":"21a7e2ff-7fd3-4b6f-b0b1-995723cd7f39","order_by":4,"name":"Catherine M. Crespi","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Catherine","middleName":"M.","lastName":"Crespi","suffix":""},{"id":357977893,"identity":"ab18a689-1cab-4b80-ac12-78d25c8a0b66","order_by":5,"name":"Folasade P. May","email":"","orcid":"","institution":"Vatche and Tamar Manoukian Division of Digestive Diseases, Department of Medicine, University of California, Los Angeles","correspondingAuthor":false,"prefix":"","firstName":"Folasade","middleName":"P.","lastName":"May","suffix":""},{"id":357977894,"identity":"10945682-d853-4121-9ab8-8358fd3fba40","order_by":6,"name":"Douglas Bell","email":"","orcid":"","institution":"Division of General Internal Medicine, Department of Medicine, University of California, Los Angeles","correspondingAuthor":false,"prefix":"","firstName":"Douglas","middleName":"","lastName":"Bell","suffix":""},{"id":357977895,"identity":"ec9d0090-0a6a-44af-b4d5-d11fab54a506","order_by":7,"name":"Beth Glenn","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Beth","middleName":"","lastName":"Glenn","suffix":""},{"id":357977896,"identity":"2fab153a-2dfa-4571-ba1e-56ba268a93df","order_by":8,"name":"Roshan Bastani","email":"","orcid":"","institution":"UCLA Center for Cancer Prevention and Control and UCLA-Kaiser Permanente Center for Health Equity","correspondingAuthor":false,"prefix":"","firstName":"Roshan","middleName":"","lastName":"Bastani","suffix":""}],"badges":[],"createdAt":"2024-08-28 18:51:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4993106/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4993106/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13104-025-07465-z","type":"published","date":"2025-10-01T15:57:56+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":66942661,"identity":"7b926c52-f93c-47ec-b33e-d4e604c52636","added_by":"auto","created_at":"2024-10-18 09:10:42","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":65217,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFlowchart of Study Population\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4993106/v1/3739f784e46ad83496b2ea4b.png"},{"id":92884663,"identity":"6cb90148-1806-4dbb-b38d-a956ad7bc574","added_by":"auto","created_at":"2025-10-06 16:13:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":688097,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4993106/v1/35cfb29f-96f8-4b9a-b519-5e4128c10a29.pdf"},{"id":66942659,"identity":"0d821f7e-5332-4a51-b290-d1cf2069d963","added_by":"auto","created_at":"2024-10-18 09:10:42","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":18028,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryMaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-4993106/v1/0ba2352236995b6667200232.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eImproving the Accuracy and Precision of Disease Identification When Utilizing Ehr Data for Research: the Case for Hepatocellular Carcinoma\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eHepatocellular carcinoma (HCC) is the most common type of liver cancer among adults and the sixth leading cause of cancer deaths in the United States(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). Epidemiological trends for HCC vary based on race/ethnicity, sex, and age, and outcomes are conditional on patient and tumor characteristics and treatment options. Population-based estimates of the epidemiology and outcomes of HCC have been largely derived from Surveillance Epidemiology and End Results (SEER) or Veterans Affair (VA) administrative data(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). SEER captures large patient samples using validated measures from histology and radiology reports but lacks treatment-related factors, which limits its ability to compare treatments and outcomes(\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). While the use of a single International Classification of Diseases (ICD) code for HCC has been previously validated in the VA with a high positive predictive value (PPV) of 86%, findings from VA administrative data have limited generalizability beyond the veteran population(\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eLarge data from health systems that provide longitudinal care for patients with HCC are needed to obtain more precise estimates of HCC epidemiology and treatment outcomes. A study based in an academic health system examined the accuracy of ICD, Ninth Revision, Clinical Modification (ICD-9-CM) codes from an administrative database to identify patients with HCC and found that a combination of two occurrences of ICD-9-CM codes for HCC and two ICD-9-CM codes for chronic liver disease and/or cirrhosis were needed to achieve an 87% PPV in capturing true HCC cases.(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e) Since transitioning to the ICD, Tenth Revision, Clinical Modification (ICD-10-CM) coding system, the precision, accuracy, and validity of ICD codes in identifying HCC cases have not been thoroughly assessed using health system administrative data outside of SEER and VA sources.\u003c/p\u003e \u003cp\u003eWe sought to assess the performance of ICD-9-CM and ICD-10-CM diagnostic codes to identify patients with HCC with established ambulatory care at a large academic health system and determine whether employing an ICD code-based algorithm could deliver higher accuracy and precision in identifying HCC cases.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eWe used ICD-9-CM and ICD-10-CM codes (Table 1 in Supplement) to assemble a study cohort of adult patients with chronic liver disease between 2006 and 2022 using electronic health record (EHR) data at an academic health system (UCLA Health), which included two hospitals and over 200 medical clinics across Southern California (Figure 1). \u0026nbsp;All patients had a minimum of two ambulatory care visits in primary care at least one year apart to be considered established ambulatory care patients in the health system (n=26,439). \u0026nbsp;We defined the follow-up period as the time between the first ICD code entry for HCC to the most recent encounter. \u0026nbsp;We first queried for patients with potential HCC, which we defined as having at least one entry of ICD-9-CM (155.0) or ICD-10-CM (C22.0) code for HCC (n=1,007). \u0026nbsp;To assess the performance of a single ICD code entry for HCC, we assembled a development sample, which was a random pool of 300 patients from the potential HCC sample, for chart review by three physicians (SJS, AB, SB) using a structured abstraction form. \u0026nbsp;Separate chart reviews were conducted by a transplant hepatologist (CRW) for any discrepancies. \u0026nbsp;HCC diagnosis was established using a combination of imaging, histology, and clinical notes in the EHR. \u0026nbsp;Abstraction of clinical data from chart reviews provided information about true HCC cases (true positive) and false HCC cases (false positive), which served as the gold standard comparison. \u0026nbsp;From our chart reviews using the development sample, we identified the most frequent non-HCC malignancies, which were included in our algorithm. \u0026nbsp;Since the algorithm was based on the development sample using manual chart reviews as the gold standard, we also performed a sensitivity analysis using the institution\u0026rsquo;s cancer registry as a reference. \u0026nbsp;This sensitivity analysis was applied to 285 patients from the development sample diagnosed with HCC between 2006 and 2020. \u0026nbsp;The sensitivity analysis excluded 15 patients because they were diagnosed with HCC outside of the registry date range (e.g. after 2020 when registry data was unavailable at the time of this study).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ePerformance measurements, including sensitivity, specificity, PPV (precision), negative predictive value (NPV), F-score (harmonic mean of PPV and sensitivity), and accuracy (percentage of patients who were correctly classified) accompanied each algorithm iteration (Table 1). \u0026nbsp;We selected the best performing algorithm with the highest PPV, F-score, and accuracy to reduce the number of false positive and negative cases. We internally validated the highest performing algorithm using a different random sample of 300 patients from the pool of potential patients with HCC (Figure 1). \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1. Performance measurements of ICD code-based algorithm iterations\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"672\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSensitivity\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSpecificity\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePPV\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eNPV\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eF-Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" valign=\"top\" style=\"width: 88.3929%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAlgorithm 1: Increasing number of ICD code entries for HCC\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;9 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e93.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e91.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e91.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;10 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e94.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e92.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;11 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e85.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e94.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e90.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e91.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"7\" valign=\"top\" style=\"width: 100%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAlgorithm 2: Exclusion of each non-HCC malignancy with\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e\u0026sup3;\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;1 ICD code entry for HCC\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eSecondary malignancy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e95.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e17.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e45.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e83.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.62\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e50.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eCholangiocarcinoma\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e89.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e17.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e44.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e69.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e47.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003ePancreatic cancer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eColorectal cancer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eNeuroendocrine tumor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e42.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"7\" valign=\"top\" style=\"width: 100%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAlgorithm 3:\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e\u0026sup3;\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;10 ICD code entries for HCC and exclusion of each non-HCC malignancy\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eSecondary malignancy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e83.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e95.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e88.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e90.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eCholangiocarcinoma\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e77.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e98.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e98.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e86.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.87\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e90.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003ePancreatic cancer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e94.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e92.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eColorectal cancer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e94.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e92.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003eNeuroendocrine tumor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e94.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e92.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e92.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"7\" valign=\"top\" style=\"width: 100%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAlgorithm 4: Increasing number of ICD code entries for HCC and sum of ICD code entries for HCC exceed sum of ICD code entries for non-HCC malignancies\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;9 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e97.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e95.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e93.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;10 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e88.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e98.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e97.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e91.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e94.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 32.4405%;\"\u003e\n \u003cp\u003e\u0026sup3;\u0026nbsp;11 ICD code entries for HCC\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.0952%;\"\u003e\n \u003cp\u003e85.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 13.6905%;\"\u003e\n \u003cp\u003e98.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 9.52381%;\"\u003e\n \u003cp\u003e97.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 8.92857%;\"\u003e\n \u003cp\u003e90.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 10.7143%;\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 11.6071%;\"\u003e\n \u003cp\u003e93.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003ePPV; positive predictive value, NPV; negative predictive value; HCC, hepatocellular carcinoma\u003c/p\u003e\n\u003cp\u003ePerformance measurements of each algorithm were obtained using the Development Sample, which was a random pool of 300 potential HCC patients with at least 1 ICD code entry for HCC. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlgorithm 1 included an increasing number of ICD code entries for HCC using ICD codes 155.0 and C22.0. \u0026nbsp; Performance measurements in Algorithm 2 were for exclusions of each respective non-HCC malignancy: secondary malignancy (ICD codes 197.7, C78.7), cholangiocarcinoma (ICD codes 155.1, C22.1), pancreatic cancer (ICD codes 157.9, C25.9), colorectal cancer (ICD codes 153.9, C18.9), or neuroendocrine tumor (ICD codes 209, 209.0, 209.00-209.03, 209.1, 209.10-209.17, 209.72, C7A.1, C7A.010-C7A.012, C7A.019-C7A.026, C7A.029, C7B.02). \u0026nbsp;Algorithm 3 included the best performing iteration in Algorithm 1, which included at least 10 ICD code entries for HCC, and excluded each respective non-HCC malignancy, which were evaluated in Algorithm 2 (secondary malignancy, cholangiocarcinoma, pancreatic cancer, colorectal cancer, neuroendocrine tumor). \u0026nbsp; Algorithm 4 included iterations from Algorithm 1, which included an increasing number of ICD code entries for HCC and required the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e(secondary malignancy, cholangiocarcinoma, pancreatic cancer, colorectal cancer, neuroendocrine tumor). \u0026nbsp;Highlighted are the highest performing iterations in Algorithm 1 and Algorithm 4.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThe cohort of 26,439 established patients with ICD codes indicative of chronic liver disease included 1,007 patients with potential HCC based on a single ICD-9-CM or ICD-10-CM code entry. Chart reviews of the random sample of 300 potential HCC cases revealed that 58% of the potential cases of HCC were false positives (not true HCC cases). Of the 174 false HCC cases, 64 patients (36.8%) were found to have non-HCC malignancies, including 20 with cholangiocarcinoma, 16 with metastatic colorectal cancer, 9 with metastatic neuroendocrine tumors, 4 with metastatic pancreatic adenocarcinoma, and 15 with other metastatic malignancies. The second largest group of false positives (32.1%) included patients undergoing surveillance for HCC in the setting of chronic liver disease (n\u0026thinsp;=\u0026thinsp;32) and other malignancies (n\u0026thinsp;=\u0026thinsp;24) including history of renal cell carcinoma, breast cancer, and head and neck cancers. Other false HCC cases included patients who were found to have non-HCC liver lesions (n\u0026thinsp;=\u0026thinsp;32; 18.4%) such as hemangiomas and hepatocellular adenomas or had diagnostics performed for other indications including abnormal liver enzymes (n\u0026thinsp;=\u0026thinsp;22; 12.6%).\u003c/p\u003e \u003cp\u003eIn the development sample (n\u0026thinsp;=\u0026thinsp;300), the median age (interquartile range [IQR]) was similar between the true and false HCC cases, which were 64 (57\u0026ndash;71) and 65 (53\u0026ndash;73) years, respectively (p\u0026thinsp;=\u0026thinsp;0.69). Median follow-up was also not significantly different between the HCC cases and non-cases: 6.4 (2.6\u0026ndash;10.3) versus 5.7 (2.2\u0026ndash;9.4) years, respectively (p\u0026thinsp;=\u0026thinsp;0.21). Notably, there were significantly more females among false HCC cases (50%) compared to true HCC cases (31%), p\u0026thinsp;\u0026lt;\u0026thinsp;0.01.\u003c/p\u003e \u003cp\u003eWe first examined the frequency of entries for HCC using ICD codes based on prior work(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e), which revealed that the best iteration of Algorithm \u003cspan refid=\"FPar1\" class=\"InternalRef\"\u003e1\u003c/span\u003e included at least 10 ICD code entries for HCC (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e1\u003c/span\u003e). We then compared the performance metrics after excluding specific non-HCC malignancies that were frequently miscoded using ICD codes for HCC (Algorithm 2). We combined the best performing iteration from Algorithm \u003cspan refid=\"FPar1\" class=\"InternalRef\"\u003e1\u003c/span\u003e with exclusion of each non-HCC malignancy (Algorithm 3). To account for all common non-HCC malignancies, we developed Algorithm 4, which included the best performing iteration from Algorithm \u003cspan refid=\"FPar1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and required the sum of ICD code entries for HCC to exceed the sum of ICD code entries for the most common non-HCC malignancies, including cholangiocarcinoma, colorectal cancer, pancreatic cancer, neuroendocrine tumor, and other secondary malignancies.\u003c/p\u003e \u003cp\u003eThe best performing algorithm was an iteration of Algorithm 4 which had a of PPV of 97.4%, F-score of 0.92, and accuracy of 94% using the development sample. When we tested the algorithm using the cancer registry as the gold standard, the algorithm had similar performance metrics (PPV 94.3%, F-score 0.93, accuracy 94.4%). Our internal validation results revealed a consistently high performance with a PPV of 92.3%, F-score of 0.90, and accuracy of 91%.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study assessed the performance of multiple ICD-9-CM and ICD-10-CM for HCC to precisely and accurately identify patients with HCC and longitudinal ambulatory care in a large academic health system. In contrast to findings from the VA Corporate Data Warehouse(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e), a single ICD code for HCC performed poorly with a 58% false positive rate (42% accuracy). An algorithm requiring at least 10 ICD code entries for HCC in combination with the sum of HCC ICD code entries exceeding the sum of non-HCC malignancy ICD code entries identified true HCC cases with at least 90% accuracy and precision in our development and validation samples.\u003c/p\u003e \u003cp\u003eThe discrepancy between the performance metrics of a single ICD code entry in our health system compared to the VA may be related to different coding practices and different patient populations including a higher prevalence of patients undergoing specialized cancer treatments in our health system. Compared to the VA, our health system includes patients with more female-specific malignancies, such as breast cancer, which require monitoring for liver metastasis. Specialized cancer treatments offered in our health system also require frequent surveillance imaging. As a referral center for specialty care, including transplant hepatology and hepatobiliary surgery, our health system likely encounters more consultations for evaluation of suspicious liver lesions, which may be miscoded using ICD-9-CM and ICD-10-CM codes for HCC. The PPV of the algorithm increased with an increasing number of ICD code entries for HCC as similarly reported in a previous study conducted in an academic health system(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eLimitations and Strengths\u003c/h2\u003e \u003cp\u003eSeveral limitations should be considered when interpreting our findings. First, UCLA Health includes a tertiary care center, which receives referrals for specialized care in oncology, hepatobiliary surgery, and liver transplantation. Therefore, the prevalence of HCC is expectedly high in the study\u0026rsquo;s population, which could have inflated the PPV of our algorithm. Nonetheless, a higher prevalence of non-HCC malignancies would also be expected, which was countered by including the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies in the algorithm. We also included additional performance metrics, such as the specificity and F-score, which are not as affected by disease prevalence. Second, while chart reviews were considered the gold standard to develop the best performing algorithm, human error during manual abstractions of EHR data could bias results. Third, the study cohort was defined using ICD codes for chronic liver diseases, which were similarly subject to misclassification and could have led to downstream effects on the development and performance of our algorithm. In recognition of these limitations, we tested the performance of the algorithm using cancer registry data, which identified HCC cases independently from our study team using separate criteria, as a sensitivity test which showed similar results.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn conclusion, this study demonstrated that a combination of ICD-9-CM and ICD-10-CM codes for HCC can be used via an algorithm with over 90% accuracy and precision, whereas the use of a single ICD code entry to identify HCC cases requires caution in similar large health systems. \u0026nbsp; With emerging new treatments for HCC(7) and use of artificial intelligence to capture HCC characteristics and outcomes using EHR data(8), large samples of patients with accurate, precise, and valid HCC diagnoses are needed to derive population-based estimates of treatment receipt and response, compare effectiveness of different treatments, and assess for differences in treatment uptake and outcomes by patient characteristics. \u0026nbsp; Future studies that leverage EHR-based data to conduct studies about HCC epidemiology and treatment outcomes are encouraged to validate the accuracy and precision of ICD codes for HCC diagnoses before generating population-based estimates.\u003c/p\u003e\n"},{"header":"Abbreviations","content":"\u003cp\u003eHCC hepatocellular carcinoma\u003c/p\u003e\n\u003cp\u003eSEER Surveillance Epidemiology and End Results\u003c/p\u003e\n\u003cp\u003eVA Veterans Affair\u003c/p\u003e\n\u003cp\u003eICD International Classification of Diseases\u003c/p\u003e\n\u003cp\u003ePPV positive predictive value\u003c/p\u003e\n\u003cp\u003eICD-9-CM ICD, Ninth Revision, Clinical Modification\u003c/p\u003e\n\u003cp\u003eICD-10-CM ICD, Tenth Revision, Clinical Modification \u003c/p\u003e\n\u003cp\u003eNPV negative predictive value\u003c/p\u003e\n\u003cp\u003eEHR electronic health records\u003c/p\u003e"},{"header":"Declarations","content":" \u003cp\u003eThe current study was approved by the Institutional Review Board at the University of California, Los Angeles. Written informed consent was not required for the use of de-identified electronic health data.\u003c/p\u003e\u003ch2\u003eFunding\u003c/h2\u003e \u003cp\u003eJonsson Comprehensive Cancer Center, David Geffen School of Medicine, and Clinical Translational Science Institute at the University of California, Los Angeles (UL1TR001881)\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eC.R.W., Y.N.F., A.A., L.T., F.P.M., D.B., B.G., and R.B. contributed to the study conception and design. Y.N.F., C.M.C., F.P.M., D.B., B.G., and R.B. acquired the data. A.A. provided primary statistical support. All authors contributed to the methods and statistical analyses. C.R.W. wrote the main manuscript text. All authors reviewed and edited the manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eWe would like to acknowledge Drs. Saroja Bangaru, Aileen Bui, and Sarah J. Stettner and the Cancer Registry Department of Performance Strategy and Analytics at the University of California, Los Angeles for their contributions.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eStudy data cannot be openly shared to protect the private health information of patients at UCLA Health.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eCauses LC, Factors R, and Prevention - NCI [Internet]. 2022 [cited 2024 Aug 15]. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cancer.gov/types/liver/what-is-liver-cancer/causes-risk-factors\u003c/span\u003e\u003cspan address=\"https://www.cancer.gov/types/liver/what-is-liver-cancer/causes-risk-factors\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSEER [Internet]. [cited 2024 Apr 2]. Cancer of the Liver and Intrahepatic Bile Duct - Cancer Stat Facts. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://seer.cancer.gov/statfacts/html/livibd.html\u003c/span\u003e\u003cspan address=\"https://seer.cancer.gov/statfacts/html/livibd.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJu MR, Karalis JD, Chansard M, Augustine MM, Mortensen E, Wang SC, et al. Variation of Hepatocellular Carcinoma Treatment Patterns and Survival Across Geographic Regions in a Veteran Population. Ann Surg Oncol. 2022;29(13):8413\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNoone AM, Lund JL, Mariotto A, Cronin K, McNeel T, Deapen D, et al. Comparison of SEER Treatment Data With Medicare Claims. Med Care. 2016;54(9):e55\u0026ndash;64.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOVERVIEW OF VHA PATIENT VETERAN, AND, NON-VETERAN POPULATIONS, AND CHARACTERISTICS. In: National Healthcare Quality and Disparities Report: Chartbook on Healthcare for Veterans [Internet]. Agency for Healthcare Research and Quality (US); 2020 [cited 2024 Aug 22]. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.ncbi.nlm.nih.gov/books/NBK578553/\u003c/span\u003e\u003cspan address=\"https://www.ncbi.nlm.nih.gov/books/NBK578553/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoldberg DS, Lewis JD, Halpern SD, Weiner MG, Lo Re V. Validation of a coding algorithm to identify patients with hepatocellular carcinoma in an administrative database. Pharmacoepidemiol Drug Saf. 2013;22(1):103\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLlovet JM, Kelley RK, Villanueva A, Singal AG, Pikarsky E, Roayaie S, et al. Hepatocellular carcinoma. Nat Rev Dis Primers. 2021;7(1):1\u0026ndash;28.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGe J, Li M, Delk MB, Lai JC. A Comparison of a Large Language Model vs Manual Chart Review for the Extraction of Data Elements From the Electronic Health Record. Gastroenterology. 2024;166(4):707\u0026ndash;e7093.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-research-notes","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"resn","sideBox":"Learn more about [BMC Research Notes](http://bmcresnotes.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/resn/default.aspx","title":"BMC Research Notes","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4993106/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4993106/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eObjective\u003c/h2\u003e \u003cp\u003eWe assessed the performance of ICD codes to identify patients with hepatocellular carcinoma (HCC) in a large academic health system and determined whether employing an algorithm using a combination of ICD codes could deliver higher accuracy and precision than single ICD codes in identifying HCC cases using electronic health record (EHR) data.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eThe use of a single ICD code entry for HCC (ICD-9-CM 155.0 or ICD-10-CM C22.0) in our cohort of 1,007 established ambulatory care patients with potential HCC yielded 58% false positives (not true HCC cases) based on chart reviews. We developed an ICD code-based algorithm that prioritized positive predictive value (PPV), F-score, and accuracy to minimize false positives and negatives. The highest performing algorithm required at least 10 ICD code entries for HCC and the sum of ICD code entries for HCC to exceed the sum of ICD code entries for non-HCC malignancies. The algorithm demonstrated high performance (PPV 97.4%, F-score 0.92, accuracy 94%), which was internally validated (PPV 92.3%, F-score 0.90, accuracy 91%) using a separate sample of potential HCC cases. Our findings support the need to assess the accuracy and precision of ICD codes before using EHR data to study HCC more broadly.\u003c/p\u003e","manuscriptTitle":"Improving the Accuracy and Precision of Disease Identification When Utilizing Ehr Data for Research: the Case for Hepatocellular Carcinoma","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-18 09:10:37","doi":"10.21203/rs.3.rs-4993106/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-07-14T07:38:36+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-13T14:38:06+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-06T15:50:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"129155859387820881657621158178658738303","date":"2025-07-05T16:48:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"190660676137110835397603934185486574609","date":"2025-07-05T13:35:13+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"212629767853276036288942416733216314359","date":"2025-07-05T11:23:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"210870779071579386569676645143707847341","date":"2025-07-03T14:58:36+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-10-14T06:07:56+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-09-06T13:07:10+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-09-02T08:44:05+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-09-02T08:41:12+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Research Notes","date":"2024-08-28T18:49:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-research-notes","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"resn","sideBox":"Learn more about [BMC Research Notes](http://bmcresnotes.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/resn/default.aspx","title":"BMC Research Notes","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"afb8a68c-843f-4d4b-ac6d-e75eb3343369","owner":[],"postedDate":"October 18th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-10-06T16:11:48+00:00","versionOfRecord":{"articleIdentity":"rs-4993106","link":"https://doi.org/10.1186/s13104-025-07465-z","journal":{"identity":"bmc-research-notes","isVorOnly":false,"title":"BMC Research Notes"},"publishedOn":"2025-10-01 15:57:56","publishedOnDateReadable":"October 1st, 2025"},"versionCreatedAt":"2024-10-18 09:10:37","video":"","vorDoi":"10.1186/s13104-025-07465-z","vorDoiUrl":"https://doi.org/10.1186/s13104-025-07465-z","workflowStages":[]},"version":"v1","identity":"rs-4993106","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4993106","identity":"rs-4993106","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}