GERBEHRT: A BERT-based Model Tailored for German Electronic Health Records – Potential in Chronic Kidney Disease Prediction

Abstract Routinely collected electronic health records (EHRs) contain rich longitudinal information that enables the prediction of patient outcomes at scale. We developed GERBEHRT, a transformer model adapted from BEHRT and specifically tailored to German EHRs. GERBEHRT was pretrained on outpatient claims from more than 9 million statutorily insured patients and fine-tuned with nearly 1 million additional patients to predict chronic kidney disease (CKD) - a serious condition whose progression can be delayed by early detection. GERBEHRT incorporates EHR features not previously explored in BERT-based approaches and introduces an efficient method to represent multiple attributes per medical concept, such as diagnoses and medications. In a test cohort of 3.7 million patients with 1.5% CKD positives, GERBEHRT achieved an area under the receiver operating characteristic curve (AUROC) of 87.9 and an average precision (AVPR) of 11.4 for a three-year prediction of incident moderate-to-severe CKD, outperforming riskfactor-based models (AUROC/AVPR: 83.6/6.4) and more traditional algorithms using the full EHR (AUROC/AVPR: 86.9/10.1). Although CKD risk prediction remains challenging, GERBEHRT’s superior performance underscores the importance of comprehensive EHR utilization and highlights the potential of tailored deep learning models for personalized CKD risk prediction and targeted patient screening. Competing Interest Statement This work and the Central Research Institute of Ambulatory Health Care in Germany (Zi) are funded and contracted by the Associations of Statutory Health Insurance Physicians in the German Federal States. It is the purpose of Zi to support and further develop the health care assurance mandate under German law. F. A. von Samson-Himmelstjerna reports receiving lecturing fees and travel support from Astra Zeneca GmbH, Chiesi GmbH and Lilly GmbH and is supported by the Medical Faculty of the Christian-Albrechts-University Kiel. Funding Statement This work and the Central Research Institute of Ambulatory Health Care in Germany (Zi) are funded and contracted by the Associations of Statutory Health Insurance Physicians in the German Federal States. F. A. von Samson-Himmelstjerna is funded by the Medical Faculty of the Christian-Albrechts-University Kiel. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethical approval for this study is not required. The use of claims data for this analysis is governed by the German Code of Social Law (SGB X 80 in conjunction with SGB V 68c). The study aims to improve health care quality by predicting health care-relevant outcomes. Because the analysis is based on de‑identified, routinely collected data from statutory health insurance, obtaining approval and consent from individual patients is not feasible and, according to the German Code of Social Law, not required for this type of retrospective study. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Footnotes This work and the Central Research Institute of Ambulatory Health Care in Germany (Zi) are funded and contracted by the Associations of Statutory Health Insurance Physicians in the German Federal States. F. A. von Samson-Himmelstjerna is funded by the Medical Faculty of the Christian-Albrechts-University Kiel (e-mail: esteiger{at}zi.de). (e-mail: Friedrich.vonSamson-Himmelstjerna{at}uksh.de). (e-mail: lkroll{at}zi.de). Data Availability This study is based on German Statutory Health Insurance (SHI) claims data for ambulatory care, specifically drug prescription data from pharmacies and other institutions according to Section 300 (2) of the German Social Code Book V, as well as diagnosis data of SHI-accredited physicians or psychotherapists according to Section 295 (2) of the German Social Code Book V. These data are subject to strict legal and data protection restrictions and therefore cannot be made publicly available. Access to SHI claims data is regulated by German law and requires approval from the relevant authorities. The authors are available to discuss such possibilities within the limits of research regulations.