Assessing GPT and DeepL for Terminology Translation in the Medical Domain: A Comparative Study on the Human Phenotype Ontology

Assessing GPT and DeepL for Terminology Translation in the Medical Domain: A Comparative Study on the Human Phenotype Ontology | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Assessing GPT and DeepL for Terminology Translation in the Medical Domain: A Comparative Study on the Human Phenotype Ontology Richard NOLL, Alexandra BERGER, Dominik KIEU, Tobias MUELLER, and 9 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4836251/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Jul, 2025 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted 9 You are reading this latest preprint version Abstract Background This paper presents a comparative study of two state-of-the-art language models, OpenAI's GPT and DeepL, in the context of terminology translation within the medical domain. Methods This study was conducted on the Human Phenotype Ontology (HPO), which is used in medical research and diagnosis. Medical experts assess the performance of both models on a set of 120 translated HPO terms, employing a 4-point Likert scale (strongly agree = 1, agree = 2, disagree = 3, strongly disagree = 4). An independent reference translation from the HeTOP database was used to validate the quality of the translation. Results The average Likert rating for the 120 selected HPO terms was 1.29 for GPT-3.5 and 1.37 for DeepL. The comparison with HeTOP revealed a high degree of similarity between the machine translations and the reference translations. Conclusions The results indicate that both GPT and DeepL are effective at translating HPO terms from English to German. Statistical analysis revealed no significant differences in the mean ratings between the two models, confirming their comparable performance in terms of translation quality. The study not only illustrates the potential of machine translation but also shows incomplete coverage of translated medical terminology. This underscores the relevance of this study for cross-lingual medical research. However, the evaluation methods need to be further refined, and specific translation issues need to be addressed. Artificial Intelligence Controlled Vocabulary Translations GPT Figures Figure 1 1. BACKGROUND Terminology translation is a crucial task in the field of medical informatics, as it allows for the sharing and integration of knowledge across different languages and medical domains. Medical ontologies are formal representations of medical knowledge that enable the semantics of medical concepts and relationships to be expressed in a structured and machine-readable format. They play a vital role in many medical applications, such as clinical decision support and biomedical research [ 1 ]. However, the translation of ontologies and their medical terminology is a challenging task, as it requires expertise in both the source and target languages and a deep understanding of domain-specific concepts and relationships. Recently, interest in the use of natural language processing (NLP) techniques, such as machine translation, to automate the ontology translation process in the medical domain has increased [ 2 ]. One of the most promising approaches is to use commercial translators that are trained on large-scale text corpora to generate high-quality translations [ 3 ]. In a previous study, DeepL proved to be the most accurate translator out of 12 different commercial translators (including Google Translate, Bing, etc.) [ 4 ]. With OpenAI's generative pretrained transformer (GPT) making waves in the technology industry, many experts are excited about its potential to revolutionize NLP [ 5 ]. It is therefore a logical step to consider its potential for medical terminology translation. In this paper, we present a comparative study of terminology translation via GPT-3.5 and the commercial translation software DeepL. Specifically, we focus on the translation of standardized medical terminology contained in the human phenotype ontology (HPO) into German [ 6 ]. As of January 2024, 11 languages have been integrated into the HPO web interface. These include English, Chinese, Czech, Dutch, Dusun, French, Japanese, Nyangumarta, Spanish, Tiwi, and Turkish [ 7 ]. Although there are already initial versions of German translations of the HPO, these translations are incomplete [ 8 ]. Translation studies often overlook the range of synonyms for medical terms, which affects their practical use in clinical settings and automated analysis tasks. Many approaches focus on primary terms, resulting in technically correct but incomplete translations. The key value of our study is the comprehensive inclusion of synonyms, which improves accuracy and applicability in medical contexts. GPT-3.5 is a large language model that uses deep learning techniques to generate natural language text. It is trained on massive amounts of text data via an unsupervised learning approach, which allows it to learn patterns and relationships in language without the need for explicit annotations. GPT-3.5 has been shown to be highly effective in a wide range of NLP tasks, including machine translation, text generation, and question answering [ 9 ]. DeepL is a commercial machine translation software developed by the German company DeepL GmbH. It is based on neural machine translation techniques, which use deep learning algorithms to learn the statistical patterns of language from large amounts of parallel corpora. The software supports a wide range of languages and domains, including medical terminology, and is optimized for various translation tasks [ 3 ]. Our study aims to answer the following research questions: How are the terminology translations produced by GPT-3.5 and DeepL evaluated by medical experts in terms of quality, and to what extent do these translations correspond to a reference translation? When medical terminology is translated, how good are the respective translators in terms of error proneness? 2. METHODS 2.1. Study design We selected 100 random and 20 common terms from the HPO for translation. The 20 common terms were identified by medical professionals from 178 letters from doctors at the Frankfurt Reference Centre for Rare Diseases at the University Hospital Frankfurt. The common terms were selected on the basis of their frequency of use and their significance to the clinical profiles. Importantly, the rare disease community makes extensive use of HPO to perform differential diagnoses. The translations were performed from English to German via GPT-3.5 and DeepL. We collected the original terms and the translated terms in a spreadsheet for further analysis. The synonyms of a term given in the HPO have been included in the translation. The exemplary structure of an HPO term can be seen in Table 1. The English version of the HPO is freely available and can be downloaded from the HPO website as an ‘open biomedical ontologies (OBO)’ file [ 10 ]. In GPT-3.5, we created the following prompt for the translation of terms, as it is necessary to specify the intention: the following terms are used in the Human Phenotype Ontology. Translate them into German. Ensure that the translation has a scientific and medical context. The terms and synonyms to be translated were subsequently imported into GPT-3.5. GPT-3.5 was used via a web application [ 11 ]. The translations made by DeepL in the pre-study are used again in this study as a reference translation for comparison with GPT-3.5 [ 4 ]. In the previous study, six medical experts rated the terms. Three medical experts from this study were also involved in the pre-study. The translations generated by DeepL are evaluated again in this study to determine whether the results are consistent with those of the previous study. DeepL was used via the application programming interface (API) [ 12 ]. We invited ten medical experts (with medical degrees and several years of clinical or medical research experience) who were fluent in both English and German to evaluate the translations. The experts were blinded to the source of the translations and were not told which translation software was used for each translation, both to reduce the risk of bias. Each translation was rated on a 4-point Likert scale: strongly agree = 1, agree = 2, disagree = 3, strongly disagree = 4. We asked the experts to consider the approval for the respective translation. In addition, medical experts can make comments on individual term or synonym translations. To answer the question of whether the commercial translators examined are suitable for translating medical terminology, the trend of the experts' ratings was explored, and an error analysis was conducted. The study was conducted between March 2023 and April 2024. 2.2. Data analysis We compared the performance of GPT-3.5 and DeepL on the basis of the evaluations provided by medical experts. We calculated the mean and standard deviation (SD) of the Likert ratings (LRs) for each translation software and analyzed the results via statistical tests. The synonyms were always rated together, and the results were then combined with the LR of the main term, resulting in an average LR for each term. Whether the mean values of the average LR differed significantly between the two systems was measured via the Mann‒Whitney U test. This is a nonparametric statistical test used to compare two independent groups of ordinal scaled variables [ 13 ]. On the basis of the 20 common HPO terms, further research was conducted regarding the quality of the translations. We examined interrater reliability via the intraclass correlation coefficient (ICC) and Fleiss's kappa. The ICC is used to assess the agreement or reliability of ratings from different raters, with values ranging from 0 (no agreement) to 1 (perfect agreement). The ICC is not a hypothesis test but a measure of the consistency of ratings [ 14 ]. Fleiss' kappa also measures the reliability of multiple raters and considers both the observed agreement and the agreement that would be expected by chance, providing a measure of interrater reliability that accounts for the possibility of random agreement. It can take values between − 1 and 1, with 1 indicating perfect agreement, 0 indicating agreement no better than chance, and negative values indicating disagreement beyond chance [ 15 ]. To evaluate the quality of the translations, we calculated the Jaro–Winkler similarity between the two translation systems themselves and the similarity to a reference translation of the Health Terminology/Ontology Portal (HeTOP). HeTOP is a comprehensive medical terminology database with translations for diverse medical and clinical applications [ 16 ]. Notably, HeTOP does not contain any official German translations for the HPO. For comparison, English HPO terms were searched for in HeTOP and, if available, a German translation was extracted. For the sake of simplicity, the synonyms were not included in this analysis. The Jaro–Winkler similarity is a string metric that measures the edit distance between two sequences. It uses a prefix scale that rates strings that match from the beginning more favorably. The metric ranges from 0 (indicating no similarity) to 1 (representing identical strings) [ 17 ]. Using the Jaro–Winkler similarity metric with a threshold of 0.6, we evaluated the degree of similarity between the machine-generated translations and the HeTOP reference translations. The purpose of this threshold was to focus on significant similarities and ignore instances with low similarity. Any similarity below this value was set to 0. To assess the impact of term length on translation quality, we categorized the terms into four groups on the basis of their length: terms consisting of 1 word, 2–3 words, 4–7 words, and terms longer than 7 words. LRs were calculated separately for each group to assess differences in translation accuracy. 3. RESULTS As shown in Table 2 , the average LR for the 100 randomly selected HPO terms and synonyms was 1.36 (SD = 0.65) for GPT-3.5 and 1.28 (SD = 0.56) for DeepL. For the 20 common terms, the average LR was 1.22 (SD = 0.44) for GPT-3.5 and 1.46 (SD = 0.64) for DeepL. In the pre-study, the LR at DeepL for the same 100 random terms was 1.23, and for the 20 common terms, it was 1.28 [ 4 ]. There was no distinct difference between the ratings of this study and those of the pre-study. It can therefore be assumed that the ratings for the translations remain consistent across different experts. Table 2 Average Likert ratings (LRs) and standard deviations (SDs) for the translated HPO terms and synonyms. HPO terms Pre-Study: DeepL DeepL GPT-3.5 100 HPO terms 20 HPO terms LR = 1.23; SD = 0.50 LR = 1.28; SD = 0.54 LR = 1.28; SD = 0.56 LR = 1.46; SD = 0.64 LR = 1.36; SD = 0.65 LR = 1.22; SD = 0.44 The Mann‒Whitney U test was conducted to assess potential differences in the ratings between DeepL and GPT-3.5, encompassing the 100 randomly selected terms (p = 0.27) and the 20 commonly used terms (p = 0.06). The null hypothesis for this test states that both groups have the same mean. As the obtained p values are not less than the predefined significance level of 0.05, we are unable to reject the null hypothesis. Consequently, there is insufficient evidence to assert that a significant difference exists in the true mean between the two groups. However, it is worth noting that there is a uniformly positive trend in the ratings for both translation tools. To assess the quality of the translations, we conducted further research focusing on the 20 common terms. Analyzing the ratings assigned to these translations, we observed a consistent pattern of data homogeneity, as depicted in Fig. 1 . This homogeneity was manifested by limited variability among the ratings, leading to both the ICC and Fleiss' kappa yielding low values for all these assessments, as shown in Table 3 . Table 3 Average scores for the Fleiss' kappa coefficient and the intraclass correlation coefficient (ICC) for the ratings of the translations of the 20 common HPO terms by GPT-3.5 and DeepL. The scores indicate the interrater reliability of the expert ratings. GPT-3.5 DeepL ICC 0.17 0.36 Fleiss 0.20 0.17 Among the 20 common terms, 15 (75%) had corresponding reference translations in the HeTOP database. After selecting the threshold, the calculated similarities for ratio comparisons yield values of 0.70 for GPT-3.5 versus HeTOP, 0.76 for DeepL versus HeTOP, and 0.76 for GPT versus DeepL, as shown in Table 4 . A noteworthy observation made by several experts pertained to the technicality of translations produced by DeepL in contrast to those generated by GPT-3.5. Another finding was that GPT-3.5 sometimes produced fewer synonyms in the German translation compared to the source text. Upon closer examination, this reduction seems to occur when several synonyms lead to the same German term. Furthermore, when synonyms are available in both singular and plural forms in English, GPT-3.5 often omits the plural in the translation. Among the 120 HPO terms, 20 consisted of a single word (17%), 55 terms consisted of 2–3 words (46%), 36 terms consisted of 4–7 words (30%), and 9 terms contained more than 7 words (8%). Notably, among the 20 common HPO terms, 13 out of the 20 terms consisted of a single word, representing 65% of these terms. LR performance varies only slightly among categories: single words score 1.40 (GPT) and 1.14 (DeepL), 2–3 word terms score 1.31 (GPT) and 1.30 (DeepL), 4–7 word terms score 1.37 (GPT) and 1.24 (DeepL), and terms with more than 7 words score 1.49 (GPT) and 1.29 (DeepL). Table 4 Jaro–Winkler similarities for the 20 common HPO terms. The similarities between the translations of GPT-3.5 and DeepL, as well as the respective comparisons to reference translations from HeTOP, were determined. HPO terms GPT-3.5 DeepL HeTOP GPT/HeTOP DeepL/HeTOP GPT/DeepL Headache Kopfschmerz Kopfschmerzen Kopfschmerz 1 0.97 0.97 Paresthesia Parästhesie Parästhesie Parästhesie 1 1 1 Fatigue Müdigkeit Müdigkeit Ermüdung 0 0 1 Vertigo Schwindel Vertigo Vertigo 0 1 0 Hypoesthesia Hypoästhesie Hypoesthesie Hypästhesie 0.98 0.87 0.91 Back pain Rückenschmerzen Rückenschmerzen Rückenschmerzen 1 1 1 Arthralgia Arthralgie Arthralgie Arthralgie 1 1 1 Myalgia Myalgie Myalgie Myalgie 1 1 1 Abdominal pain Bauchschmerzen Unterleibs- schmerzen Adominal-schmerzen 0.62 0.65 0 Diarrhea Durchfall Diarrhöe Diarrhoe 0 0.95 0 Pain Schmerz Schmerz Schmerzen 0.96 0.96 1 Nausea Übelkeit Übelkeit Nausea 0 0 1 Asthenia Asthenie Asthenia Asthenie 1 0.95 0.95 Weight loss Gewichtsverlust Gewichtsverlust Gewichtsverlust 1 1 1 Diminished ability to concentrate Verminderte Konzentrations- fähigkeit Verminderte Konzentrations- fähigkeit n/a - - 1 Diminished physical functioning Verminderte körperliche Funktion Eingeschränkte körperliche Leistungsfähigkeit n/a - - 0.66 Dysesthesia Dysästhesie Dysästhesie n/a - - 1 Palpitations Herzklopfen Herzklopfen n/a - - 1 Chest pain Brustschmerzen Schmerzen in der Brust Brustschmerzen 1 0 Night sweats Nachtschweiß Nächtliche Schweißausbrüche n/a - - 0.67 Jaro mean 0.70 0.76 0.76 4. DISCUSSION The ratings provided by the experts for both GPT-3.5 and DeepL indicated that, on average, there were favorable assessments for translation quality. However, the subjectivity of the evaluation and the lack of explicit evaluation criteria, such as completeness, comprehensibility, or technicality, pose challenges. Despite these limitations, the ratings remained consistent with those from a pre-study, suggesting consistency in expert evaluations and a common understanding of accurate translations. Statistical analysis via the Mann‒Whitney U test revealed that there were no significant differences in the mean ratings between GPT-3.5 and DeepL for both the 100 randomly selected terms and the 20 common terms. This suggests that both machine translators performed comparably in terms of translation quality. High ratings for terms with more than 7 words, such as "Elevated proportion of CD4-negative, CD8-negative, alpha-beta regulatory T cells" or "Anomalous insertion of papillary muscle directly into anterior mitral leaflet", show promising results even for more complex terms. DeepL is slightly ahead here. The observation that the multi-word translations exhibit comparable performance to their shorter counterparts may be attributed to the incorporation of additional contextual information. In assessing interrater reliability, the study revealed data homogeneity among the ratings for the 20 common terms. This resulted in low values for both the ICC and Fleiss's kappa, indicating that traditional measures of interrater reliability may not be suitable in such cases of minimal variance and uniform ratings. In addition, our analysis revealed instances where the same expert rated identical German translations produced by both translators differently, indicating some degree of inconsistency in rating assignment (intrarater reliability). To validate translation quality, an independent reference translation from the HeTOP database was employed for 15 of the 20 common terms. The Jaro–Winkler similarity metric revealed high similarities between the machine-generated translations and the HeTOP reference translations. However, it is essential to acknowledge potential nuances, as the threshold may exclude moderately similar yet semantically relevant translations. In the cases where the similarity of the terms only slightly exceeds the predefined threshold, as in the comparison between "Bauchschmerzen" and "Abdominalschmerzen", with a similarity value of 0.62, it becomes clear that the degree of similarity requires careful examination, since the similarity in this case lies solely at the end of the term. There are various similarity metrics for measuring text similarity, including the Levenshtein distance, cosine similarity, and Jaccard similarity. However, Jaro‒Winkler stands out because of its ability to weight the prefix (the beginning of words), which is useful for capturing similarities related to singular/plural differences. This allows for a more precise capture of similarities in words, enhancing the detection of semantic similarities. Metrics such as the BLEU (bilingual evaluation under study), which is used in many translation studies [ 2 ], are designed primarily for evaluating machine translations when performing 1-to-1 text comparisons with n-gram decomposition and are not necessarily suitable for direct 1-to-1 string comparisons, such as for our comparison of the 20 common terms with the HeTOP database. When comparing a machine translation to a reference translation, there are several limitations and challenges to consider that can impact the evaluation process. These limitations include subjectivity. Different medical experts may have different interpretations and preferences for how a particular text should be translated. It was challenging for some experts to evaluate the quality of the translated synonyms in comparison to their English counterparts. There was a tendency to evaluate the synonyms in relation to the main term. This intricacy is attributable to the specific study design and could be mitigated through the adoption of a randomized presentation format for the translations under evaluation. Challenges in interpretation occurred in cases of spelling errors in translations, such as "Hypoesthesie" instead of "Hypoästhesie" in DeepL. Experts have also observed instances where English synonyms are inaccurately associated with specific terms. For example, in the case of the term "fractured facial bone", which was one of the 100 randomly selected terms, an English synonym stored as "bone facial bone" was identified that appeared to be mislabeled and that may be more appropriately labeled "broken facial bone". Since this made it difficult to evaluate the quality of the translated synonym, the rating for this synonym was removed from the overall rating. GPT-3.5 has several limitations, such as the risk of generating incorrect or biased translations. Providing additional details and contexts through the prompt in GPT-3.5 could improve the accuracy and quality of the translation, especially in regard to medical terminology information, e.g., providing information that many terms might have their roots in Latin or Greek. However, we acknowledge that the optimization of language models such as GPT-3.5 falls under the domain of prompt engineering and that simply adding more information does not guarantee improved results. One possible approach to improve translation quality is to combine translations from multiple translation engines and select different translation candidates from them. This can even be done on the basis of different input languages and support languages [ 18 ]. In addition to analyzing the translation quality of commonly used HPO terms and the influence of term length, an alternative approach could involve a range of medical experts in the selection of HPO terms on the basis of their significance, difficulty of translation, phenotype, and frequency of use. However, this approach is not without limitations. It is conceivable that medical experts without expertise in translation may subjectively assess the complexity of terms, which could lead to inconsistencies in the selection process. The generalizability of our results to other languages must be viewed critically. In this study, the focus is clearly on translating terminology into German and investigating how well an automated process performs. For validation purposes, it was important to us that medical experts were fluent in both the source language and the target language. However, DeepL has more than 30 source and target languages and can therefore be used for many languages, including French, Korean and Spanish [ 3 ]. The GPT models also include various languages in their training data. The accelerated development of large language models has led to the introduction of newer GPT models during and following the course of this study. These models are anticipated to introduce novel innovations and enhancements [ 9 , 19 ]. To facilitate the transfer of the study findings to the translation performance of the current models, the 20 common terms were retranslated with the GPT-4o model. This resulted in translations that were identical except for a few instances of singular/plural differences and minor adjustments to the translations of "Diminished physical functioning" and "Hypoesthesia." The Jaro–Winkler similarity between GPT-3.5 and GPT-4o was 0.99, whereas the similarity between DeepL and GPT-4o was identical to the similarity between DeepL and GPT-3.5, which was 0.76. These values indicate comparable results, thereby demonstrating uniform validity. Notably, our study revealed that 75% of the common HPO terms had German reference translations in the HeTOP database. Given the limited sample size, the results are not yet statistically significant. However, given the paucity of studies on extensive translations, these findings underscore the incomplete coverage of translated medical terminology and highlight the importance of our study, particularly for the documentation and diagnosis of rare diseases where precise distinctions in disease characteristics are vital. 5. CONCLUSIONS In our evaluation of machine translation of the HPO, GPT and DeepL emerged as viable choices. DeepL provides very good translations in the eyes of medical experts, which are notably rather technical. GPT has significant challenges, including unpredictable ambiguities in output, such as sporadic synonym reduction. The presence of arbitrariness and inconsistency in translation, especially within medical terminology, could be a nonnegligible problem in contexts such as clinical practice and automated analysis. As language models continue to evolve, the choice between machine translators should be made with consideration of their respective strengths. For the initial translation of extensive terminologies such as the HPO, translators such as DeepL show great promise but require additional manual annotation and validation by medical experts. However, the information gain that could be obtained by translating these extensive terminologies, especially for the purpose of describing more complex cases such as rare diseases, should not be neglected. The complete translation of the HPO into German with DeepL is provided by the corresponding author as an 'OBO' file in a repository. In addition, an interactive script was created to perform search queries in the German HPO. Abbreviations API application programming interface GPT generative pretrained transformer HeTOP health terminology/ontology portal HPO human phenotype ontology ICC intraclass correlation coefficient LR Likert rating NLP natural language processing OBO open biomedical ontologies SD standard deviation UMLS unified medical language system Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Availability of data and materials The complete translation of HPO (v2024--01--16) with DeepL can be found at https://github.com/RichardNoll/HPO_German. The German terms, definitions, and synonyms for a list of HPO codes can also be output via an attached script in Python. It is also possible to identify corresponding HPO codes by entering terms and synonyms in German. The script can be run interactively via a Jupyter notebook in Google Colab. Competing interests The authors declare that they have no competing interests. Funding This study is part of the SATURN project and is funded by the Federal Ministry of Health in Germany (Reference: 2520DAT02B). Authors' contributions The study was designed by RN, and the manuscript was written by the same author. HS and JS provided research support and advice. The remaining authors are clinical experts who answered medical questions, evaluated the translations, and suggested adaptations to the manuscript. Acknowledgements The author was supported by medical experts from the University Hospital Frankfurt. We thank all the participants for their participation. References Ivanović M, Budimac Z. An overview of ontologies and data resources in medical domains. Expert Syst Appl. 2014;41:5158–66. Noll R, Frischen LS, Boeker M, Storf H, Schaaf J. Machine translation of standardized medical terminology using natural language processing: A scoping review. N Biotechnol. 2023;77:120–9. https://doi.org/10.1016/j.nbt.2023.08.004 . DeepL. https://www.deepl.com/en/whydeepl/ . Accessed 12 October 2023. Noll R, Berger A, Facchinello C, et al. Translation of Ontological Concepts from English into German Using Commercial Translation Software and Expert Evaluation. Stud Health Technol Inf. 2024;310:89–93. https://doi.org/10.3233/SHTI230933 . Dale R. GPT-3: What’s it good for? Nat Lang Eng. 2021;27(1):113–8. https://doi.org/10.1017/S1351324920000601 . Köhler S, Gargano M, Matentzoglu N, et al. The Human Phenotype Ontology in 2021. Nucleic Acids Res. 2021;49(D1):D1207–17. https://doi.org/10.1093/nar/gkaa1043 . Gargano MA, Matentzoglu N, Coleman B, et al. The Human Phenotype Ontology in 2024: phenotypes around the world. Nucleic Acids Res. 2024;52(D1):D1333–46. https://doi.org/10.1093/nar/gkad1005 . Robinson P, Köhler S et al. https://github.com/obophenotype/human-phenotype-ontology . Accessed 26 July 2024. Koubaa A. GPT-4 vs. GPT-3.5: A Concise Showdown. Preprints. 2023. https://doi.org/10.20944/preprints202303.0422.v1 HPO. https://hpo.jax.org/. Accessed 12 July 2024. ChatGPT. https://chat.openai.com/ . Accessed 12 July 2024. DeepL API. https://www.deepl.com/en/docs-api/ . Accessed 12 October 2023. Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. 1947;18:50–60. https://doi.org/10.1214/aoms/1177730491 . Koch GG. Intraclass correlation coefficient. In: Kotz S, Johnson NL, editors. Encyclopedia of statistical sciences. New York: John Wiley & Sons, Inc.; 1982. pp. 212–7. Nelson KP, Edwards D. Measures of agreement between many raters for ordinal classifications. Stat Med. 2015;34(23):3116–32. https://doi.org/10.1002/sim.6546 . HeTOP. https://www.hetop.eu/hetop/ . Accessed 12 July 2024. Jaro MA. Advances in Record-Linkage Methodology as Applied to Matching the 1985 Census of Tampa, Florida. J Am Stat Assoc. 1989;84(406):414–20. https://doi.org/10.2307/2289924 . Prunotto A, Schulz S, Boeker M. Automatic Generation of German Translation Candidates for SNOMED CT Textual Descriptions. Stud Health Technol Inf. 2021;281:178–82. https://doi.org/10.3233/SHTI210144 . OpenAI. https://openai.com/gpt-4. Accessed 12 July 2024. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 01 Jul, 2025 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted Editorial decision: Revision requested 10 Apr, 2025 Reviews received at journal 09 Apr, 2025 Reviewers agreed at journal 25 Mar, 2025 Reviews received at journal 20 Aug, 2024 Reviewers agreed at journal 15 Aug, 2024 Reviewers invited by journal 15 Aug, 2024 Editor assigned by journal 01 Aug, 2024 Submission checks completed at journal 01 Aug, 2024 First submitted to journal 31 Jul, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4836251","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":344546717,"identity":"66095791-2683-48f1-838d-12c595676d71","order_by":0,"name":"Richard NOLL","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBklEQVRIie3PsUrEMBjA8S8UekvBNYcUX6GS4Sj09FVSCp16Lk5u31Fw8uh6vkWmgJMJAV0qt+oi9wAON56DaNqDDmKqo0P+Q0lDfvkIgM/3LwuWagdA0C4VZPYbkW233+/8HEG9HkjZkSAB4OPERMOdfyGzyRLNvMpipAutgL+ezCITHl/tIW4cJL3RaBayZEgvuCWXp3erOpy2HNitY0zynHfE5EirxHzsORGbo4cpcsiFGiGp/OyJncLPxSYIe3I/RohUA8nFU30gwvGWpNWoV7Jg19FbTwrRmiDFkrK1a8pjrbfv8ixuJhXbWTIXbUFeMMvixvH8ofDbP/3lvM/n8/nG+gJyqmRWC+NXZgAAAABJRU5ErkJggg==","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":true,"prefix":"","firstName":"Richard","middleName":"","lastName":"NOLL","suffix":""},{"id":344546718,"identity":"272585a4-fba8-4fcb-978e-dc41ba62ed3a","order_by":1,"name":"Alexandra BERGER","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Alexandra","middleName":"","lastName":"BERGER","suffix":""},{"id":344546720,"identity":"85b0c2b4-faef-48d0-a7b4-1d7d2f3ff0fa","order_by":2,"name":"Dominik KIEU","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Dominik","middleName":"","lastName":"KIEU","suffix":""},{"id":344546721,"identity":"028748b0-2248-410e-83a3-217260b6ab1b","order_by":3,"name":"Tobias MUELLER","email":"","orcid":"","institution":"University of Applied Sciences","correspondingAuthor":false,"prefix":"","firstName":"Tobias","middleName":"","lastName":"MUELLER","suffix":""},{"id":344546722,"identity":"dbae4eb6-2704-4a60-acca-7ea124d431bb","order_by":4,"name":"Ferdinand BOHMANN","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Ferdinand","middleName":"","lastName":"BOHMANN","suffix":""},{"id":344546723,"identity":"c442a84f-6655-431a-a2d9-c3aca861eb70","order_by":5,"name":"Angelina MÜLLER","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt, Institute of General Practice","correspondingAuthor":false,"prefix":"","firstName":"Angelina","middleName":"","lastName":"MÜLLER","suffix":""},{"id":344546724,"identity":"96e2bebf-eb36-4c67-93db-4484d2d4fb1a","order_by":6,"name":"Svea HOLTZ","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt, Institute of General Practice","correspondingAuthor":false,"prefix":"","firstName":"Svea","middleName":"","lastName":"HOLTZ","suffix":""},{"id":344546725,"identity":"8f0b63f4-0bd5-42fb-8f7c-98539fa84d96","order_by":7,"name":"Philipp STOFFERS","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Philipp","middleName":"","lastName":"STOFFERS","suffix":""},{"id":344546728,"identity":"14026682-1fc8-4e4f-b262-190b97008b9f","order_by":8,"name":"Sebastian HOEHL","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Sebastian","middleName":"","lastName":"HOEHL","suffix":""},{"id":344546729,"identity":"a31f588c-a312-43fb-b4cc-e0c7cdfe41c1","order_by":9,"name":"Oya GUENGOEZE","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Oya","middleName":"","lastName":"GUENGOEZE","suffix":""},{"id":344546730,"identity":"40733f7e-68f8-443c-bd71-5e76012abdcd","order_by":10,"name":"Jan-Niklas ECKARDT","email":"","orcid":"","institution":"Technical University Dresden","correspondingAuthor":false,"prefix":"","firstName":"Jan-Niklas","middleName":"","lastName":"ECKARDT","suffix":""},{"id":344546731,"identity":"94db9798-2171-48f1-ab11-ea600c21957b","order_by":11,"name":"Holger STORF","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Holger","middleName":"","lastName":"STORF","suffix":""},{"id":344546732,"identity":"f78ded98-b562-4900-8a84-b13e78e14a01","order_by":12,"name":"Jannik SCHAAF","email":"","orcid":"","institution":"Goethe University Frankfurt, University Hospital Frankfurt","correspondingAuthor":false,"prefix":"","firstName":"Jannik","middleName":"","lastName":"SCHAAF","suffix":""}],"badges":[],"createdAt":"2024-07-31 14:15:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4836251/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4836251/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12911-025-03075-8","type":"published","date":"2025-07-01T15:58:43+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":63477900,"identity":"589f6b76-d4bb-419b-9966-e0bab4924dd8","added_by":"auto","created_at":"2024-08-28 14:23:55","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":97548,"visible":true,"origin":"","legend":"\u003cp\u003eExpert ratings for the 20 translated common HPO terms and their synonyms.\u003cstrong\u003e \u003c/strong\u003eThe scale ranges from strongly agree (1) with the translation to strongly disagree (4). The color differences indicate the ratings of the different experts.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4836251/v1/829943a411031e56c54fbd30.png"},{"id":86180019,"identity":"6262009a-b37c-468e-9521-70e0c951e6fa","added_by":"auto","created_at":"2025-07-07 16:20:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":746703,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4836251/v1/8c0c0e38-75db-440a-8ea5-8006531fe9d4.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Assessing GPT and DeepL for Terminology Translation in the Medical Domain: A Comparative Study on the Human Phenotype Ontology","fulltext":[{"header":"1. BACKGROUND","content":"\u003cp\u003eTerminology translation is a crucial task in the field of medical informatics, as it allows for the sharing and integration of knowledge across different languages and medical domains. Medical ontologies are formal representations of medical knowledge that enable the semantics of medical concepts and relationships to be expressed in a structured and machine-readable format. They play a vital role in many medical applications, such as clinical decision support and biomedical research [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHowever, the translation of ontologies and their medical terminology is a challenging task, as it requires expertise in both the source and target languages and a deep understanding of domain-specific concepts and relationships. Recently, interest in the use of natural language processing (NLP) techniques, such as machine translation, to automate the ontology translation process in the medical domain has increased [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. One of the most promising approaches is to use commercial translators that are trained on large-scale text corpora to generate high-quality translations [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. In a previous study, DeepL proved to be the most accurate translator out of 12 different commercial translators (including Google Translate, Bing, etc.) [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. With OpenAI's generative pretrained transformer (GPT) making waves in the technology industry, many experts are excited about its potential to revolutionize NLP [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. It is therefore a logical step to consider its potential for medical terminology translation.\u003c/p\u003e \u003cp\u003eIn this paper, we present a comparative study of terminology translation via GPT-3.5 and the commercial translation software DeepL. Specifically, we focus on the translation of standardized medical terminology contained in the human phenotype ontology (HPO) into German [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. As of January 2024, 11 languages have been integrated into the HPO web interface. These include English, Chinese, Czech, Dutch, Dusun, French, Japanese, Nyangumarta, Spanish, Tiwi, and Turkish [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Although there are already initial versions of German translations of the HPO, these translations are incomplete [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eTranslation studies often overlook the range of synonyms for medical terms, which affects their practical use in clinical settings and automated analysis tasks. Many approaches focus on primary terms, resulting in technically correct but incomplete translations. The key value of our study is the comprehensive inclusion of synonyms, which improves accuracy and applicability in medical contexts.\u003c/p\u003e \u003cp\u003eGPT-3.5 is a large language model that uses deep learning techniques to generate natural language text. It is trained on massive amounts of text data via an unsupervised learning approach, which allows it to learn patterns and relationships in language without the need for explicit annotations. GPT-3.5 has been shown to be highly effective in a wide range of NLP tasks, including machine translation, text generation, and question answering [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDeepL is a commercial machine translation software developed by the German company DeepL GmbH. It is based on neural machine translation techniques, which use deep learning algorithms to learn the statistical patterns of language from large amounts of parallel corpora. The software supports a wide range of languages and domains, including medical terminology, and is optimized for various translation tasks [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eOur study aims to answer the following research questions: How are the terminology translations produced by GPT-3.5 and DeepL evaluated by medical experts in terms of quality, and to what extent do these translations correspond to a reference translation? When medical terminology is translated, how good are the respective translators in terms of error proneness?\u003c/p\u003e"},{"header":"2. METHODS","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n \u003ch2\u003e2.1. Study design\u003c/h2\u003e\n \u003cp\u003eWe selected 100 random and 20 common terms from the HPO for translation. The 20 common terms were identified by medical professionals from 178 letters from doctors at the Frankfurt Reference Centre for Rare Diseases at the University Hospital Frankfurt. The common terms were selected on the basis of their frequency of use and their significance to the clinical profiles. Importantly, the rare disease community makes extensive use of HPO to perform differential diagnoses.\u003c/p\u003e\n \u003cp\u003eThe translations were performed from English to German via GPT-3.5 and DeepL. We collected the original terms and the translated terms in a spreadsheet for further analysis. The synonyms of a term given in the HPO have been included in the translation. The exemplary structure of an HPO term can be seen in Table\u0026nbsp;1. The English version of the HPO is freely available and can be downloaded from the HPO website as an ‘open biomedical ontologies (OBO)’ file [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoMAAADcCAYAAAALKauFAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAE99SURBVHhe7Z29rhXHtraXv8uwkGWZfQ0EyOZIDowvgMD2OQERknds2QmBj0TiLZKdeEtEBEewAy4AEyALLAdcg9lCCHEb/ubTzHf6XYOq7p5/62++j1RrdtfPqFGjqrvHquqfD/5ccBRCCCGEEA6S/7f8DSGEEEIIB0icwRBCCCGEAybOYAghhBDCARNnMIQQQgjhgIkzGEIIIYRwwMQZDCGEEEI4YOIMhhBCCCEcMHEGQwghhBAOmDiDIYQQQggHTJzBEEIIIYQDJs5gCCGEEMIBE2cwhBBCCOGAiTMYQgghhHDAxBkMIYQQQjhgRp3Bf/zjH0cffPBBN/ztb39b5mzz73//e5X3t99+W8b+xd///vdV+jogS+UI1DOX//znP6tyX3755bBPOw8B78+pNpOn1WdzYWyoLoLwOPLUfIqv1D6fGnubwDjah1yHMU84b2B/6c0xIzu1+sSPMQLHme97/Bg+XpHpbJo2ds4ZS6NtvTS1189D0gG76Tw4dcy1kIwpsGVPfk0bk6njbF+gx1S/n3XGbH2e2MW5aFNb7MKG68ggL+Nax+HcMT51TdiFDc8Mf47w008//fn8+fPVNtkvX7487MP169eXW33IQznJqZA2oUYXyX748OEyZhr0V350ojxtE99+++1ya7fsS+5caKv6i/Z7P8LLly9XdiBtrM/mgrzWGCHObV7zUbfrRzpx6ChacWcV9b103sVYOOnx5PXVutlv6ePHGng++g1btMYHMPY0BpRXbJqGLqrP88FYGvFqB33oaeSlDoK3VZBf+PYckIfclm0d0snXkj+WBthoXb02Re3p9flJM2VXMTffoTE1tnpsWs5ZRwZ5lI9y2/QncjhmLiKjM4Pff//90aeffrrce5/Hjx8vt84H/Ae/6MijK1euDPu0jX1B+r/+9a/l3u44C/9F/v7778uto6Ovvvrq6I8//ljuvePu3bvLraP30k6axQE39Av9Qfjhhx+OFhfdo08++WSZ493YXFxUjr744otlzNmE/yxpA6Dz4kQ0bG/DaYwn2sB/0/x+/PHHw/acGase9CW2+OWXX5Yxx7lz584QgLz0tWbeNk27ffv2EEDnNbWhl0Z7OR44ZoA+ZGyqHHkX59GjhYM47AvZi/zA7MKNGzeG7bk8e/ZsOBamzkk///zz0M4WY2ngx/2+wYa05yxA/8w518/Nd4hMja0em5Zz1pHx9OnT5da7coRN4Vp0UdnJPYOcbDnxKbQuVg8ePFil6+TcQssUPTnbwMWBk7Y7EMRxwqZendBVN3pyEvflI9dLS0e6MIAvJxFPWQYQJxTiNF3dk8E+acSxDdonjE1Ju+1UFqgTHbjwklbtTzr6kYd0R/pJX+jVsys++uij5dZfF6vWPyU3b948dmEW0q/aT7bThZp00tgHtrEFKI9+CW4375NWOpDn66+/Xtnd9VT5aj/Zu+YXdTxJd5XxOPISJFNtQa7i1H6VbdUJ/OOH06Pjhe1Wn6wDTqWQDrIhNtM/bfD5558PzhFskoZ8xorrzHmAf5LG0t6+fbuM+QsuQv7PVQvshI0EDiVxc0En7CMHstUvGkMas04vTXbmt3Xc65znkKZAOcF+HUvCx/HYOauiMi7PdVaaxslYGqCf4tUu4uq53ssTkNHKB8jxOmRrgreVeGxMXqULxekY7dGTrTiC9AL2e31SQTelow9l3F7YRFC34lsy1R6NN+X3vGx7Hoc4yXfbVsZk+JijHcA25wSNcdqktgrlb9mMberS+ADGhPqCNM8vOxBcR8nxdCGbkz42FvbO4oQ1C6ZHyb4wxDLmL4hTPL/kW5xch/3FiXPY19RsTWebAFpy0VQs28T1kOzW8kwPySXU6WLV79tVPnWinyBdOpKmvMhmX9teV0+G2uM2Jt7Len2O9BXIcjnUJ31atHQiYC/a5LJ8u5ZziJecGrxM1Y101VHb4ajNrf6vOnsf8Eu7NBbURralB9sKgI6SV23NdksHqG2jbvKr/WyrP0nTdtXf8bYA8mkDqD5+kU1QGvI8TnWoHmS6rnNRm1rB7eJ6y/at+pQmvUHt2jSt9hlIn3XTkIdcBxv2xsAmIF/toD7ZTfj4kI7SaSyNbYLLVpryqiywTzyQT2XJ47Kok21gG7kgmaoPGUqrkKZ2ujzVo33yScexNNUtqLeXRr2UhSkZ7KMfkNfbQxqypL/2AZmqQ2XIp7oqPTsio2Un5KjO2icVypOGLMkmqD5+VUevPiCf2sSvyoOneTtVn5cjHaq9nTEZ6EcceD5wPVRO6WM207baRLrygdsQar1sU1ZyCLIjaSon+WNtPwl2MjPIf70EvNqFoZaxx/nmm2+GX83KvXjxYvh1mD2Eq1evrv6LnvoPfF30H/uiU1azK/ovwmGWYNGJw38BWiIag/8c+A9EeZmKXncZnfzUp2UuwCbSU/9NtGxCPtokqJ++aLVtLovBOdjr0qVLq37lPxu2pQ9t9mn4ymKgD/b2QFwFOd7GfS9VM+OoscBvXcJSe/Xrs5Vv3rxZzRgAdn/9+vVybxrya/kQOcgD+vmzzz4bbMCMInX7f+ctNO6QQznNACOfegg6lmTT2q+K95m6daGe2s9uI6GxTBrhrN9qotlC/48de/l42AevXr1a9Rsz4NjNYUn7/v37wzY6+jE1lqbx3ELnPMGxTh/JBowp9h89etQcS4JjSf364YcfDr9zQL6W8Fze2LE4lrbOOZE0HZNjfatztLh3797QPwL70Vc6n2B7tclno7Af9iWfbFmhDyWbPtC5qmensT6pUF7jQv3ux6PGHtBGXb/RFz3WZWxMIp/zHecFzn/Q6qMxGXPPnWqrmBrHfk2ouA0B3W7durXce7fPuZiVirGxAMwcqo9Pi504g5pCvXbt2rEDZVPUqcBJcR/QKRieDvKDeVNYTtpF2yvYloGErgo6aTnk84u5H8y7BKcHm7k+u7igV5mCduhEX5ET5cuCc6Dv5ZT4VP5cOEn4yYbfbZ0DyUKu22GqH7WM6WUIZxV3GnsXwV6bid80reeUcMyMpQEXCo0XAn3EGNgXXAi9Pi5uihe9YwLG0taBY71etOr+GFzgNjkn4njLKdiGTc6JnA9k7zlga3cgxhwwB0dBDpAvMzocH2PydmWnKWjjOk59i6nxyjHm5y8co0pPxqbnzl1Df/l1YK7NuB5pGfs0l4l34gwy28dBv84JcmyQ47mrQ+VJ74pqbP7bGBuoc6HjkaOBuSsY0HMcYvK1Zui2PYgrDPbeRXwf6D9S/ouucC8YTuQmB73GF/+59U7GY+CgM+Y5gKl/W+dAbWjdozaG+nfX4+60oV999YCxzT+bsEka9qW/3KF68uTJahWilwb6j51Av/vMwj5gRkv1KdAurZwI/TPUYixtLhzrHB+VqX98sCPHBaxzbtX9VPQXzsG20K9zz4mcA6ib68E6/cu4qSs1xE2hMUU7cQR87Akc79Yq0K7tNAXtaa3krUtvTCJ/7nht5dv03Llr6C/du+xIvx6kMxY4VvgnsHWtOwm2dga5CNEIOUK9g1+dyEmWzm95/rrwc0AC8tb1lHVQ9y7w1dicYDnRwhzHiY6TcyY5+u+MdvnTedK9Lr/1ZLQu6Bzw6KyTBXlabcN2nLglj1/aNTUQxdx8zMLRx67DPv+bYZxwAea/aLcP9WOX3j8L/LPhY5G8BMqhr2Qhe13oC/pPF+qxf1jWmTGkv3yWGl1bY6LOdjDuvA/22R8nBXbQkhD2ZmzL4d40jSUcOVQ6TnQeGksTmkXe1vEfo9XfwDmR8at0Zlh9Bot24lSg+1haZey4Vzs1nqjb7dkDO6JDawVjDPqN43FX9h07J9ZzPUuVOIG1z6euCYwbbKt+oe2+VNhDY4n6dP2p8PATsgXjmTbs2k5TMNmjazKgA7pUfKJAY2XOmES+p/Xkj8mYe+7chqlrJPX7tZrbKdB5Co0F5M/JvzcWF7NJFoYebmz0sLjQLlP/ugmTX+VlG9gnr/IQxKLhqzi2YTHIV3GSUVn8N7TKo0A9IJnIaUG811vrkJ6ur+fxuiWHOKE0gnTwMmz3ZLidFyemoSy4TQg9KKM8rnPtP+874WU91Hhw/QnsV9x+BOFx5Onlq9Q6vX09XDZ2JgD2VJrkuI19fBCqXbGfpyu08LxVn1Z/1zwt3BayvfYJtKW2B1x27Vff17E0B5chW1b71PGnfI7KoLfwNtQxu2ma922lleZtaY1zTye4/j2wR8sG3gcup1eH9ycyvUwrrcoBr/Of//znaltjoJZhH1x+HUs+Pj3f4sK92m6NbZfj5TyM5alpLZmOyqML9vF8nt/zaZug8eDjRu3yerGl56GuVpkWnk/69Gzg256HUEEnpf3444+rbWS4LaSb59fY8DjqA+0jh/SpMSla8luMyfA06a19Aum1X7xMtdmYHfjVNkHt9zKkQ63T+5T8Y2Uk9yT4gD+LSkMIM+E/TpYkfBaBOJZSTuq/9RBCCBcbZjjXnWHflJ3cMxjCIaEn4h1uD4gjGEIIYVtY+uZ2t3VfVL8NmRkMYU24J6Q+xffw4cM4gyGEEM4lcQZDCCGEEA6YLBOHEEIIIRwwcQZDCCGEEA6YOIPhwsGNtwqHCE820/Zdv2drLtxTeV5tvwvbUb71nrQpu/A+v9N6R+RY3fu0CfhXTdhWPdxEv84XT+agtqitbM+p4yyM6V30wxhjfbQOPAGLLIKPKT0U0dK/lZ9+UTxlw3658M7gPl4+Gc4uvMDz+fLTRpcvXz7Ikwjt3gc9Z8HhWDuJT2S1mKPfFLz4lbEz9YLZHj3HYo5deHn5rr+4NAfOkbwst8e+bAL0mV5UjB58lED18EDWrr92pLboxe1sT70k+jTHtLOv41pgi/rS7U25fv3d50U1nulbf2G0wzmbB/DID+QF+p64fbc7vOPCO4P+Bvdw8eGt9IKTySE+4fuy8xWgbdAJegoutjjjJw1O/1n4p6/nvJyWXebAe8y+3eOXD8YcOpwyzfzw+T8u/HPH2qbgfOjLQHPqOit9t4/j+qRgjLX0ZybSz9N8NYavwYSTZ9IZ5ASrqVqC49O4Pr1MPAc3Bx1p+o9deTVbI9leB2maklea4MBVvGT2ZGgfOMGQX3HohyyX3UNtQCZlXAe1WftAPQTlq/8Vk6b85BHUw77S2Ve88oNso/S5tgblUVvm4PVLFmWJV5rs0BsPrXq9HdK5Rasu72sCSB7wX7zsTp3KhwzBPrI9vtU3qsvrdJv25Esfgo8B1UG5XrvV1iqz19dC9uczSGOo3VWGbO1tAvLwTxWzR8TpuHE93SYtemPD7aQgGxHUfumsfnGIY9aBfwTI43Whq/pO/YB8BY8HH0dCsghqu9uQsA4+ltwWtNXb53ZQvPRT/fx6fxFcR2Sqn1RXK6+Q3E1s0tNjDJwEZn9wBpiVYlsv2ZX+Qv1KPVPjDdx+3p7Hjx+vnA/qUn0tVL5+I9jbWnUkTbqig5AdCa6P+l36ujyQrHpc+zgiiLnyCBoTkuV9pjwer3Yj29u2KdjV39vKGMBpnDN2wo5ZHHyjkMU/QaTPv7CtT6Xo0yl8nsc/0cI+gW3yA+W1rXwEUJo+yaJPtwBytA2S35MB6E289Kc82zW+B/LVXnTRNu11XYhHFr/SRXnZlp0oo3LSgXzEqRx5lUb9QLpkgGSsY+teW8agHuWTLPW16wdKA+WRTq162Qbp3GKsLoGOsgd4XuyoekA68ks+TyNeusn++lUA2QGoR9u1z1xHtZs80rXqLXoykSE92CewTR5QHYBcT3PI4+0mH/nd1tILPVym6+v1SRdk+L5AjtJUD3nAt5HpdVDO2+Bpldou9qUfqCxx1On6Ug/psjdBkNbSjzzSze0EpKmMI7uQH5AlefySJjnopm3p9euvvw6/ng9aOsrOBNel155av8qtY5NePFQ9ppD+yNG+95e2e0hvwfZUmYrXgx6Sh2zpBcqnPATySAfXW5AXW8vuyqcybkf1ifKSB1weaQpT8qQPedUOlZFs4rVN/cqnX2R4/Y7nd6SL5IJ0dlxf8H4I++Ovo6UBHdDqVDqKDnPIp071zqsDwAfRWBr4QOGXvB4YdOvId3lzaLVTeLzLrHW4LSjjgxz9pSv200EPnpdf9QNt8Xwuf8wWY23p4bKcerC3ZGs89Oqt7e3Rq8uD2gjskwcoW/OqbzwfUEfNi/3GbIqsVhvIU2Uhv8qaA/mlZ6+vSXcbjNXjMkC6QrW195GP65Z80lRWfVS3BfnIX+Wgi9dP3aqfNO+vStUdmW4T6Q7eFkC28lbdXUYPtxNQvqVrtUXV2eWwTV4PsoHLqHjbqvyx9lSb+DjxOsdkOFUe5Sm7Dq6/99EmeHvmQN5qO9kAvdj2oLbWemQH4moZyedX/Q6kUV+1tR8v0seD8vbkqXwLl63tGmCOHeu4E16HqOMEyIO+Yt2+C5sxukz8+vXr1c28zps3b967qbOVb5cwbbwYZIzIVRib3m/Bzaxa6qpT5y2Ysl4M1PemygFdmIpnmv3atWvL2D4q++GHHw6/oPtWptBNvdTFUsEmn6gZa0sLpc/p17Hx0KuXvmDpkTgtN2hZQ6GF6vJx0Lsn6dWrV0Pdnrd3gz4yFiecY3mn7jekLa0+5LhZnAyPyWJpCpswbtCf9mmJpoWWY+aATeayOBEfXbp0abl3dGx7LnyXGXxs6Ib8ytjY4Bc78U1nqOcbjm/d8/vs2bPVcTBnnCCH5SfsSOBepB49G6C7L+dV6H/q9/tUd8Um4xFdxh4EmWrPHObImNJjE2i7L2/ORUuzjPt1YCz2WOe8ouOid04YY+y41nHl8sbuzQSO23osttDx7bIJcP/+/eGWDGzK9W9beucNv06Gk2HUGeRCx9NdFU6eHFzVoeh17C7g5M5BuC0a2JzA5wxmDnLyc/D7vQ04ZNzoyj0PUydp0EVOFz4x9+R88+bNowcPHgzbfsFch15bWqiOMYdFTI2HVr3IJ45yXDi4YCufQgvVNQfqrzr1oB/GLgAtaANOSoXjpndixsGhbTiFrScU5WAgt2eDFnNtwsWg3v805wLh6ERdx0ZrXE6Njdu3b68uLhxP9aLKmGFs+LllzjgBHEAuXvSrHMkWPQcH3XuOHvkZy9TPRX7XrDMe+ccW+6EL9uox1p65jMmYq8em4DzJ3nP+mUeXO3fuDGXWHePQO4bXOa9wzSL/2DlhjN5xvc55UHDcto7Fio7vVj6OI+z5/Pnz4R+1OdeHMXhoyH0M5NFXm17jwuaMOoNXrlwZBg8nY8F/5QwIOsxvIOWCvs+PKjP7Rh0afAzUKWeuDij01QDnYjwFdamO7777bvgVyOaE/fTp02XMNJwg/fF6LlQ4eXPA4dzmv+2xtvTgpMvJVFC+d4LojYdevTqZY8d1Lhw+OyR82+FEw4Wrjt8Wn3/++XByU/vQ28u10JhUGY1JHTdqN2jsqf7erDYzv9i9OkVj6J8RydY/HK0LIK/R8HbyD8bUqzXAnTH6DB197OLItWbfps4VyODiQmhdLJHJMUNfjtGaoaVuZLb+ifQLUO841Fir/cjYoH83ubjPZe54JJ0xPscx6LVnHXoy1tFjE2i72s8/EFOQl3E3NfvWg/Hpx7D+geKcP3Ve8X8QNdZ754Qxxo5r9YPLmJLHcUvZu3fvLmPaZZSvJVvnberfxT9BagfjG7jezDkfhT2wOAmPUu9N8LX8VvxigKzi6n0Svr8YbKNpC2dttb9wFgbZHkcA368yQPrwS3nVq/TFATrsqw6HNnl7vO3APnWKqnO1BXgc+YG8Hue28TpJQ19R5Wu77iOv1xbSZIsWrgt6ulzkOYonSH6vXrdVlSN6danPFMiHXI9Tf1a7ULZnX+8H2UT7hGpT8HYQRNWHfep2WyCvUtvmuiq02uT1qQ7iW3g7W3aivOehjS5fNnPd1BbPJxuB4ggqD632eTpIxzHcbm5XtlvyvB8kv6V77Q/Jdr21XceC278lR9vo0hoX3gfUUWUIL9uyJ+mipYfrXW3z448/rranbNLSw9tAcJuMUXWiDpc/R47yEqSPdJ0DeVVedQtPI0gf6nE9vT4fXwT23T602fuPdC8juaqr1Q9T8kD7ylPlCI8jH7h8tluQF12dai/pIsZk0gbZUfWH3ROrLtFgX4fewbAv9lUfB9qck2sIu8YvlsKPRcblJsemaB0zxJ30sRsOA3dcDpWWM7gNsenJcOFfOj0FyxvcW9JaZjpLMI0+50GVdWHZg2UNlgZCOElYeqr3xbH05kvCmz4wBRzbvrwdQgihw9IpDDPhPxTMdlKmYwaDunb5n1YIZwUdSwqasdPy1iazgpShLDIqSvO6wvlEY6QVtplN3pTe8vChsatjzPs3M4P75wP+LIwdQgghhBAOkINfJg4hhBBCOGTiDIYQQgghHDBxBkMIIYQQDpg4gyGEEEIIB0ycwRBCCCGEAybOYAghhBDCARNnMIQQQgjhgGm+Z5AvcoQQQgghhNNn36+EzkunQwghhBAOmCwThxBCCCEcMHEGQwghhBAOmDiDIYQQQggHzKQz+O9///vob3/723LvOGNpzpdffjk8lKLwn//8Z4inrMfXfApjIEv5/vGPfyxj3zGW9ttvv63SaIczlsa+0sjnUEevHNDev//978u9EEIIIYQzAA+QbMLz58958OTPy5cvL2PGefjwYTPvt99+OwRR87E9pibp6ALXr18fyouxNGS+fPly2PZ80EsjznXxfMimDlC+KpPgbQ0hhBBCOG02Xib+9NNPjxYO0HJvf9y5c2f4rbNwoJlJdIGbN28e3b9/f9geS2MGb+GUHX3yySfD/q1bt44ePHgwbI+l3b179+inn34atoF8jx49GraRffv27WGbsqSpHCxsPcSFEEIIIZwlJp1BnCOWb4Uvvb5+/XoZ+w7i6nLstly6dGm59Q5fan327NnR559/PmzDlStXjn755Zdheyzt6dOnR9euXRu24erVq0dPnjwZtsfS+GVfkI/88Mcffwy/gjSVCyGEEEI4q4w6g8yu/fDDD8u9d1y+fPno+fPnw0zXvXv3lrHvIO77779f7r3Py5cvV46kwr/+9a9lahtm9EAzfDhdP//887Ctew8rxI+lVccN0A3G0vTreP43b94st0IIIYQQzgejzuBXX311bFlUS6hyzLSEOxccSRxGD62lU3ca2W45YWeNL7744ujrr79e7h0Ns6bMYoYQQgghnGXWumfw1atXy639Up1G3b9XGYsfS2s5adQJY2n6dZSf2UrS5cQyo6pZzRBCCCGEs8raD5D0ll9PA79nD168eHF0/fr1YXssjXsJuadQ/P7778PMHoyl8cu+qPclsmSM88pMJnUxsxpCCCGEcJZZyxn85ptvhocw9A49nCEcH3/A5CTB2UIfOag8zavZuLG0GzduHLtXkVk82gZjafzqPknkko/8DrbBaXz8+PEyJoQQQgjhDPPnCD/99NN778fzON6rd/nyX+8E9HwO+VSGsHAgh3jKenwvn0MZ8gnyKD+6OWNpvANQaf7+QRhLY19p/h5B6d5qP3hbXf8QQgghhNPkA/4sHJQQQgghhHCArH3PYAghhBBCuDjEGQwhhBBCOGDiDIYQQgghHDBxBkMIIYQQDpg4gyGEEEIIB8y5dwZ53x9f/Gi9DJvP553WOxDX5e9///sQTgPejbiLT+dha2y+Lruqf9+MjbUpTrN/QwghhDHOvTP46NGjY78CB4MXRp8HcKD8RdcnyW+//Xbsm8ot5jgx5OEl3+syp/6zAp8b3ITT7N8QQghhinPvDPK95G+//Xb1ZRDBF0h++umn5d7Z5vvvvx/acBp8+umnRw8fPlzuvc/cmT6+zXx9+bm/dZiq/yzB13Y24TT7N4QQQphi0hlkWYzZG5bx2HbngGVB4gi+HEteZuaUrpkl5SVNsN2SQZmppUP04hvEfCaOC3Vv+Q6dVYdAPkFptS7SVKa22eX50qHiaBO6eR6hOILLHaPWCT27sd/rL8UR1CdjkIfZVWa1VI/LlDzqc5RPemlf/c42oddfbrveGEAWadXOslMt1+tP4glqi/rR5Xp+oTS1CXwMEMbo9R+4rgr/+7//u9pWna5zCCGEsDHDd0g6+CfU+LSbPsUGfIqNdNBn34jzT8qxr0+7KS+fhfNy/mk24uvn38bwT79Rtn5yjn3qVjx1ERTvaWyrbuRKttpGPuJUTmhfceRDF7WLX8mSDHBbgtfptOrs2Y1f5a39xbbKqE/IAyrbouol+wnkIA9I8322lVc6gmzaq9+3a30g/QlqE7+UU17SVJ+3QXWTjyA50gUZigPVxa/KKo3yqr+2Sf0mXIdWeyXH0yRT9uRX+YC8SgshhBA2ZXRm8I8//hh+Fxeco08++eTo0qVLwz6wvKd00sTjx4+He6sWF6ohz4cffjjEP3nyZPj96KOPhl/gPj/uM9OMx+Lid/Ts2bNl6nrcunXrvaViWFw8h2U6uH379lCflu0ISkNnwUwYs41A2xYX/UG2lkLZF+jsv2ofdgC3DXUhA9yWY7Tq7NltrL9YNpdO6pN9gJ70u7afPn06bM+FWS/ao7bRzipDS8v0mdv5iy++WKs/fRyon9yGQF3Y//fffx/2wfta+ekT8kmO+tlnDsX9+/eH8SrYp53M8L1+/Xo1q4ksZL5582bY1zGnmUD6XLYOIYQQNmXrewa13LUJut9v4ZSugi6iU1Cvli8JLGdykR5bMpvjBKm853UHdhewDPjZZ58t99ZnG7ux5OmO0j7ZxG44QzhA3jY5fJuwi/6UgzcGfVLz9eyMQ+c6uG43btwYHENBXnfq79y5c3T37t2hXR9//PEyNoQQQticjZ1BLkY4YcxOcMHeBC5mY87bGKrXAxff+lSx8/bt20lHSBf0Fy9eDL+i3oO2CXKcuc9RM0+bsInddA8caGZr3+DYzXGkHJ9t2wW76E9sPeVA0iea/XZaM8DU3ZoBR1cCjr7+yWEG0Wf/mOHlnyDag+MYQgghbMvGzqCWxebOSLW4evXqMAviS2nMmk2BY9OaFWHWpL5OxmdZSCdMwcXYX3fCMt7NmzeXe5uDHJYouaBvwyZ2e/DgwdAuLaPOpdoZZ4VZMFD9zHLKOfUlXfriu+++G7ZxgHAOQQ57yzG/cuXK4KzWhzy2YZP+1LIw7cLWU32GY+Z6M0ahtYxL3Th0yoM90BGwKTbXPzit/iIvtzys62iHEEIITRYXnC6Li/VwAzuBm9W1TVhc+I7te16FWsb3yQ8tubC44A37lYUztcpLWcGN9Ir3tFZ+j6OehVP7Xh6PIz9IJ4/TPsHbQvtqPTXd07QtuaKX5rII2G2sv9w+nu+f//znaps2V7wc274v3YgTbjf1Jbg+Kke6x6t+r4Pg8sHTW3ae25+1nGDf86t+7RNcb3SAqrdo9aHXrXZDlUGQfIHdJCeEEELYlg/4s7jghBCWsDy7cMpO5eEMZgu5h9Bn/eosI7OPzERmZjCEEMIu2PoBkhDC7mg9WMQSvzumrYdVQgghhE2JMxiCoQdLcMo0I3eSPFy+MkcPkBB4LQ66aF/3YYYQQgi7IMvEIYQQQggHTGYGQwghhBAOmDiDIYQQQggHzFrOoO5bmgPvZ9M9TlMv+CUd2Twl2XunHB/z93fP7RLq3PZddidFtcM+7eJQh/pzrD7ekzfV36fNrnTc1Pa7tNGcYyeEEEIYY7YziHO3zifU+E7sy5cvhxfn8rvNzfh68W/r28PbwkWUFwCfB7jY+0u0gU+1+YuJ9+EQ0He8TJq+5AGH2g+qk3z+cuezyBwd59iw1RdzOA82CiGEcFjMdgZ5lQXvXpsDzhsOoMCJGHtnG7Mb+j4rn2qr8IUGnJBtncoWOFLfLr/+cNbhay/Xr19f7r0PM07YftfoaxzAlzj8c3HUJ2eaPqafzjJTOs614VRf9Ni1jaaOnRBCCGGKWc6glgfdKQAcM6Vp2YuLKa/GAH5ZSoOx5UUcTQLff219y5X3quGEIK+nAxdw6dJzGJnNUZ7WMp3SPU1LgUpTewT7kknbBTIoJ3k9PSmjfeF5CC63gmzSqYsZJ2arKEO8yiNPMr1tTqsvgfbxWTnJdV0oo74mzfvX5VG38D7ojQfZXPnA7eR9wD51qb0u0/vGy/SoNlR5dAbt1/5QvOpQG6WLdEPPFj3bywb8SofK1LETQgghTMKrZcZYXOzf+6SXIE2Qpk9k8bks8vG7LXyeS/Uj3+tUPQTF63NkFeUVbEuuPhemfWQR558RI00y0Am8zfqMGPlkJ+nU01P1ANuSRZy2a5u9TtXj9vG2s6+8oLoq0l0gw+uscp1aFl3YV35+pQNp2pZN+HXc5oI8Xj+6IYtf5SWP6gb0UhtUl/pNZVvUtqKP2011A/lcLttqn7cbxur3bS8nPapOIYQQwi4ZnRlkBoSZCn2kn4/jC9IWF9nVjAazKdxXtmv4+oLq5xNc1KkZFl+61tIlH/lvQd5Fe5d772YtncUFf1XPnTt3jp48ebJaClxcjIc0ZAhmu2gzOgHLf8i4f//+cB8f8pEDPT2xLXWAzwgRp/sAmfHpoXp6oNuc+yyxMboL6nc7rws6oRvQdmZ2Adswy8h4kd4vXrwYfoXbXHCbgGbrCOj27NmzlS2xLfX4zBj9oXTvt12DntSl7XWPgbHjiG36gLEge4YQQgi7ZtQZfP369XLrfUjjoo2DpbCPCxb3o+lCKQeiLhWvg5bkuAD3mLPc9vbt2+HXHY2eI7opLDtu87ABuvEgDw4HgS9ZtMCxdd335TzhnD18+PDYmJEDPgbOJM6ql5MTPQXtpr9PgjHHvcfYcYSteGgL/WlHCCGEsA8m7xnU7EqFC18vbVdwAWTWxy+UzL5s+lQxF1Vm65AzNqP25s2bYzN1LXTTfp0924UjpXvFmInFIdgGHEBm5HA6NINVQefWjJbauCuw6dg/GD1wVP2+wznoHklmEOnvk4C2rdv/Y8cRjjK6Mwb4p2BdG4QQQghzGHUGtSyrG+E1I8dF/cqVK8fSoHeT+6bgxFQHhiUz6l13poT8vnxZYVlY4IT5kngLLvrM6Ny8eXMZczQ4qb3Zt3VADg5Az3nr0ZqZ0nKplmpboDNLkrIpv7RtjmOzjsP4+eefD8vEcmpwpOf049WrV4/pB1NjjaVl2jB3BlFUG7ojir6MPRwz/RPgTjRt03eDsZ1sLr2Z5asO3dhxpAdScArH/nkJIYQQtuLPCRZOyXDzO2FxcR1+BTfEK43Avucn/LS8GZ5tvxF/jMXFcVWeOp3FRXGVdu3atdU2oepa8bySQxlAN6XRjhpHO7xu5fE4yZKdFOftUZy2yVvrIWhf8vmtcqs+Xo90Abalbw/XCbnC6yRQR8V1VD62vR3qe2+r1yOqLYTrR0APr6+V7vvK++OPP67iWmOk2tD3yY8c4oXbx22DvRWvNhHneqp+z6t84LJlC5Wf6s8QQghhLh/wZ3FxCRcYZprWnSELIYQQwmEw6z2D4fzCsuSuH2wJIYQQwsUhzuAFRQ+h8DSxf64uhBBCCMHJMnEIIYQQwgGTmcEQQgghhAMmzmAIIYQQwgETZ7AD71LknrtW0HvgyDPnPXlnBXSdepn2uvCACjap78+bA+/R8/frHRJnqe37GBenAeOwvgR+Uy6CTThPTb2Pcxu2OfZ7rHNO3WV/i/N2Tp/DeTjP1uMNffWe1V1wHo/nkx6LF94ZZFBtcrKiI7idknD9+rtv5bL98uXLQR4Dle3zAifNdT5tR/45A/Hyhi9D5iLFi6QPkdNue+1bXmq9768J7ZtdnOjlOK17rDj7dL7WhddJ7fOVUpse+z3WPadyPl73xfwVzuVylM7bOb1HvebxoYWz/BBhPd44N/EC/2047+e40xiLF94Z3HRQ8bWMFnxZgjQOsF2fDPcJJ82Ha3zajs/2zWHTActFCif7EDntts/t2/PEtid6Lp58Bx3WPVaEyzgEdn2xOo1z6t27d5dbp1P/PtjWkTpp6vGG48bkyzac93PcaYzFSWeQExzT8QoO/40rHk9caHoT75Y0/besvPLYJdvrIA1Z2idN8B+P4iWzJ0P7gFHJrzj/j6HH2H9SNa22E7wNc2ctkONtBHTVPumiZ2NQXC0zF2Qzc8V/a2wTkEWb3M7O27dvV/Gyr/YpI3uM2YI2qIz+WwfVj9w5fec2JAjisYfsU9vjecdAT4L08vYRvI3eplbbpYv3k/Tn1/u1NxZa7aIu8ktHyWHf+xZUHpSfAGqX0qWby5yiZ2OX5fHIJShdekKvTEW2kqwxaKNOvJShDuH9Sjugtoe6xmT0cBnKL9lqp3RX3xLmHANAGcnVWKjt8X3Vxa/XI108T8XzqJzGkuymdI8D1deT3UJ2khy1VbI0XscgD847zhNlHNnbdVKdCnOQXaSb6NnU+8PHvWTwq7aprUqTfqBrHiDH+5Ly7Kuso3Z78L6aw5idNCYItHUuPXu16qKOsXOcynhZ2Qe8rlb6HKoMMWV/9am38cT4cwKyLP4DHLYXA2z1WSy2+TQW+Cey/BNa7OtTW+QHymtb+QigNGQAv/qMGXK0DZLfkwHoTbz03xT0ULsd6pEe0kVIB+iVd/wzbAK9ZQtAJrYmTnm9bvKzXdtPHFDW9RpDdQmXo/pAdUiuxgLxSuNXaV6/24U0bXs54qVH1amH10kZ5FKOeILaAa4PfeBjrAWyJEd1QKtdaodgW/qrD6UL22q/+px9bVPO62CbtFa7JJuADtJDdVNW2yqveoB0t5H0Iq51DE4hPQDZkteKJxBP8HzStycLSHMbAOXINwXlKC+qXfhVXdhA2/xKfpUxBuVkS9UF/BLczuRV++fWQXnyUVZlCN4e1UF6tZG3T2XAddO4Un94m4B46Y0MzyeZvi07qMwYag8y1VaVlV60awrqVlsBO6hstbXbiDq9rT1a7USul1V90luwTXukk8oD8dJb5fj1baCM5BDYJqh+ZEtO1dHlrIOXc/lsowNIF+rRvtt3jr20PbcuyWNbASijutVuwbbkrAPlql7SgzBlf+XdpO5NGT2roIiUc6rBgHzeQDUCg7hhiJfhx9IAeZLJL3k9YLx15G8KbVOHOb12EicdFVp2rNR62K5y5tjYIb53wI3h8sHleP+36lbZmlbr9/ayTV4P5F9H5xZeB7/eD2qHhzl1+bgEdKxyvB7hNnW9wHWjvGwtan7pjm1ru8DrAte5ptXy7Htd2qa8t5Hg+VpQT9Wthbev2rfqK6pN0Ed265Xp4WMaKIsMUXUSnq/KmIuX6x1PxHlQO8dw+0y1hzTViWzJJ97tyDb1Q9WVX69DdStfDcCvykOtr0ete2wsjFHLef1eh/rIg7e1B/mqHti9ykIHQmuMQZWD3lUGele7gLeJOigrvP2tMTHHhg71uHwh+znkU32Uc3u6nj179eoCbzO4vGojr7vqQd2yz6ZQr2S4HuBprhPUNuyb0WXi169fD/fIVd68efPeenYr3y5hSndhNEbTKpzVm2Kx26KTj+nKPQDr8urVq6PFYDwmZ+4N4ZqGPg18eWMu3PO1GPjH2sq9IwRfWpiLlgDGHtTQOPY6N7n3bKq/tfSzONCXMe/z0UcfLbeOBjt89tlnQxktT6CX5/nwww+XW/NY55OEN27cOLp3796wje5Xr14dtjc5BnvnEDGnnypzyty/f39YJiIfSzP7gOUev/F9E1gOoq/HYNwsLqbH7L7tgxOVW7duHT169GjY/v3331fyqfvSpUvDNvh2hX7mWK3Lf9xCAq4/gX48L2x6rmgdy73jiPP93OO0d77chm+++eboyZMny713fb/ueWYfPkPPXlPnlk3AhrRbY5NfP++uA2XnnttU32ky6gxiBB8cghOCG0zs8xu4dDoHy3kAu23iVFSw57qDRAPw2bNnw0FzGtD2sYtGCy4iHNwtcKxoCw6X7vsYA1l8hk9lemgcb8tYf9MX3MyMLvVk6PiJjRMS+TnZ42zQp7SJPq3MPRmuc6FBpi7q7hhscgz2ziEwt5+cuWXQmTw4UdwTts79SVPofqDbt28PfbQJ+mft2rVrg45jMG64mO4T/wfAoW7GgDM2jm/evHn04MGDYczqIiqHoncuk7N4ltn0XNE6lnvHEcfn06dPl3vjcBz0zpebwjGDXMYlAZ3nnl/EPnyGnr3Gzi3bgOPJGMcG1L2Jk73JuQ1O81gYdQavXLkydKD+owH+k2XQYCy/yZEbcTmh7AtOmtShkzqDauo//nUH8q6Q3Vy/TW4IZUaG/yqq/cfgv3sG37avlGAwO35B4mQPnufFixfDL7oSzxiR/TXAma3BLi2Hjie0uWjrZEE/I0sBuPhOQTnqmOOMy8lxm27ST73+Rm/s1psV9hM/bf/uu++GbdmHk5AuvFxkffzTz8waj+HOox+ftW9b6KLubHIM9s4h6/STWKeMbEgfzzkZrzMDgtPEhVLjR6wjg2OBi86cCw0XFZ+BxJbqg13Bscq4oD81EwzMGPpxyZggrgftYYwwPtU2ZNfrBdvE0zeMNaAO+pe27rp9PeZeIzY9V7SO5d5xpPO9t71XR+98Obc9LdCBvsCBIcwZm5V9+Aw9e/XqgjnnuBbUgeMpG2xyHUXGuue2s3As0OBRtNavwL5oxS8atIpbnDCP5fH9xcAYTVucKFf7uq/A4wjg+1UGSB9+F8YdtpEzB+VXQIYYayflxuzWwu+LcP1assfq/vXXX4/ty84//vjjKs7b0cNtXfelq3A7VdneLrZrvxBoA3he5SPN81LXFGqz5PgvwXWsfSxdelQ7iF5/e5x0UB29dnm8jwWvW23w/vd2UZfL8Xa5nF57ZJc6bj0/YQ4927T6yQP61Da0ylSd0N3HkrdrDMmrdVQb+b6XcRlTdXq/eX0eemPC+7mH5/djv9UeQbza4bRsWY8bh7pb7ff81CUUR90EH6stat2un9qg/Tp+K71+8HgCddZ6p/QE7wdvs/cBQbTq7bXH200eoTr59bL1GlDtVusmkA+Q43WM0TveoRXv9VKf20bjqGevXl2ev8rTNsHrpn21jxVcprenh9td2x7Xsj9on7yEOWNsV3zAn0XlIYQLBP8Zszy9yX/3IYTDgxm2er4gjhk4Zq58+6LCjBwrWT7rTxwrX7INs5Jn9XmFbbjwL50OIYQQQh8cHr8VQXCrCc4f/1xq+yLDLRkVXkyOI4gTyH2EF9ERhMwMhnDB4D4lPcH28OHDzA6GECbB2eE+ROfly5cX3gF0uD+vPt1/KOfQOIMhhBBCCAdMlolDCCGEEA6YOIMhhBBCCAfMqTqDrM9zQ+Yu4YmnTd8xtCncfKsXdU7VTbq/F2nfSDd+QwghhBAqp+YM4pxMfYZpDjiU7lxxo+c6L3vcBTyBxI223H7Jr78kknZyYy5wYz/pJwk3/6LXId0EHEIIIYT5nJoziHPyfOIzTHPgXWqnCc6eO3g4Xv6OIh5LF3yJ4vLyLfQhhBBCCGeBSWdQy4wKDkueivfZMC2FMhNGWv2kjsr4Ny8lCzlep2jpQRleocH7kdgGZuH0CSCV8bI+i6j3BnnoLeG22kpeOXf8ql7BPp/R4XF9yjkt2yBXdag9jtK9rS4DfUhjn3j2VUaQTvwm9YcQQgjhAsKrZcYgy8uX7z6LxOdR9NkU/1SKPunCZ1r4zArb2idUGSqnvEJlQOVElTGmhz6hw7YCUIb80JIvOZVeWwGd2JduFX1eRyBL5asO0g1qOUH9no9P2hAkiyBdpZvqmNM3old/CCGEEC4WozODzCAtnILV/Wbci8fbt5lBWjgQqxcx8ks+PmKupdCFQzIsl/rH2zVzpXK3b98efqfo6VFB7sKBWe69e2Gm/3700UfDL7x58+bYku3CoTp6/fr1cu8vxtq6KdU2zFzSRurRzBwznk+fPh3S54A85NIm6VqX4qf6Zpv6QwghhHA+GXUGcY5aDx5URwpa+SotZ2sOPT22AYcJ5wdHDPh1Z1Fs2tZ1oY04mQsHfRVw3k6K064/hBBCCKfDqDOIc/TkyZPl3l9cunTpmCMlPv744+VWn02e9O3psS3MIuLoMROGg6cZNWebtq4DbTzpp6Cd064/hBBCCKfDqDN45cqVwRFiCVHwwAFLjDhR/vABD0rcuHFjudeGdOTpVSt6gEQPKyCTmTjQMixpPT1AZdeF5d9Xr16tZsF+/vnnZcpxNm2rmDuLqDbKNuB1CjmnAl0IXm4T5tYfQgghhAvGwhEaxR9MILAvWvHX7SEFHmTwPAtn41ic8goeWFAaD0V4Wk8PL9Mqr+D1Lpy7Yw9XeOjheVR3bR/1V2q92m7ZptfGistpPUBCOtS4qb7p1U85yQwhhBDCxeID/iwu/AcHy75v374dZv4EcS9evGguF4cQQgghXEQm3zN4UeGrIRVeEB1HMIQQQgiHxME6g/fv3x8+h6dXqRCuXbu2TA0hhBBCOAwOdpk4hBBCCCEc8MxgCCGEEEKIMxhCCCGEcNDEGVzy5ZdfDu8xJLDdgvfu9d69xz2HvLvwrMMT0+iql2jX/X3COyF1f+ZYfdh/2/cm7ptd6LiN7efUz3jUezj5PQ/jM4QQwskTZ3AN9ILnyqYvvj4NLjc+rcdto3o5Ng6Dv9x7V+BE37lzZ6iLz949evRomfIOOdn88l3ks8yUjnNtWPtiLufBRiGEEM4PcQaX4AzxdQ/4/PPPh1+Hi/vDhw9X2855+owbXxkZA4dtH/jnBPnm8ffff7/cOzo2w8WXYHAWzzJTOs614VRf9Jhrow8//PDYPyr+Ts0QQghBTDqDLEexlIUDxEWbWQlf6tNSly46pBPI6/FC8gjuVGnZS/LZB8q7HMqwTz4gnjjJVTzbBK+DvEqv8K1hLp44hHynt/Ls2bPhHYRchNnuoXrl4PhSYEsnjycIZpfYlx1lG0K1bW07KI9CD9UDyGPG6euvvx62VVZ94bq0oH6VUR61D8cHuWw7lPnhhx+GGVfSyC8kT/UL4hQ8v1Cd0ld2kZ0I6gN+aavsQJBMyallxqg2lB2R7/IcXn5e69A+ZaQbsnq0bM8/OJrxbb1XM4QQQhjg1TI9+ByZPrG2cIJW2/z6p9f4HBoQh0iC0tnm02fgMvTpM9L803Hs82k0tv1zaNoG1Ue8ypEumfp0GnXt4jNq6CO99Rm3ivQA6aFfT3Od1E5+QXZQPEH18lvj2EaWt12yfJs8KuN1ej2CvOovYF9yQLavEK80yVWdUOU6XhbUrz4WpANp2kY++xW1yWWSV/W3+kdy+FU5ftUGfmmDIJ+3z6ltVV2gOkFtk1zKsE98bTdpvfpJ03YtF0IIIUzxvldj+IXL4ULjFya/6LLt+7owti5S5POLsF9cyasLKOUlkzj2hV94ax3Eu56bgl6uN3W4DuD6gtozphPpar+Q7Jbte7aFWo/jtq35aj0uEyinstKrBTI8jTJu+yrXqe3qjQXpXkOlZQvqr+WQiU6uZ9VF1HxVR6e2VXUBv+xDT0/K1rSx+tkmrwevP4QQQhhjdJmYe4wWF8ZjS1bA0hPLTixpEb755pshfgyWwkDLVsDS7BxYnuWeM+p/8ODBiX8y7t69e8PN/rIDjC0Vg7ezx6tXr97LRz27QsuS2zxswL19LOMCbW7dd6ZxwTK7aC21b4vG0GLcHgtzWDhVRwtH7Fi5uffQsbzMsu9JMLYU3IN7VhfO37G25bOKIYQQ5jJ5zyA3q3NxwSn0+45wAPmk2+vXr2ddVOUocP+TM8dpglu3bg1PoM51IHcF+uopWAUuvK2nih0cpCmHiLb4gxVCD7JsA04F/YW+17d8IIO+x+nv2V59+OLFi+FXbOLYjKExJOdzHXCy37x5s9ybh+45vH379urhoX2DY7du/2NnjsMQQghhE0adQRwh3Yz+3XffDb8CB5ALF7Nbc8BhwCm5efPmMubdjNucWUW4cePGMEO1jxmnMXAE6ywL+zgXso34/fffh1/shm2mZmdoEzNWkiNHee6MVQ/kIHeTp5xbDhx9xMzY1atXlzHvg8Pos2f8o+B9PcZcB58xhN31QAj49hg4xq4fzq3s3YPxiRO4bn9UG7ojysw2eB450ehEPPXJwdZsKPakT+vDNMDT7xwbcpJpF7JCCCGEWfw5Avc3+f1Iuu9JcF+Sx3EPk/Jy35WXJS8sLozvxZFXcbrXTPsun3xOla/tuo884Jcyc6nlhdfraWNxhJZOtM/zAPeJ1bgp22qbQHnka1/bC4dhFUfwuqWP1+NU27dwnXQ/W21Ly/5VD233xoL2lafi6dQvXD+2a73Vxr4vPfittq94ubqvsS7cPtU2yqty1A2t+j2v8oUQQghz+IA/iwvIRjArwzLySXHS9YV3MOPEEr2/GzCEEEIIF4ONXzqNg3CS9+/NfVAl7B4cQZa0QwghhHDxWNsZ1E313IN1EjNFelFw70nWsD+4f01PT8990CeEEEII54utlolDCCGEEML5ZuNl4hBCCCGEcP6JMxhCCCGEcMDEGVzCvYk8pELQu9y4X47Qevee4Aln5dvkZcg9eFec7tdbF/TdxXvmqH/qXXxz2ZVOJ0Ft9y7tMIbuj52qjzE39/2Kp8WudNzU9tvUTznujabesWM/hBAuCnEGR+B2yuvXrw8v++05MnyJ5PLly0PeXT1kgVP52WefLffWA4cCfXcBbdrFQzu71GnftC7+bgf6pr5sfBcgk5dHU9e33367ejm1kGNDvqmv35w2UzrOteGmjth5sFEIIZwl4gwuwZHTZ8C4KAviuTjzBYgKDiJpu4Y6nz9/vtxbj8ePHw/O6VniLOrUY+qrLXfv3l1u7ZanT58ut959AtLfp8k406wzT/DvY8ztkikd59pwky/owLY24pVZ+tKRf4IzhBAuKpPOoJauuCDxHzczFFrK4gJFYFv/xWt5hrweL3wpzGfbiHf57APlXQ5l2NdMCfHESa7i2SZ4HeRVeoULAN++xSGsn7zjU3y//PLLe8tVOIjXrl1b7v3FVBuVJtsht5UflEf5hJepNq54nbX9rquC9yu/Qn1BqLYQnqdla6V7muQpDX0dySNIH+mntiGPbdqjMUIA0tgmTXgegrfTcTtQnhknPv1GnPTlFySz1W7o9QP7jC/JdV0ow2f0SCfN7S55ql9Ir5rfIU3lZRfZSWmgcSY7EFym4ggqM0a1Ifv8atxLVu0PxasOtRFdXLceaivBbe/1SzbHP+cBzgcn+S7VEEI4Nf4cgc916XNffAJL2/xqGxb/hQ+//tktpbOtT2a5DH0KjDTKqxz7+kSXPj9GOW2D6vPPcpEumfocF3Vt+2kur0vbQF3oSvA65rRRaF/lycc+7a9tobzqV5qgTteBbWQBupAuKCc5niaZ1A1s+z75JJPyLlOQ3pOHTtonsE1+xStNNhBsIwvQV/lURm1ROY9DtmyhMpJFvLbRWWXA80km5YG86l9gX3nB5Thj/QBVrlPLUo7yyu/6kqZt2bcim7tM8kqebPXrr78OvwTJQb7KkV9tqP1GvNIqta3qI1DdsrfqB9rFtveNtpUmvH7SXBeVI0iPMfuHEMJF56+zZ4N6ghWcqP0i4ydaPwmDTvT1JA/k04Wlnoz9RE95ySROFw4Yu5AQ73pugtfrshXvdazbxlZ+tafannKSg0zVD5Ije7lNfBvYltwqh7yS4bppex28PFQ9vG7X3dtNfso57GOLKh/cRjA2ppxqB89X66ky0VFlKeNtdMb6Aapcp7ar6uuykVmD20gQ77ZAfi2HTO8PqLqImq/q6NS2uv7V3mxXPVXW08bq55c0D8ioZUII4VAZXSbmpvnFiXS1vKKlG+5p414allYIcz4T9/bt2+HXH7KYuwTz1VdfHT158mSonxvr2T9psMXiojV8mq3HNm0UdcmvBXZwuV5fZXFRXd0LCb5Nv2FXQV6Wxiq0i7bPheW2qfxz7PL69evmEuguYVxv87DB3LE51g+7QscndS2O7VUYGx+C+/MWDtmxcnOPM5ZdN33gaV3mtKWCXRbO37G2cV9h7/wWQgiHxuQ9g9zIzsmTk6bfTI0jwT1zXLDnPHEqJ6PewzT35H7r1q3BETvNe3ju3Lkz3OuEs9NygLdtI3BRnnIUkOcPHIiWI4dT9vvvvy/33iFHjX7z+8twBlq6IhcHY+piqfvLgPxjvHr1arIvuXeLe+Uq9Z7OTdC9Yhrb2zBnbI71w65Q3+mfknVgHHAsr4PukeS+2ecbPvC0LozBdfsfuzDeWvTObyGEcEiMOoNc3HVTNQ9RODgSOC69k2yFE/L169ePbt68uYw5Orp3796sWUW4cePG4IjtwhHYFM2U4Ii1HOBN2/jixYvhV07klHONPJwk3XTPL/W2HDkcFewmR47ZK+KAvkVXzZb0ZoKQi+PiT4G2HpJANhfV3jer+b40oAuzcb18QvqoLsrR7rkzVj0kZ8phbdGy8ZyxOdYPU6wz5uv4o49V5xg8Qe/6MRY1vnrwzyAzbuv2R7WhO6KaeXdHWU40OnHOUX3kefPmzbCNPaE1c4yzynjT8UUbscvY+S2EEA6KhRPQhXtquEeHbATdnyN0T5HgPhzlXTgFx8rqnqDFCfy9OPIqDhmex+WTz6nytV33kQf8UmYuVR6gn7a9fsLCuRji57QRyM++yyFO8QreFunfah+0bOd1uw1dhgLya/3C49QGh/qU7nq07CKqrbStPFUXyapx1UZTY8ptrjQvQ/AHKAjg9UgXcLv26PWD4gjoUnEbeDuQ4e1AN6h5KtUWwvUjT7V9tbHv1zq13Ron1Ya+r7JuW6VRh4PsWg5a9Xte5avjwG0RQgiHxAf8WZwIN4IZG38f2r456fr2DTMUiwvcMEPVmnHaN8z81Fkd4q5cuXIq+pxnLtrYDCGEcDhs/NJpHJmTvH8PJ2XuknKYhv7j/XUVlnLjCK5HxmYIIYTzzNrOIPfYcNM4N1tP3fO1C3SjP07KnAdVzhPMCupX92qdFDh8P/3002BbD7l3aj4XeWyGEEI4HLZaJg4hhBBCCOebjZeJQwghhBDC+SfOYAghhBDCAXOizqDuNyTo/V4V7p1THgL3ZQme2FT8f/3XfzXzVHRfl/Jpm9C7T8/r4V1k7E/hL2+mHmS32oisGu92cZ3cFq33pwFprXpUjqD3qwHbih+zm/JMvWsuhBBCCOcc7hk8CfROL+C9YvWdYRXSyVdBjseTj2a03hGm96SpXpiqu6brPWQ9VAfvNhPoUuNA8f7uNeJUn2QJ15s8vq+8VR6QT3WrTuEyiK860n7iW7YPIYQQwsXjxGYG/VNcvNuOLwnsAp5qXjg4qy8QOHzNYOHsLPfmwZOh/lmqx48fj8pgpo/6/R1zPFm6sO17M498zo68DnEE4Alf0pmNo6zL5GsP/mk28lJHlQfYlq8ugD5RhzxmBWmPWDh8x75NTB5eN8N7D7f9ykcIIYQQzgejzqAvfeIoaJlVS4dsa4lTS46+xKrlS9L41BXOjMoT2K4O0ybcvn17+NxUlcVn49Z9F6I+XeXLo+6Uqb1AfbTJP//luOOFXXxfUJ6XPAs+C6Z3/fn7/lrfHe6BM4tNgE/d4cwiq77+pH4DGR153czly5eP9V8IIYQQLi6jziAzTMw8EXAmcHqYTWLWSPew4XAxQ4WjgwOF88U+s0v61ilpOBnIIY3yBLbd4dkUnBwcGH3XFNBFDtE6oBfOEzNkLWeVdyuiN+ibwtWpqjAjp5k6p+cIt+Lfvn07e5ZTziv641j2vozBd139+7g4pupP+tm/VRtCCCGEi8nkMjGOHE4Cs3s4FjhLoGVed7hYysSBwAnBOQM5TPuGpVbqFtu8CBjnCWcWaMfYgxZzQDfZbVOQsc4LobXU3ZoxFfSdXhyuh0w0e4m+1cEOIYQQwsVj1j2DOEY4hK3ZLQcHkRklZpYUNnWCek/P9lA9zAji/Gz7qTxmLNH/+fPnQ9t92VhoRpAZth6UozwOMoFtnFYczN6saI3XLOfcWVSWe+kr9Gc2UY65wxKw3y/YYt0+CCGEEML5Y5YzyCwTTh5Lp2PLhjgPr1+/Xu5tT0sWD6L0lmVZisZpunv37tGNGzeWsfPAgQKcJG8js4ssb7d00fI0M6ItcPhwUt05RhZ6agaOfZ89ZZnWnW5m7Nad5WQ2UDZilhMdNfMHcmzduZR8zwcfffTRciuEEEIIF5FJZxAHjxk/nBo9XNCDhx/8PjMci9aM2hy4PxFZ1Ym5d+9e1zHCAdTy7txZNKAOdwD9aWLimc27evXqMuY4esq3LiVjt969eg7tlDOJHsjSLCd1szzscuY81EEf6elqZGATPYBCHTiXWh4mXX3ELKKebCaf+j2EEEIIF5g/R7i+fMfew4cPj73XjrBwOFbbvMtOLByKY3lAchSQhUy2vWxFeRQkD6o+kkP92nYdCVUPDwtHdyjDb5WNHoJ04sjjeP5WukAH1SUk08vVttd08HjaLXr6ez0eHLdRrw0hhBBCuDh8wJ/FhT+EEEIIIRwgs+4ZDCGEEEIIF5M4gyGEEEIIB0ycwRBCCCGEAybOYAghhBDCARNnMIQQQgjhgJl0BnkH3Ul+iYL33ulrHQS9v4/33nn8//zP/xzbV9DLo3vwnj7lpS6HskprQXx9z5/r1fpsHfZTOkFgU48nIEu4nmP2J33uuxxrHYK6WrqfRU5SV8bD1Hi6CGxzjPfGVAghhPPDpDPIS4f1HeKTgJdF87ab6+VLHbxoWm/B4RNx//d//3csH9vE8/WN3gWcixYvrSbvy5cvj71AmwsizqHk+MVRDmoLXtJMGQIvjK5184JnpRMAPfgUnMeji14MDa9evVqlTdl/zldf5DDqZdSCeP+m81nmJHXF6WQsXXQYi3xZaBNO8p/EEEII++NCLRPjMOIc9r65i+OmL2zgdJJXThKfsSNA/TSbO6gO6XJW4eHDh8fqJv2bb75Z7v0FTp9/IYV8XFgVh07ffffdsD2HOZ+MwynFaa4Ojr4scx44SV35QgtfZLnoMNYZt5twkv8khhBC2B+TzmBdlsNp0dKkHKkxfLnTZ9coy35dqt2W6mRRh5w6Zu6uXLkybAOfz8NJQgdmCv0zd3ySju8gj+H5oX4zmVm4zz777Fi7wXUE8vFZOsHn6ZgpnDPzQr7et5oF7fv4449X32uWPSreV0L9r/6qbfGldfIKLbFqvGh2VY6v8iteaUJxyjcH9CSvdKrLyS5T4056qkzL5sisaapL8sDbojhQvKf3jp1eG9xmjuIJ1X7SDxnUR17vR9lAUL6V5u1ErzGkN6HqqngF9EE37YN0ULtDCCGcAH+OoE+iXb9+fbWvT5vxqTdtj0F5fdaMMj+VT7H1oE7KtoI+Nwfkk0ziSfdPswl9os0/sUY5yqucg4wqx+tqgT08XXUhp+rtYBfXS9uygadVKNuTK9DJZdZ2kU490p08BPU/QWW8D91GyJcMySOoXsp5HLKJIwByqBMoL7nSQZCmfA75JZ8y0ke2oYx0kQzXkzKAPqpbMskHkud1CdWnOpQHlFf7yFO7nV4byMt2tQXx0ltp6Ec827ITcewTvG3SQWWVX7YB2Uq4DGBfNiZeadJdcqpMbz9l1A5w+SGEEPbPX1eWDn4x4ITtJ/F18QvCFL28fvEB8hGnIF0r9WINapsulo5f2MSU/mO2wXYt3ai7pzNQ39jFkbLephZeXhd+R3YQbo+axrZsQB7vC+Jlg5b9PH+1ea1HzM0HrhuoPvV9DVD19DFe04iX01LraulFHeSvY8/rqPTaAG6LahegrPT1clDrdJ1qmrfb2wxse71eT62Tdkiuy1HdgjKyHWne/hBCCPtnrXsGuWfLl6XmouUxlmn3weLiwZVlCH4Pn1OXZgXx/uCGw9LqXFgS692rCNiu1f66RFxhaXdMLvTaBiy7cZ+g+kwPC/iSYqVnD0fLiJ53zr2L68CSI8vs2/L27dvhV2NEocXUknsPHvip/bBwgJZbu+fNmzfvyR8bB5W5eRfO2TGb9OwzNR5u3bo13JIB9Ifr7vfoPnr0aHU7QwghhJNh7QdIcLa4kC7+k591Xw/OI/ffqcxpQv0vXrxY7h0dPX369OjatWvDhZGLkztIOGBXr15d7o2j+7+mLrCt9lOP38fYAvv14L7HMXA23QFSP9Snip16sW6htro9gf7eFt3XRt88f/58Gbs5clDqPXItcLI2aQP/OLSc9k2dyymQi6NW2zT3HxiVmxqzjIN672xrbEyNBx7I0T8lOPjVVvxDpDE5pVMIIYTdspYzyEVajo+evB0D54oL1ll56pALDg9nALoxU8eMHTBzoYuR2lgfEGmBHGY8uNgBF1mVd5jlqjajLBfLsYsf5caeLFa9LXrOD3pwYfZ0n7X0p67H+Pbbb4+9lgTbjs1yzgU5zPaqb7YF++LA+MMPvu2OyaZtYDaLsa6HJvSPxZwxtAnIrW2iT8dm1dx5ZAaO/puC44LX+agcxwhxLcbGA/84Ur/+Ialjnr6uT7qHEEI4IRYn5i7cu0MWAvcQcc/P9et/3aO3OLkvc/ZZXLBW+bWNXN175PcYATKVn0B9QD6P/+///u9j+8rnqIzf8+RtqvrTRqVVFE8gH7gsD8LjXAchmzq1/dU+jvKiR0X2relVvtK8LdLJZWBft4/K+XhoyZKtfBy43LqPPN/3ci25ourm5WRD7SsPUM7bILm1Ls/jQXKgjlHhcbVtzlgbqs2Ex6mdrbZTnnilqW7XmbjabnC9FAeKI+hYao0HcBmtdGC/HpMhhBD2zwf8WZyYQzhINLP2888/D78XFWarmRE+rVl66q8zvcyi+sw2fXHR+yGEEM4iF+ql0yGEsweOoG7PEPUWBpbVuUc0hBDCybO1M8g9b3pStYb60tkQzhKMT+5TI/i9dxcNHC3u5Xv58uVOHvBZF80I+rmBh6KYFcTu7HOP6q7uEQ0hhLAeWSYOIYQQQjhgms4g/6mHEEIIIYTTZ9/zdpkZDCGEEEI4YPIASQghhBDCARNnMIQQQgjhgIkzGEIIIYRwwMQZDCGEEEI4YOIMhhBCCCEcMHEGQwghhBAOmDiDIYQQQggHTJzBEEIIIYQDJs5gCCGEEMIBE2cwhBBCCOGAiTMYQgghhHDAxBkMIYQQQjhg4gyGEEIIIRwwcQZDCCGEEA6YOIMhhBBCCAdMnMEQQgghhAMmzmAIIYQQwgETZzCEEEII4YCJMxhCCCGEcLAcHf1/n8w1AMnSthEAAAAASUVORK5CYII=\"\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eIn GPT-3.5, we created the following prompt for the translation of terms, as it is necessary to specify the intention: \u003cem\u003ethe following terms are used in the Human Phenotype Ontology. Translate them into German. Ensure that the translation has a scientific and medical context.\u003c/em\u003e The terms and synonyms to be translated were subsequently imported into GPT-3.5. GPT-3.5 was used via a web application [\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eThe translations made by DeepL in the pre-study are used again in this study as a reference translation for comparison with GPT-3.5 [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e]. In the previous study, six medical experts rated the terms. Three medical experts from this study were also involved in the pre-study. The translations generated by DeepL are evaluated again in this study to determine whether the results are consistent with those of the previous study. DeepL was used via the application programming interface (API) [\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eWe invited ten medical experts (with medical degrees and several years of clinical or medical research experience) who were fluent in both English and German to evaluate the translations. The experts were blinded to the source of the translations and were not told which translation software was used for each translation, both to reduce the risk of bias. Each translation was rated on a 4-point Likert scale: strongly agree = 1, agree = 2, disagree = 3, strongly disagree = 4. We asked the experts to consider the approval for the respective translation. In addition, medical experts can make comments on individual term or synonym translations.\u003c/p\u003e\n \u003cp\u003eTo answer the question of whether the commercial translators examined are suitable for translating medical terminology, the trend of the experts' ratings was explored, and an error analysis was conducted. The study was conducted between March 2023 and April 2024.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n \u003ch2\u003e2.2. Data analysis\u003c/h2\u003e\n \u003cp\u003eWe compared the performance of GPT-3.5 and DeepL on the basis of the evaluations provided by medical experts. We calculated the mean and standard deviation (SD) of the Likert ratings (LRs) for each translation software and analyzed the results via statistical tests. The synonyms were always rated together, and the results were then combined with the LR of the main term, resulting in an average LR for each term.\u003c/p\u003e\n \u003cp\u003eWhether the mean values of the average LR differed significantly between the two systems was measured via the Mann‒Whitney U test. This is a nonparametric statistical test used to compare two independent groups of ordinal scaled variables [\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eOn the basis of the 20 common HPO terms, further research was conducted regarding the quality of the translations. We examined interrater reliability via the intraclass correlation coefficient (ICC) and Fleiss's kappa. The ICC is used to assess the agreement or reliability of ratings from different raters, with values ranging from 0 (no agreement) to 1 (perfect agreement). The ICC is not a hypothesis test but a measure of the consistency of ratings [\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e]. Fleiss' kappa also measures the reliability of multiple raters and considers both the observed agreement and the agreement that would be expected by chance, providing a measure of interrater reliability that accounts for the possibility of random agreement. It can take values between − 1 and 1, with 1 indicating perfect agreement, 0 indicating agreement no better than chance, and negative values indicating disagreement beyond chance [\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eTo evaluate the quality of the translations, we calculated the Jaro–Winkler similarity between the two translation systems themselves and the similarity to a reference translation of the Health Terminology/Ontology Portal (HeTOP). HeTOP is a comprehensive medical terminology database with translations for diverse medical and clinical applications [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e]. Notably, HeTOP does not contain any official German translations for the HPO. For comparison, English HPO terms were searched for in HeTOP and, if available, a German translation was extracted. For the sake of simplicity, the synonyms were not included in this analysis.\u003c/p\u003e\n \u003cp\u003eThe Jaro–Winkler similarity is a string metric that measures the edit distance between two sequences. It uses a prefix scale that rates strings that match from the beginning more favorably. The metric ranges from 0 (indicating no similarity) to 1 (representing identical strings) [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e]. Using the Jaro–Winkler similarity metric with a threshold of 0.6, we evaluated the degree of similarity between the machine-generated translations and the HeTOP reference translations. The purpose of this threshold was to focus on significant similarities and ignore instances with low similarity. Any similarity below this value was set to 0.\u003c/p\u003e\n \u003cp\u003eTo assess the impact of term length on translation quality, we categorized the terms into four groups on the basis of their length: terms consisting of 1 word, 2–3 words, 4–7 words, and terms longer than 7 words. LRs were calculated separately for each group to assess differences in translation accuracy.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"3. RESULTS","content":"\u003cp\u003eAs shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e2\u003c/span\u003e, the average LR for the 100 randomly selected HPO terms and synonyms was 1.36 (SD\u0026thinsp;=\u0026thinsp;0.65) for GPT-3.5 and 1.28 (SD\u0026thinsp;=\u0026thinsp;0.56) for DeepL. For the 20 common terms, the average LR was 1.22 (SD\u0026thinsp;=\u0026thinsp;0.44) for GPT-3.5 and 1.46 (SD\u0026thinsp;=\u0026thinsp;0.64) for DeepL. In the pre-study, the LR at DeepL for the same 100 random terms was 1.23, and for the 20 common terms, it was 1.28 [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. There was no distinct difference between the ratings of this study and those of the pre-study. It can therefore be assumed that the ratings for the translations remain consistent across different experts.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAverage Likert ratings (LRs) and standard deviations (SDs) for the translated HPO terms and synonyms.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHPO terms\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePre-Study: DeepL\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDeepL\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eGPT-3.5\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e100 HPO terms\u003c/p\u003e \u003cp\u003e20 HPO terms\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.23; SD\u0026thinsp;=\u0026thinsp;0.50\u003c/p\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.28; SD\u0026thinsp;=\u0026thinsp;0.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.28; SD\u0026thinsp;=\u0026thinsp;0.56\u003c/p\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.46; SD\u0026thinsp;=\u0026thinsp;0.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.36; SD\u0026thinsp;=\u0026thinsp;0.65\u003c/p\u003e \u003cp\u003eLR\u0026thinsp;=\u0026thinsp;1.22; SD\u0026thinsp;=\u0026thinsp;0.44\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe Mann‒Whitney U test was conducted to assess potential differences in the ratings between DeepL and GPT-3.5, encompassing the 100 randomly selected terms (p\u0026thinsp;=\u0026thinsp;0.27) and the 20 commonly used terms (p\u0026thinsp;=\u0026thinsp;0.06). The null hypothesis for this test states that both groups have the same mean. As the obtained p values are not less than the predefined significance level of 0.05, we are unable to reject the null hypothesis. Consequently, there is insufficient evidence to assert that a significant difference exists in the true mean between the two groups. However, it is worth noting that there is a uniformly positive trend in the ratings for both translation tools.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo assess the quality of the translations, we conducted further research focusing on the 20 common terms. Analyzing the ratings assigned to these translations, we observed a consistent pattern of data homogeneity, as depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. This homogeneity was manifested by limited variability among the ratings, leading to both the ICC and Fleiss' kappa yielding low values for all these assessments, as shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAverage scores for the Fleiss' kappa coefficient and the intraclass correlation coefficient (ICC) for the ratings of the translations of the 20 common HPO terms by GPT-3.5 and DeepL. The scores indicate the interrater reliability of the expert ratings.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGPT-3.5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDeepL\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eICC\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.36\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eFleiss\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.20\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.17\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAmong the 20 common terms, 15 (75%) had corresponding reference translations in the HeTOP database. After selecting the threshold, the calculated similarities for ratio comparisons yield values of 0.70 for GPT-3.5 versus HeTOP, 0.76 for DeepL versus HeTOP, and 0.76 for GPT versus DeepL, as shown in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eA noteworthy observation made by several experts pertained to the technicality of translations produced by DeepL in contrast to those generated by GPT-3.5. Another finding was that GPT-3.5 sometimes produced fewer synonyms in the German translation compared to the source text. Upon closer examination, this reduction seems to occur when several synonyms lead to the same German term. Furthermore, when synonyms are available in both singular and plural forms in English, GPT-3.5 often omits the plural in the translation.\u003c/p\u003e \u003cp\u003eAmong the 120 HPO terms, 20 consisted of a single word (17%), 55 terms consisted of 2\u0026ndash;3 words (46%), 36 terms consisted of 4\u0026ndash;7 words (30%), and 9 terms contained more than 7 words (8%). Notably, among the 20 common HPO terms, 13 out of the 20 terms consisted of a single word, representing 65% of these terms. LR performance varies only slightly among categories: single words score 1.40 (GPT) and 1.14 (DeepL), 2\u0026ndash;3 word terms score 1.31 (GPT) and 1.30 (DeepL), 4\u0026ndash;7 word terms score 1.37 (GPT) and 1.24 (DeepL), and terms with more than 7 words score 1.49 (GPT) and 1.29 (DeepL).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eJaro\u0026ndash;Winkler similarities for the 20 common HPO terms. The similarities between the translations of GPT-3.5 and DeepL, as well as the respective comparisons to reference translations from HeTOP, were determined.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHPO terms\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGPT-3.5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDeepL\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eHeTOP\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eGPT/HeTOP\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eDeepL/HeTOP\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eGPT/DeepL\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHeadache\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eKopfschmerz\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eKopfschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eKopfschmerz\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.97\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.97\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParesthesia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePar\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePar\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePar\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFatigue\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eM\u0026uuml;digkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM\u0026uuml;digkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eErm\u0026uuml;dung\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVertigo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSchwindel\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eVertigo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eVertigo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHypoesthesia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eHypo\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eHypoesthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eHyp\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.87\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.91\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBack pain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eR\u0026uuml;ckenschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eR\u0026uuml;ckenschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eR\u0026uuml;ckenschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eArthralgia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eArthralgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eArthralgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eArthralgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMyalgia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMyalgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMyalgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMyalgie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAbdominal pain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBauchschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eUnterleibs-\u003c/p\u003e \u003cp\u003eschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAdominal-schmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiarrhea\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDurchfall\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDiarrh\u0026ouml;e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDiarrhoe\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSchmerz\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSchmerz\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSchmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNausea\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026Uuml;belkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026Uuml;belkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNausea\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAsthenia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAsthenie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAsthenia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAsthenie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.95\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWeight loss\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGewichtsverlust\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eGewichtsverlust\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eGewichtsverlust\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiminished ability to concentrate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVerminderte\u003c/p\u003e \u003cp\u003eKonzentrations-\u003c/p\u003e \u003cp\u003ef\u0026auml;higkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eVerminderte\u003c/p\u003e \u003cp\u003eKonzentrations-\u003c/p\u003e \u003cp\u003ef\u0026auml;higkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003en/a\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiminished physical functioning\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVerminderte k\u0026ouml;rperliche\u003c/p\u003e \u003cp\u003eFunktion\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eEingeschr\u0026auml;nkte k\u0026ouml;rperliche\u003c/p\u003e \u003cp\u003eLeistungsf\u0026auml;higkeit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003en/a\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.66\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDysesthesia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDys\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDys\u0026auml;sthesie\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003en/a\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePalpitations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eHerzklopfen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eHerzklopfen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003en/a\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChest pain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBrustschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSchmerzen in der Brust\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBrustschmerzen\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNight sweats\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNachtschwei\u0026szlig;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN\u0026auml;chtliche Schwei\u0026szlig;ausbr\u0026uuml;che\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003en/a\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.67\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eJaro mean\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e0.70\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e0.76\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e0.76\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"4. DISCUSSION","content":"\u003cp\u003eThe ratings provided by the experts for both GPT-3.5 and DeepL indicated that, on average, there were favorable assessments for translation quality. However, the subjectivity of the evaluation and the lack of explicit evaluation criteria, such as completeness, comprehensibility, or technicality, pose challenges. Despite these limitations, the ratings remained consistent with those from a pre-study, suggesting consistency in expert evaluations and a common understanding of accurate translations.\u003c/p\u003e \u003cp\u003eStatistical analysis via the Mann‒Whitney U test revealed that there were no significant differences in the mean ratings between GPT-3.5 and DeepL for both the 100 randomly selected terms and the 20 common terms. This suggests that both machine translators performed comparably in terms of translation quality.\u003c/p\u003e \u003cp\u003eHigh ratings for terms with more than 7 words, such as \"Elevated proportion of CD4-negative, CD8-negative, alpha-beta regulatory T cells\" or \"Anomalous insertion of papillary muscle directly into anterior mitral leaflet\", show promising results even for more complex terms. DeepL is slightly ahead here. The observation that the multi-word translations exhibit comparable performance to their shorter counterparts may be attributed to the incorporation of additional contextual information.\u003c/p\u003e \u003cp\u003eIn assessing interrater reliability, the study revealed data homogeneity among the ratings for the 20 common terms. This resulted in low values for both the ICC and Fleiss's kappa, indicating that traditional measures of interrater reliability may not be suitable in such cases of minimal variance and uniform ratings. In addition, our analysis revealed instances where the same expert rated identical German translations produced by both translators differently, indicating some degree of inconsistency in rating assignment (intrarater reliability).\u003c/p\u003e \u003cp\u003eTo validate translation quality, an independent reference translation from the HeTOP database was employed for 15 of the 20 common terms. The Jaro\u0026ndash;Winkler similarity metric revealed high similarities between the machine-generated translations and the HeTOP reference translations. However, it is essential to acknowledge potential nuances, as the threshold may exclude moderately similar yet semantically relevant translations. In the cases where the similarity of the terms only slightly exceeds the predefined threshold, as in the comparison between \"Bauchschmerzen\" and \"Abdominalschmerzen\", with a similarity value of 0.62, it becomes clear that the degree of similarity requires careful examination, since the similarity in this case lies solely at the end of the term.\u003c/p\u003e \u003cp\u003eThere are various similarity metrics for measuring text similarity, including the Levenshtein distance, cosine similarity, and Jaccard similarity. However, Jaro‒Winkler stands out because of its ability to weight the prefix (the beginning of words), which is useful for capturing similarities related to singular/plural differences. This allows for a more precise capture of similarities in words, enhancing the detection of semantic similarities. Metrics such as the BLEU (bilingual evaluation under study), which is used in many translation studies [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e], are designed primarily for evaluating machine translations when performing 1-to-1 text comparisons with n-gram decomposition and are not necessarily suitable for direct 1-to-1 string comparisons, such as for our comparison of the 20 common terms with the HeTOP database.\u003c/p\u003e \u003cp\u003eWhen comparing a machine translation to a reference translation, there are several limitations and challenges to consider that can impact the evaluation process. These limitations include subjectivity. Different medical experts may have different interpretations and preferences for how a particular text should be translated.\u003c/p\u003e \u003cp\u003eIt was challenging for some experts to evaluate the quality of the translated synonyms in comparison to their English counterparts. There was a tendency to evaluate the synonyms in relation to the main term. This intricacy is attributable to the specific study design and could be mitigated through the adoption of a randomized presentation format for the translations under evaluation.\u003c/p\u003e \u003cp\u003eChallenges in interpretation occurred in cases of spelling errors in translations, such as \"Hypoesthesie\" instead of \"Hypo\u0026auml;sthesie\" in DeepL. Experts have also observed instances where English synonyms are inaccurately associated with specific terms. For example, in the case of the term \"fractured facial bone\", which was one of the 100 randomly selected terms, an English synonym stored as \"bone facial bone\" was identified that appeared to be mislabeled and that may be more appropriately labeled \"broken facial bone\". Since this made it difficult to evaluate the quality of the translated synonym, the rating for this synonym was removed from the overall rating.\u003c/p\u003e \u003cp\u003eGPT-3.5 has several limitations, such as the risk of generating incorrect or biased translations. Providing additional details and contexts through the prompt in GPT-3.5 could improve the accuracy and quality of the translation, especially in regard to medical terminology information, e.g., providing information that many terms might have their roots in Latin or Greek. However, we acknowledge that the optimization of language models such as GPT-3.5 falls under the domain of prompt engineering and that simply adding more information does not guarantee improved results.\u003c/p\u003e \u003cp\u003eOne possible approach to improve translation quality is to combine translations from multiple translation engines and select different translation candidates from them. This can even be done on the basis of different input languages and support languages [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn addition to analyzing the translation quality of commonly used HPO terms and the influence of term length, an alternative approach could involve a range of medical experts in the selection of HPO terms on the basis of their significance, difficulty of translation, phenotype, and frequency of use. However, this approach is not without limitations. It is conceivable that medical experts without expertise in translation may subjectively assess the complexity of terms, which could lead to inconsistencies in the selection process.\u003c/p\u003e \u003cp\u003eThe generalizability of our results to other languages must be viewed critically. In this study, the focus is clearly on translating terminology into German and investigating how well an automated process performs. For validation purposes, it was important to us that medical experts were fluent in both the source language and the target language. However, DeepL has more than 30 source and target languages and can therefore be used for many languages, including French, Korean and Spanish [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. The GPT models also include various languages in their training data.\u003c/p\u003e \u003cp\u003eThe accelerated development of large language models has led to the introduction of newer GPT models during and following the course of this study. These models are anticipated to introduce novel innovations and enhancements [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. To facilitate the transfer of the study findings to the translation performance of the current models, the 20 common terms were retranslated with the GPT-4o model. This resulted in translations that were identical except for a few instances of singular/plural differences and minor adjustments to the translations of \"Diminished physical functioning\" and \"Hypoesthesia.\" The Jaro\u0026ndash;Winkler similarity between GPT-3.5 and GPT-4o was 0.99, whereas the similarity between DeepL and GPT-4o was identical to the similarity between DeepL and GPT-3.5, which was 0.76. These values indicate comparable results, thereby demonstrating uniform validity.\u003c/p\u003e \u003cp\u003eNotably, our study revealed that 75% of the common HPO terms had German reference translations in the HeTOP database. Given the limited sample size, the results are not yet statistically significant. However, given the paucity of studies on extensive translations, these findings underscore the incomplete coverage of translated medical terminology and highlight the importance of our study, particularly for the documentation and diagnosis of rare diseases where precise distinctions in disease characteristics are vital.\u003c/p\u003e"},{"header":"5. CONCLUSIONS","content":"\u003cp\u003eIn our evaluation of machine translation of the HPO, GPT and DeepL emerged as viable choices. DeepL provides very good translations in the eyes of medical experts, which are notably rather technical. GPT has significant challenges, including unpredictable ambiguities in output, such as sporadic synonym reduction. The presence of arbitrariness and inconsistency in translation, especially within medical terminology, could be a nonnegligible problem in contexts such as clinical practice and automated analysis.\u003c/p\u003e \u003cp\u003eAs language models continue to evolve, the choice between machine translators should be made with consideration of their respective strengths. For the initial translation of extensive terminologies such as the HPO, translators such as DeepL show great promise but require additional manual annotation and validation by medical experts. However, the information gain that could be obtained by translating these extensive terminologies, especially for the purpose of describing more complex cases such as rare diseases, should not be neglected.\u003c/p\u003e \u003cp\u003eThe complete translation of the HPO into German with DeepL is provided by the corresponding author as an 'OBO' file in a repository. In addition, an interactive script was created to perform search queries in the German HPO.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eAPI\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eapplication programming interface\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eGPT\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003egenerative pretrained transformer\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eHeTOP\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ehealth terminology/ontology portal\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eHPO\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ehuman phenotype ontology\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eICC\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eintraclass correlation coefficient\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eLR\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eLikert rating\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eNLP\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003enatural language processing\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eOBO\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eopen biomedical ontologies\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003estandard deviation\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eUMLS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eunified medical language system\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe complete translation of HPO (v2024--01--16) with DeepL can be found at https://github.com/RichardNoll/HPO_German. The German terms, definitions, and synonyms for a list of HPO codes can also be output via an attached script in Python. It is also possible to identify corresponding HPO codes by entering terms and synonyms in German. The script can be run interactively via a Jupyter notebook in Google Colab.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study is part of the SATURN project and is funded by the Federal Ministry of Health in Germany (Reference: 2520DAT02B).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was designed by RN, and the manuscript was written by the same author. HS and JS provided research support and advice. The remaining authors are clinical experts who answered medical questions, evaluated the translations, and suggested adaptations to the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe author was supported by medical experts from the University Hospital Frankfurt. We thank all the participants for their participation.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eIvanović M, Budimac Z. An overview of ontologies and data resources in medical domains. Expert Syst Appl. 2014;41:5158\u0026ndash;66.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNoll R, Frischen LS, Boeker M, Storf H, Schaaf J. Machine translation of standardized medical terminology using natural language processing: A scoping review. N Biotechnol. 2023;77:120\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.nbt.2023.08.004\u003c/span\u003e\u003cspan address=\"10.1016/j.nbt.2023.08.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeepL. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.deepl.com/en/whydeepl/\u003c/span\u003e\u003cspan address=\"https://www.deepl.com/en/whydeepl/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 12 October 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNoll R, Berger A, Facchinello C, et al. Translation of Ontological Concepts from English into German Using Commercial Translation Software and Expert Evaluation. Stud Health Technol Inf. 2024;310:89\u0026ndash;93. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3233/SHTI230933\u003c/span\u003e\u003cspan address=\"10.3233/SHTI230933\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDale R. GPT-3: What\u0026rsquo;s it good for? Nat Lang Eng. 2021;27(1):113\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1017/S1351324920000601\u003c/span\u003e\u003cspan address=\"10.1017/S1351324920000601\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eK\u0026ouml;hler S, Gargano M, Matentzoglu N, et al. The Human Phenotype Ontology in 2021. Nucleic Acids Res. 2021;49(D1):D1207\u0026ndash;17. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/nar/gkaa1043\u003c/span\u003e\u003cspan address=\"10.1093/nar/gkaa1043\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGargano MA, Matentzoglu N, Coleman B, et al. The Human Phenotype Ontology in 2024: phenotypes around the world. Nucleic Acids Res. 2024;52(D1):D1333\u0026ndash;46. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/nar/gkad1005\u003c/span\u003e\u003cspan address=\"10.1093/nar/gkad1005\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRobinson P, K\u0026ouml;hler S et al. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/obophenotype/human-phenotype-ontology\u003c/span\u003e\u003cspan address=\"https://github.com/obophenotype/human-phenotype-ontology\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 26 July 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKoubaa A. GPT-4 vs. GPT-3.5: A Concise Showdown. Preprints. 2023. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.20944/preprints202303.0422.v1\u003c/span\u003e\u003cspan address=\"10.20944/preprints202303.0422.v1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHPO. https://hpo.jax.org/. Accessed 12 July 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChatGPT. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://chat.openai.com/\u003c/span\u003e\u003cspan address=\"https://chat.openai.com/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 12 July 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeepL API. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.deepl.com/en/docs-api/\u003c/span\u003e\u003cspan address=\"https://www.deepl.com/en/docs-api/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 12 October 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. 1947;18:50\u0026ndash;60. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1214/aoms/1177730491\u003c/span\u003e\u003cspan address=\"10.1214/aoms/1177730491\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKoch GG. Intraclass correlation coefficient. In: Kotz S, Johnson NL, editors. Encyclopedia of statistical sciences. New York: John Wiley \u0026amp; Sons, Inc.; 1982. pp. 212\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNelson KP, Edwards D. Measures of agreement between many raters for ordinal classifications. Stat Med. 2015;34(23):3116\u0026ndash;32. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/sim.6546\u003c/span\u003e\u003cspan address=\"10.1002/sim.6546\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHeTOP. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.hetop.eu/hetop/\u003c/span\u003e\u003cspan address=\"https://www.hetop.eu/hetop/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 12 July 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJaro MA. Advances in Record-Linkage Methodology as Applied to Matching the 1985 Census of Tampa, Florida. J Am Stat Assoc. 1989;84(406):414\u0026ndash;20. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2307/2289924\u003c/span\u003e\u003cspan address=\"10.2307/2289924\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrunotto A, Schulz S, Boeker M. Automatic Generation of German Translation Candidates for SNOMED CT Textual Descriptions. Stud Health Technol Inf. 2021;281:178\u0026ndash;82. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3233/SHTI210144\u003c/span\u003e\u003cspan address=\"10.3233/SHTI210144\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOpenAI. https://openai.com/gpt-4. Accessed 12 July 2024.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Artificial Intelligence, Controlled Vocabulary, Translations, GPT","lastPublishedDoi":"10.21203/rs.3.rs-4836251/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4836251/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eThis paper presents a comparative study of two state-of-the-art language models, OpenAI's GPT and DeepL, in the context of terminology translation within the medical domain.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eThis study was conducted on the Human Phenotype Ontology (HPO), which is used in medical research and diagnosis. Medical experts assess the performance of both models on a set of 120 translated HPO terms, employing a 4-point Likert scale (strongly agree\u0026thinsp;=\u0026thinsp;1, agree\u0026thinsp;=\u0026thinsp;2, disagree\u0026thinsp;=\u0026thinsp;3, strongly disagree\u0026thinsp;=\u0026thinsp;4). An independent reference translation from the HeTOP database was used to validate the quality of the translation.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eThe average Likert rating for the 120 selected HPO terms was 1.29 for GPT-3.5 and 1.37 for DeepL. The comparison with HeTOP revealed a high degree of similarity between the machine translations and the reference translations.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThe results indicate that both GPT and DeepL are effective at translating HPO terms from English to German. Statistical analysis revealed no significant differences in the mean ratings between the two models, confirming their comparable performance in terms of translation quality. The study not only illustrates the potential of machine translation but also shows incomplete coverage of translated medical terminology. This underscores the relevance of this study for cross-lingual medical research. However, the evaluation methods need to be further refined, and specific translation issues need to be addressed.\u003c/p\u003e","manuscriptTitle":"Assessing GPT and DeepL for Terminology Translation in the Medical Domain: A Comparative Study on the Human Phenotype Ontology","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-28 14:23:49","doi":"10.21203/rs.3.rs-4836251/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-04-10T08:29:24+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-09T22:33:26+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"191492289274557348711845070986750299340","date":"2025-03-25T22:31:50+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-20T23:08:30+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"217829620154068370092056064934045734763","date":"2024-08-16T03:27:35+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-08-15T20:17:38+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-08-01T11:02:24+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-08-01T11:01:45+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2024-07-31T14:14:02+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ea83cef1-7f3a-499b-8df0-5af33f6b12c8","owner":[],"postedDate":"August 28th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-07-07T16:13:25+00:00","versionOfRecord":{"articleIdentity":"rs-4836251","link":"https://doi.org/10.1186/s12911-025-03075-8","journal":{"identity":"bmc-medical-informatics-and-decision-making","isVorOnly":false,"title":"BMC Medical Informatics and Decision Making"},"publishedOn":"2025-07-01 15:58:43","publishedOnDateReadable":"July 1st, 2025"},"versionCreatedAt":"2024-08-28 14:23:49","video":"","vorDoi":"10.1186/s12911-025-03075-8","vorDoiUrl":"https://doi.org/10.1186/s12911-025-03075-8","workflowStages":[]},"version":"v1","identity":"rs-4836251","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4836251","identity":"rs-4836251","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}