Is ChatGPT's Knowledge on Rhinology Accurate? Can It Be Utilized in Medical Education and Patient Information?

Is ChatGPT's Knowledge on Rhinology Accurate? Can It Be Utilized in Medical Education and Patient Information? | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Systematic Review Is ChatGPT's Knowledge on Rhinology Accurate? Can It Be Utilized in Medical Education and Patient Information? Aykut ÖZDOĞAN, Burçay TELLİOĞLU, Oğuzhan KATAR, Serdar ÖZER This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5788648/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: ChatGPT is a new artificial intelligence model designed to create human-like chat. As a result of advancing knowledge and technological improvements, it is promising in the field of medicine, especially as a resource that patients and clinicians can apply. Objective: The aim of our study is to measure the accuracy and consistency of ChatGPT's answers to questions in the field of rhinology. Methods: In March 2024, the ChatGPT (ChatGPT version 4) was presented with 130 questions in rhinology. Each question was asked to ChatGPT twice and the consistency/reproducibility of the answers was investigated. The answers were evaluated by three ENT physicians. The physicians followed a standardised 4-point format (1:Completely correct, 2:Partially correct, 3:A mix of accurate and inaccurate/misleading, 4:Completely incorrect/ irrelevant). Results: The answers given by ChatGPT were consistent at a rate of 91.5%(119/130). Among the inconsistent answers, the second answer was found to be more correct in 10/11. Statistically, the second answer was found to be more correct (p: 0011). In 130 questions, as a result of the controllers' evaluation, the number of answers evaluated as completely correct was 99/81/80(76.2%/62.3%/61.5%) respectively. However, completely incorrect answers were 7/6/7(5.4%/4.6%/5.4%), respectively. Accordingly, it is seen that there is no statistical difference between the controllers(p:0.270). Conclusion: The inaccuracy of ChatGPT in patient information and education process is considered to be at an acceptable level and reliable. However, it is also seen that ChatGPT answers are not completely correct and can give misleading answers to some questions. We believe that it would be safer and more accurate to use ChatGPT as an informative and educational material for patients with the control of experts. ChatGPT 4 rhinology artificial intelligence chatbot machine learning head and neck Introduction Chat-Generative Pre-trained Transformer (ChatGPT) is a technology based on an Artificial Intelligence(AI) language model developed by OpenAI Corporation (California, USA)( 1 ). This model, the first version of which was published in November 2022, firstly builds its database from books, articles and websites in active use published until 2021, but now it is updated and is now searching the literature until October 2023( 2 – 6 ). Studies on AI in the health sector have gained momentum in the field of patient education and support for clinicians in the decision-making process. Facilitating patients' access to fast and accurate information on issues such as what their health problems are, how to treat them, how the treatment process will work and what realistic benefits can be obtained from the treatment is seen as a potential area where AI can provide serious benefits to the health sector. In addition, as AI technology develops, it is thought that it is possible to develop automated diagnosis-treatment mechanisms. There are opposing opinions about the use of ChatGPT in the medical field. In addition to the authors who emphasise the positive contribution of the studies, there are also authors who mention the misinformation and ethical problems that may occur( 7 – 9 ). In particular, misleading patients/patient relatives and physicians in the training process with misinformation and unwanted situations that may occur afterwards carry the risk of a significant health problem. Recently, many publications have discussed the reliability of information provided to patients by AI( 10 – 14 ). With all these studies, we think that further studies should be continued in order to provide clearer results and opinions. Despite the growing interest in data generated by AI, important insights into the quality of information in various medical fields are still missing. Our study aims to evaluate the accuracy of ChatGPT's answers to questions that patients and ENT trainees may ask about the field of rhinology. Materıals and Methods Since our study did not require access to patient information, ethics committee approval was not required. The investigators have no financial connection with the ChatGPT software development process and OpenAI company. a. Study Design In March 2024, 130 questions on rhinology were asked to ChatGPT (Chat GPT version 4). All questions were asked in English, each time a new chat was created and asked twice to assess the reproducibility of the answers. The questions were taken from board exams and reference books. The questions were divided into 11 subheadings. For the standardisation of the evaluation of the answers, the scheme in the study by Kuşcu et al.(15) was used. According to this scheme, the answers were classified as 1: Completely correct, 2: Partialy correct, 3: A mix of accurate and inaccurate/misleading, 4: Completely incorrect/ irrelevant. The answers were first evaluated by three different ENT specialists (one of whom was an ENT professor with experience in rhinology). During these evaluations, it was ensured that the experts did not exchange information with each other and then the results were statistically analysed. The consistency of the two answers and the analysis of the inconsistent answers were evaluated by an experienced ENT professor in the field of Rhinology. In case of inconsistent answers, the first answer was considered valid and analysed. Our main rationale for considering the first answer as correct is that it is the first answer encountered by patients or people in the education process and that it directs people who cannot evaluate the accuracy of the answer. a. Statistical Analysis The analyses of the data included in the study were performed with SPSS (Statistical Program in Social Sciences) 26 programme. Descriptive statistics were calculated as number, percentage, mean, standard deviation, median and minimum-maximum. The consistencies between the measurements were evaluated with the intraclass correlation coefficient(ICC) in the Kruskall Wallis H Test. The cases where the p value was less than 0.05 were considered statistically significant. In the evaluation of ICC values, a correlation below 0.4 was considered as poor, between 0.4-0.59 as moderate, between 0.60-0.74 as good and above 0.75 as excellent. Results ChatGPT was asked 130 questions related to rhinology. The consistency/reproducibility of the two answers was found to be 91.5%. For 11 inconsistent questions, the first answers given by ChatGPT were evaluated. When the inconsistent answers were evaluated by an ENT professor (3rd controller) experienced in rhinology, it was seen that the second answer was more correct in 10 questions. ChatGPT's answers were found to be more correct in repetitive questions (p:0011). (As shown in Table 1 ) Table 1 Evaluation of incompatible answers by the 3rd Controller First answer Second answer P General rhinology Q-9 2 1 0.0011* General rhinology Q-11 2 1 General rhinology Q-13 3 2 Acute Rhinosinusitis Q-8 2 1 Chronic Rhinosinusitis Q-6 2 1 Deviated septum/turbinate hypertrophy Q-4 2 1 Deviated septum/ turbinate hypertrophy Q-8 3 2 Epistaxis Q-6 2 1 Olfaction Q-1 2 1 Nasal skeleton and rhinoplasty Q-10 2 1 Sinosasal tumors Q-11 1 2 Q: Question, Independent Samples t-test, p < 0.05 considered statistically significant The number of answers assessed as completely correct by the controllers was 99/81/80 (76.2%/62.3%/61.5%) respectively. However, the rates of completely incorrect answers were 7/6/7 (5.4%/4.6%/5.4%) respectively. Detailed analysis of answers as shown in Table 2 . Table 2 Evaluation of ChatPT responses by controllers Controllers Completely correct n(%) Partialy correct n(%) A mix of accurate and inaccurate/misleading n(%) Completely incorrect/ irrelevant n(%) p ICC 1 99(76.2) 21(16.2) 3(2.3) 7(5.4) .270 .928 2 81(62.3) 31(23.8) 12(9.2) 6(4.6) 3 80(61.5) 37(28.5) 6(4.6) 7(5.4) ICC: Intraclass Correlation Coefficient, p; Kruskall Wallis H Test, p < 0.05 considered statistically significant A high correlation was observed between the controllers, with no statistically significant difference between controllers(p = 0.270, ICC = 0.928). Sub-headings were then analysed separately. The analysis made according to the sub-headings is shown in Table 3 . Table 3 Analysis of ChatGPT responses according to sub-headings Sub-headings Completely correct n(%) Partialy correct n(%) A mix of accurate and inaccurate/misleading n(%) Completely incorrect/ irrelevant n(%) p ICC General rhinology 1 7(46.7) 4(26.7) 1(6.7) 3(20.0) .895 .962 2 6(40.0) 4(26.7) 3(20.0) 2(13.3) 3 5(33.3) 6(40.0) 1(6.7) 3(20.0) Acute Rhinosinusitis 1 7(70.0) 3(20.0) .677 .746 2 6(60.0) 3(30.0) 1(10.0) 3 5(50.0) 5(50.0) Chronic Rhinosinusitis 1 10(100.0) .163 .750 2 7(70.03) 3(30.0) 3 7(70.03) 3(30.0) Deviated septum/ turbinate hypertrophy 1 8(80.0) 2(20.0) .792 .850 2 7(70.0) 2(20.0) 1(10.0) 3 7(70.0) 2(20.0) 1(10.0) Epistaxis 1 5(62.5) 2(25.0) 1(12.5) .892 .954 2 4(50.0) 3(37.5) 1(12.5) 3 4(50.0) 3(37.5) 1(12.5) Olfaction 1 7(77.8) 1(11.1) 1(11.1) .824 .975 2 6(66.7) 2(22.2) 1(11.1) 3 6(66.7) 2(22.2) 1(11.1) Nasal skeleton and rhinoplasty 1 6(60.0) 2(20.0) 2(20.0) .717 .949 2 4(40.0) 3(30.0) 1(10.0) 2(20.0) 3 4(40.0) 4(40.0) 2(20.0) Congenital diseases 1 9(90.0) 1(10.0) .105 .750 2 5(50.0) 4(40.0) 1(10.0) 3 5(50.0) 5(50.0) Sinosasal tumors 1 12(78.6) 1(7.1) 1(7.1) .843 .873 2 11(78.6) 1(7.1) 2(14.3) 3 11(78.6) 1(7.1) 2(14.3) Noninfectious rhinitis 1 9(100.0) .191 .643 2 6(66.7) 3(33.3) 3 7(77.8) 2(22.2) Other topics 1 19(76.0) 5(20.0) 1(40) .992 .905 2 19(76.0) 3(12.0) 2(8.0) 1(4.0) 3 19(76.0) 4(16.0) 1(4.0) 1(4.0) ICC: Intraclass Correlation Coefficient, p; Kruskall Wallis H Test p < 0.05 considered statistically significant The rates of 3 (A mix of accurate and inaccurate/misleading) and 4 (Completely incorrect/ irrelevant) answers of ChatGPT were found to be 7.7%, 13.8% and 10%, respectively. When the complete accuracy of the answers was evaluated, the rate of 1 (Completely correct) answers was found to be 76.2%, 62.3% and 61.5% in the controllers, respectively. The rates of 1 (completely correct) and 2 (partially correct) answers were 92.3%, 86.2% and 90%, respectively. Discussion AI has started to be used in the field of health as it is used in many fields. Its use in the field of health is seen as the potential benefits of improving patient education, increasing the knowledge of patients about the disease, helping professional health workers and utilising it in health education. At the same time, ethical problems are also important limitations of the use of AI in the field of health. For these reasons, determining the limits of AI together with the studies will maximise the benefit to be obtained. In this study, we investigated the accuracy and reliability of ChatGPT answers related to rhinology. Due to its recent emergence, there is a limited number of studies on the role of ChatGPT in the field of otolaryngology, but new studies are rapidly being conducted and added to the literature. Kuşçu et al. tested the accuracy of ChatGPT by asking 154 questions about head and neck cancers. According to their results, they stated that ChatGPT produced significantly accurate (86.4%) and reproducible information (94.1%) to various medical questions about head and neck cancers. Despite its limitations, it was stated that the source could prove useful for both patients and medical professionals( 15 ). In their study in 2023, Park et al. tested the potential role of ChatGPT in clinical otolaryngology. And they concluded that it has great potential in decision-making processes for clinicians. However, they also emphasized the importance of using the model with caution due to its potential for error and its high dependence on existing pre-trained data( 16 ). Ayoub et al. evaluated the effectiveness of ChatGPT and Google Search as resources for postoperative patient instructions compared to their institution-specific instructions. They showed that ChatGPT scored lower than Google Search and institution-specific instructions in terms of understandability, actionability, and procedure-specific content( 17 ). Chiesa-Estomba et al. stated that ChatGPT shows promise as a tool in the clinical decision-making process, especially for patients considering sialendoscopy( 18 ). Hoch et al. evaluated the accuracy of ChatGPT answers to 2576 (479 multiple-choice and 2097 single-choice) questions in otolaryngology board certification exams. And as a result, they found that 57% of the total questions were answered correctly. They showed in their study that ChatGPT could potentially serve as a complementary tool for preparation for otolaryngology board certification exams( 19 ). Looking at the literature on ChatGPT and rhinology, Riestra-Ayora et al. asked the ChatGPT version (GPT-3.5) 65 questions about the most common pathologies in rhinology. They reported that ChatGPT answers were accurate and reliable. They evaluated the reliability and quality of ChatGPT answers. They also evaluated the number of words in the first and second answer. And they stated that it was able to create simple and easy-to-understand definitions for pathologies, which was useful to facilitate patient understanding( 20 ). Maniaci et al. used AI to optimize decision-making protocols for the use of biologic agents in Chronic Rhinosinusitis with Nasal Polyps (CRSwNP). The place of biologic agents in the treatment of CRSwNP was compared with the preferences of expert physicians in decision-making and biologic drug selection. As a result of the study, they stated that biological agent treatment in CRSwNP management will contribute more in the future and may be involved in the creation of treatment protocols ( 21 ). Patel et al. evaluated the performance of ChatGPT 3.5 and GPT 4 on Otorhinolaryngology (Rhinology) Standardized Board Examination questions (93 questions) in comparison with Otolaryngology residents. It was shown in this study that ChatGPT4 (86% correct answers), which is an advanced AI model, has a much higher success rate than ChatGPT 3.5 (45.2% correct answers). They showed that ChatGPT 3.5 will not pass the American Board of Otorhinolaryngology Written Question Examination (ABOto WQE), while GPT4 has a high chance of passing. As a result, it has been stated that as AI technology advances, it may be possible to use it as support material in medical education, diagnosis and treatment planning( 22 ). Radulesco et al. evaluated the clinical approach of ChatGPT-4 in rhinology in comparison with general practitioners. They found that ChatGPT-4 made a reasonable and accurate primary diagnosis in 62.5% of cases. They also found that the correct choice of additional examinations and treatment was 7.5%-30.0% and 7.5%-32.5%, respectively. They found that the consistency of indication for additional investigations between ChatGPT-4 and the general practitioner could vary significantly from one examination to another, but the stability of ChatGPT-4 responses was moderate to high( 23 ). Workman et al. evaluated the applicability of ChatGPT based on the International Consensus Statement of Allergy and Rhinology: Rhinosinusitis (ICAR-RS). They reported that the ChatGPT performed satisfactorily, correctly answering more than 80% of the questions in all categories( 24 ). Ye F. et al. evaluated the accuracy of ChatGPT in answering 189 questions about allergic rhinitis (AR) and 249 questions about chronic rhinosinusitis (CRS). They found that the ChatGPT had a high rate of conveying accurate information for AR and CRS patients( 25 ). Bellinger et al. asked questions about epistaxis, chronic sinusitis, sinus infection, allergic rhinitis, allergies and treatment of nasal polyps to ChatGPT, ChatGPT Plus, Google Bard and Microsoft Bing chatbots. They reported that ChatGPT Plus and Google Bard are currently the most patient-friendly chatbots for the treatment of common pathologies in rhinology( 26 ). In our study, questions consisting of general information about rhinology and detailed information were directed to ChatGPT to evaluate whether ChatGPT can be a part of the education process and informing patients. If we look at the strengths of our study, it can be said that the accuracy and reproducibility of ChatGPT was evaluated in 130 questions, which reduced the margin of error. In addition, the fact that 3 different ENT specialists answered the answers independently of each other enabled us to get away from errors that may be caused by the person. The answers given by ChatGPT to the same question when asked for the first and second time were found to be 91.5% consistent. Kuşcu et al. also found 94.1% in their study and this result is compatible with our study( 15 ). And the reproducibility of ChatGPT about rhinology was found to be satisfactory. In 11 questions, there was a significant inconsistency between the two answers. In our study, general rhinology was seen as the subheading with the highest number of inconsistent answers with 3 inconsistent answers. When evaluated between the first and second inconsistent answers, the second answer was found to be more accurate than the first. The rates of 3(A mix of accurate and inaccurate/misleading) and 4(Completely incorrect/ irrelevant) answers of ChatGPT in rhinology were found to be 7.7%, 13.8% and 10%, respectively. In the light of this result, the inaccuracy of ChatGPT in patient information and education process is considered to be at an acceptable level and reliable. When the complete correctness of the answers was evaluated, the rate of 1(Completely correct) answers was found to be 76.2%, 62.3% and 61.5% in the controllers, respectively. Kuşçu et al. found a completely correct response rate of 86.4% in their study on head and neck cancers with ChatGPT( 15 ). The reason for the lower rate of correct answers in our study despite similar standardization may be attributed to the high number of questions requiring detailed information and based on interpretation in our study. Hoch et al. evaluated the accuracy of ChatGPT's answers to 2576 (479 multiple-choice and 2097 single-choice) questions in otolaryngology board certification exams. And as a result, they found that 57% of the total questions were answered correctly( 19 ). Patel et al. ChatGPT4 was reported to be 86% correct and was found to be higher than our study( 22 ). Ye F. et al. found the accuracy of ChatGPT to be 80% in a study of 438 questions related to allergic rhinitis (AR) and chronic rhinosinusitis (CRS)( 25 ). The reason for this difference may be related to the number of questions, but it may also be due to the fact that the answers were standardized in more detail in our study, while the answers were evaluated as true/false in other studies. We believe that evaluating the questions similar to the standardization we used in our study, rather than evaluating them as true/false, will give more accurate results and will be more accurate in terms of developing AI. At the same time, we think that the re-evaluation of questions that require clear information and questions that require interpretation after classification will contribute to the literature. Although the rate of giving correct information in ChatGPT answers is at a satisfactory level, it is seen that there are deficiencies in terms of relying on complete correctness. The rates of 1(Completely correct) and 2(partially correct) answers were 92.3%, 86.2% and 90%, respectively. With these results, the rate of ChatGPT answers containing correct information was found to be satisfactorily high. This information increases the possibility of being used as informative and educational material for patients under the control of experts. We think that it may be right to be cautious for now due to the possible risks of using it without expert control. As a result, we think that it is an important resource under expert team control. Conclusıon ChatGPT has proven to be a reliable source of information for patients and clinicians by answering questions in the field of rhinology with a low margin of error and high reproducibility rate. It is indisputable that AI will become an increasingly reliable source with technological improvements and intensified studies on this subject. In addition to being a potential source of information for now, studies are needed to make it a useful tool in the decision-making process of clinicians in diagnosis and treatment in the future. Declarations Acknowledgments: None Conflict of interest: All the authors declare that there is no conflict of interest related to this study. Funding: Source of financial support or funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Ethical approval and informed consent statements: The study does not require ethics committee approval and informed consent. Data availability and material: This article includes all of the data that were gathered or examined during this investigation. Author contribution statement: Preparing, organizing and asking questions to ChatGPT: Aykut ÖZDOĞAN Evaluation of the answers: Burçay TELLİOĞLU, Oğuzhan KATAR, Serdar ÖZER Analyzing and writing up the data: Aykut ÖZDOĞAN, Serdar ÖZER Aykut ÖZDOĞAN 1,* , Burçay TELLİOĞLU 2 , Oğuzhan KATAR 3 , Serdar ÖZER 4 1 Department of Otorhinolaryngology, Ankara Etlik City Hospital, Ankara, Türkiye 2 Elbistan State Hospital, Kahramanmaraş, Türkiye 3 Polatlı Duatepe State Hospital, Ankara, Türkiye 4 Department of Otorhinolaryngology, Hacettepe University, Ankara, Türkiye References Biswas SS. Role of chat GPT in Public Health. Ann Biomed Eng 2023; 51:868–869 Munoz-Zuluaga C, Zhao Z, Wang F, et al. Assessing the accuracy and clinical utility of chatGPT in laboratory medicine. Clin Chem 2023; 69:939–40. The Lancet Digital H. ChatGPT: friend or foe? Lancet Digit Health (2023) 5(3): e102. doi:10.1016/S2589-7500(23)00023-7 van Dis EAM, Bollen J, Zuidema W, et al. ChatGPT: five priorities for research. Nature 2023; 614:224–6. OpenAI. ChatGPT. 2023. https:// openai.com/blog/chatgpt (Accessed March 28, 2023). Exploding Topics. Number of ChatGPT users 2023. https://explodingtopics.com/blog/ chatgpt-users (Accessed March 30, 2023). Else H. Abstracts written by ChatGPT fool scientists. Nature 2023; 613:423. Haupt CE, Marks M. AI-generated medical advice-GPT and beyond. JAMA 2023; 329:1349–50. Jeblick K, Schachtner B, Dexl J, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol 2024;34:2817-2825. Allahqoli L, Ghiasvand MM, Mazidimoradi A, et al. The diagnostic and management performance of the chatGPT in obstetrics and gynecology. Gynecol Obstet Invest 2023; 88:310–3. Samaan JS, Yeo YH, Rajeev N, et al. Assessing the accuracy of responses by the language model chatGPT to questions regarding bariatric surgery. Obes Surg 2023;33:1790–6. Yeo YH, Samaan JS, Ng WH, et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol . 2023;29:721-732. Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6:1169595. De Angelis L, Baglivo F, Arzilli G, et al. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front Public Health 2023;11:1166120 Kuscu O, Pamuk AE, Sütay Süslü N, et al. Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer? Front. Oncol. 2023;13:1256459. Park I, Joshi AS, Javan R. Potential role of ChatGPT in clinical otolaryngology explained by ChatGPT. Am J Otolaryngol 2023; 44:103873. Ayoub NF, Lee YJ, Grimm D, et al. Comparison between chatGPT and google search as sources of postoperative patient instructions. JAMA Otolaryngol Head Neck Surg 2023; 149:556–8. Chiesa-Estomba CM, Lechien JR, Vaira LA, et al. Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol 2024;281:2777 Hoch CC, Wollenberg B, Luers JC, et al. ChatGPT's quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol 2023;280:4271–8. Riestra-Ayora J, Vaduva C, Esteban-Sánchez J, et al. ChatGPT as an information tool in rhinology. Can we trust each other today? Eur Arch Otorhinolaryngol. 2024;281:3253-3259. Maniaci A, Saibene AM, Calvo-Henriquez C, et al. Is generative pre-trained transformer artificial intelligence (Chat-GPT) a reliable tool for guidelines synthesis? A preliminary evaluation for biologic CRSwNP therapy. Eur Arch Otorhinolaryngol. 2024;281:2167-2173. Patel EA, Fleischer L, Filip P, et al. Comparative Performance of ChatGPT 3.5 and GPT4 on Rhinology Standardized Board Examination Questions. OTO Open. 2024;8:e164. Radulesco T, Saibene AM, Michel J, et al. ChatGPT-4 performance in rhinology: A clinical case series. Int Forum Allergy Rhinol. 2024;14:1123-1130. Workman AD, Rathi VK, Lerner DK, et al. Utility of a LangChain and OpenAI GPT-powered chatbot based on the international consensus statement on allergy and rhinology: Rhinosinusitis. Int Forum Allergy Rhinol. 2024;14:1101-1109. Ye F, Zhang H, Luo X, et al. Evaluating ChatGPT's Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngol Head Neck Surg. 2024;171:571-577. Bellinger JR, Kwak MW, Ramos GA, et al. Quantitative Comparison of Chatbots on Common Rhinology Pathologies. Laryngoscope. 2024;134(10):4225-4231. Additional Declarations No competing interests reported. Supplementary Files supplementaryfile.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5788648","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Systematic Review","associatedPublications":[],"authors":[{"id":403195322,"identity":"e69efe8a-6371-49e4-ba20-5b0b1c37fc95","order_by":0,"name":"Aykut ÖZDOĞAN","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+ElEQVRIiWNgGAWjYFACNih9gPnggw8gPjsxWg6AtbAlG84A8ZmJ18JjJs0DYhDSwj8jLU36A4ONPd8BHgNpm1/b5PmYGRg/fMzBrUXiRtoxiQMMacySB9gKjHP7bhu2MTMwS87chseaG+ltQC2H2QwOMG9Izu25zQjUwsbMi0eLPFQLj8EBBoPDlj237QlqMYA47LCEwQEWw2aGH7cTCWoxPPMs2eKMQZqB5GG2ZMbehtvJbcyMzXj9Inc8zfBGRQUwxI43H//x489t2/ntzQc/fMTnfYEEkPMYINHB2AYSYmzAox4I+A8g8/7gVzwKRsEoGAUjEwAAONxQPBm8ZAIAAAAASUVORK5CYII=","orcid":"","institution":"Ministry of Health","correspondingAuthor":true,"prefix":"","firstName":"Aykut","middleName":"","lastName":"ÖZDOĞAN","suffix":""},{"id":403195323,"identity":"288174ee-8d4a-4860-871e-3f87c361a3ae","order_by":1,"name":"Burçay TELLİOĞLU","email":"","orcid":"","institution":"Ministry of Health","correspondingAuthor":false,"prefix":"","firstName":"Burçay","middleName":"","lastName":"TELLİOĞLU","suffix":""},{"id":403195324,"identity":"fd8a8e06-2ac9-4da2-a9a5-39f109128ddf","order_by":2,"name":"Oğuzhan KATAR","email":"","orcid":"","institution":"Ministry of Health","correspondingAuthor":false,"prefix":"","firstName":"Oğuzhan","middleName":"","lastName":"KATAR","suffix":""},{"id":403195325,"identity":"16ae81fa-7105-43fc-b01c-d2238b654cc0","order_by":3,"name":"Serdar ÖZER","email":"","orcid":"","institution":"Hacettepe University Hospital","correspondingAuthor":false,"prefix":"","firstName":"Serdar","middleName":"","lastName":"ÖZER","suffix":""}],"badges":[],"createdAt":"2025-01-08 11:38:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5788648/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5788648/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":74217604,"identity":"0367373e-1c47-4a20-bbce-709ddeb7e466","added_by":"auto","created_at":"2025-01-20 06:07:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":796474,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5788648/v1/6df133a6-97ee-417b-8891-36c509fe4b92.pdf"},{"id":74217272,"identity":"1bc9f068-de80-4240-80d0-ef28ddc0f9fe","added_by":"auto","created_at":"2025-01-20 05:59:07","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":174072,"visible":true,"origin":"","legend":"","description":"","filename":"supplementaryfile.docx","url":"https://assets-eu.researchsquare.com/files/rs-5788648/v1/76bbdaed7bb5ebc3c2cadf3f.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Is ChatGPT's Knowledge on Rhinology Accurate? Can It Be Utilized in Medical Education and Patient Information?","fulltext":[{"header":"Introduction","content":"\u003cp\u003eChat-Generative Pre-trained Transformer (ChatGPT) is a technology based on an Artificial Intelligence(AI) language model developed by OpenAI Corporation (California, USA)(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). This model, the first version of which was published in November 2022, firstly builds its database from books, articles and websites in active use published until 2021, but now it is updated and is now searching the literature until October 2023(\u003cspan additionalcitationids=\"CR3 CR4 CR5\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eStudies on AI in the health sector have gained momentum in the field of patient education and support for clinicians in the decision-making process. Facilitating patients' access to fast and accurate information on issues such as what their health problems are, how to treat them, how the treatment process will work and what realistic benefits can be obtained from the treatment is seen as a potential area where AI can provide serious benefits to the health sector. In addition, as AI technology develops, it is thought that it is possible to develop automated diagnosis-treatment mechanisms.\u003c/p\u003e \u003cp\u003eThere are opposing opinions about the use of ChatGPT in the medical field. In addition to the authors who emphasise the positive contribution of the studies, there are also authors who mention the misinformation and ethical problems that may occur(\u003cspan additionalcitationids=\"CR8\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e). In particular, misleading patients/patient relatives and physicians in the training process with misinformation and unwanted situations that may occur afterwards carry the risk of a significant health problem. Recently, many publications have discussed the reliability of information provided to patients by AI(\u003cspan additionalcitationids=\"CR11 CR12 CR13\" citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). With all these studies, we think that further studies should be continued in order to provide clearer results and opinions.\u003c/p\u003e \u003cp\u003eDespite the growing interest in data generated by AI, important insights into the quality of information in various medical fields are still missing. Our study aims to evaluate the accuracy of ChatGPT's answers to questions that patients and ENT trainees may ask about the field of rhinology.\u003c/p\u003e"},{"header":"Materıals and Methods","content":"\u003cp\u003eSince our study did not require access to patient information, ethics committee approval was not required. The investigators have no financial connection with the ChatGPT software development process and OpenAI company.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ea.\u0026nbsp; \u0026nbsp;Study Design\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn March 2024, 130 questions on rhinology were asked to ChatGPT (Chat GPT version 4). All questions were asked in English, each time a new chat was created and asked twice to assess the reproducibility of the answers. The questions were taken from board exams and reference books. The questions were divided into 11 subheadings. For the standardisation of the evaluation of the answers, the scheme in the study by Kuşcu et al.(15) was used. \u0026nbsp;According to this scheme, the answers were classified as 1: Completely correct, 2: Partialy correct, 3: A mix of accurate and inaccurate/misleading, 4: Completely incorrect/ irrelevant.\u003c/p\u003e\n\u003cp\u003eThe answers were first evaluated by three different ENT specialists (one of whom was an ENT professor with experience in rhinology). During these evaluations, it was ensured that the experts did not exchange information with each other and then the results were statistically analysed. \u0026nbsp;The consistency of the two answers and the analysis of the inconsistent answers were evaluated by an experienced ENT professor in the field of Rhinology. In case of inconsistent answers, the first answer was considered valid and analysed. Our main rationale for considering the first answer as correct is that it is the first answer encountered by patients or people in the education process and that it directs people who cannot evaluate the accuracy of the answer.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ea.\u0026nbsp; \u0026nbsp;Statistical Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe analyses of the data included in the study were performed with SPSS (Statistical Program in Social Sciences) 26 programme. Descriptive statistics were calculated as number, percentage, mean, standard deviation, median and minimum-maximum.\u003c/p\u003e\n\u003cp\u003eThe consistencies between the measurements were evaluated with the intraclass correlation coefficient(ICC) in the Kruskall Wallis H Test. The cases where the p value was less than 0.05 were considered statistically significant. In the evaluation of ICC values, a correlation below 0.4 was considered as poor, between 0.4-0.59 as moderate, between 0.60-0.74 as good and above 0.75 as excellent.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eChatGPT was asked 130 questions related to rhinology. The consistency/reproducibility of the two answers was found to be 91.5%. For 11 inconsistent questions, the first answers given by ChatGPT were evaluated. When the inconsistent answers were evaluated by an ENT professor (3rd controller) experienced in rhinology, it was seen that the second answer was more correct in 10 questions. ChatGPT's answers were found to be more correct in repetitive questions (p:0011). (As shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eEvaluation of incompatible answers by the 3rd Controller\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFirst answer\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSecond answer\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eP\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGeneral rhinology Q-9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\" morerows=\"10\" rowspan=\"11\"\u003e \u003cp\u003e\u003cb\u003e0.0011*\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGeneral rhinology Q-11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGeneral rhinology Q-13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAcute Rhinosinusitis Q-8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChronic Rhinosinusitis Q-6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeviated septum/turbinate hypertrophy Q-4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDeviated septum/ turbinate hypertrophy Q-8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEpistaxis Q-6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOlfaction Q-1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNasal skeleton and rhinoplasty Q-10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSinosasal tumors Q-11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"4\"\u003eQ: Question, Independent Samples t-test, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05 considered statistically significant\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe number of answers assessed as completely correct by the controllers was 99/81/80 (76.2%/62.3%/61.5%) respectively. However, the rates of completely incorrect answers were 7/6/7 (5.4%/4.6%/5.4%) respectively. Detailed analysis of answers as shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eEvaluation of ChatPT responses by controllers\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eControllers\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCompletely correct\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePartialy correct\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eA mix of accurate and inaccurate/misleading\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCompletely incorrect/ irrelevant\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003ep\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eICC\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e99(76.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e21(16.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(2.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e7(5.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.270\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.928\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e81(62.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e31(23.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e12(9.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e6(4.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e80(61.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e37(28.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e6(4.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e7(5.4)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"7\"\u003eICC: Intraclass Correlation Coefficient, p; Kruskall Wallis H Test, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05 considered statistically significant\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eA high correlation was observed between the controllers, with no statistically significant difference between controllers(p\u0026thinsp;=\u0026thinsp;0.270, ICC\u0026thinsp;=\u0026thinsp;0.928).\u003c/p\u003e \u003cp\u003eSub-headings were then analysed separately. The analysis made according to the sub-headings is shown in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAnalysis of ChatGPT responses according to sub-headings\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSub-headings\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCompletely correct\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePartialy correct\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eA mix of accurate and inaccurate/misleading\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eCompletely incorrect/ irrelevant\u003c/p\u003e \u003cp\u003en(%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003ep\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eICC\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eGeneral rhinology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(46.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4(26.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(6.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e3(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.895\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.962\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4(26.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e2(13.3)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5(33.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e6(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(6.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e3(20.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eAcute Rhinosinusitis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(70.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.677\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.746\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(60.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(30.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eChronic Rhinosinusitis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10(100.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.163\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.750\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(70.03)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(30.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(70.03)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(30.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eDeviated septum/ turbinate hypertrophy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8(80.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.792\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.850\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(70.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(70.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eEpistaxis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5(62.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(25.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(12.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.892\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.954\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(37.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(12.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(37.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(12.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eOlfaction\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(77.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1(11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.824\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.975\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(66.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(22.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(66.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(22.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eNasal skeleton and rhinoplasty\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(60.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.717\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.949\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(30.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eCongenital diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9(90.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.105\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.750\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4(40.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5(50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eSinosasal tumors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e12(78.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1(7.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(7.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.843\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.873\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11(78.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1(7.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2(14.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11(78.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1(7.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2(14.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eNoninfectious rhinitis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9(100.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.191\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.643\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6(66.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(33.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7(77.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2(22.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eOther topics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19(76.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5(20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(40)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.992\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e.905\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19(76.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3(12.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2(8.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1(4.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19(76.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4(16.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1(4.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1(4.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003eICC: Intraclass Correlation Coefficient, p; Kruskall Wallis H Test p\u0026thinsp;\u0026lt;\u0026thinsp;0.05 considered statistically significant\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe rates of 3 (A mix of accurate and inaccurate/misleading) and 4 (Completely incorrect/ irrelevant) answers of ChatGPT were found to be 7.7%, 13.8% and 10%, respectively. When the complete accuracy of the answers was evaluated, the rate of 1 (Completely correct) answers was found to be 76.2%, 62.3% and 61.5% in the controllers, respectively. The rates of 1 (completely correct) and 2 (partially correct) answers were 92.3%, 86.2% and 90%, respectively.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eAI has started to be used in the field of health as it is used in many fields. Its use in the field of health is seen as the potential benefits of improving patient education, increasing the knowledge of patients about the disease, helping professional health workers and utilising it in health education. At the same time, ethical problems are also important limitations of the use of AI in the field of health. For these reasons, determining the limits of AI together with the studies will maximise the benefit to be obtained.\u003c/p\u003e \u003cp\u003eIn this study, we investigated the accuracy and reliability of ChatGPT answers related to rhinology. Due to its recent emergence, there is a limited number of studies on the role of ChatGPT in the field of otolaryngology, but new studies are rapidly being conducted and added to the literature.\u003c/p\u003e \u003cp\u003eKuş\u0026ccedil;u et al. tested the accuracy of ChatGPT by asking 154 questions about head and neck cancers. According to their results, they stated that ChatGPT produced significantly accurate (86.4%) and reproducible information (94.1%) to various medical questions about head and neck cancers. Despite its limitations, it was stated that the source could prove useful for both patients and medical professionals(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eIn their study in 2023, Park et al. tested the potential role of ChatGPT in clinical otolaryngology. And they concluded that it has great potential in decision-making processes for clinicians. However, they also emphasized the importance of using the model with caution due to its potential for error and its high dependence on existing pre-trained data(\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). Ayoub et al. evaluated the effectiveness of ChatGPT and Google Search as resources for postoperative patient instructions compared to their institution-specific instructions. They showed that ChatGPT scored lower than Google Search and institution-specific instructions in terms of understandability, actionability, and procedure-specific content(\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eChiesa-Estomba et al. stated that ChatGPT shows promise as a tool in the clinical decision-making process, especially for patients considering sialendoscopy(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). Hoch et al. evaluated the accuracy of ChatGPT answers to 2576 (479 multiple-choice and 2097 single-choice) questions in otolaryngology board certification exams. And as a result, they found that 57% of the total questions were answered correctly. They showed in their study that ChatGPT could potentially serve as a complementary tool for preparation for otolaryngology board certification exams(\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eLooking at the literature on ChatGPT and rhinology, Riestra-Ayora et al. asked the ChatGPT version (GPT-3.5) 65 questions about the most common pathologies in rhinology. They reported that ChatGPT answers were accurate and reliable. They evaluated the reliability and quality of ChatGPT answers. They also evaluated the number of words in the first and second answer. And they stated that it was able to create simple and easy-to-understand definitions for pathologies, which was useful to facilitate patient understanding(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eManiaci et al. used AI to optimize decision-making protocols for the use of biologic agents in Chronic Rhinosinusitis with Nasal Polyps (CRSwNP). The place of biologic agents in the treatment of CRSwNP was compared with the preferences of expert physicians in decision-making and biologic drug selection. As a result of the study, they stated that biological agent treatment in CRSwNP management will contribute more in the future and may be involved in the creation of treatment protocols (\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e).\u003c/p\u003e \u003cp\u003ePatel et al. evaluated the performance of ChatGPT 3.5 and GPT 4 on Otorhinolaryngology (Rhinology) Standardized Board Examination questions (93 questions) in comparison with Otolaryngology residents. It was shown in this study that ChatGPT4 (86% correct answers), which is an advanced AI model, has a much higher success rate than ChatGPT 3.5 (45.2% correct answers). They showed that ChatGPT 3.5 will not pass the American Board of Otorhinolaryngology Written Question Examination (ABOto WQE), while GPT4 has a high chance of passing. As a result, it has been stated that as AI technology advances, it may be possible to use it as support material in medical education, diagnosis and treatment planning(\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eRadulesco et al. evaluated the clinical approach of ChatGPT-4 in rhinology in comparison with general practitioners. They found that ChatGPT-4 made a reasonable and accurate primary diagnosis in 62.5% of cases. They also found that the correct choice of additional examinations and treatment was 7.5%-30.0% and 7.5%-32.5%, respectively. They found that the consistency of indication for additional investigations between ChatGPT-4 and the general practitioner could vary significantly from one examination to another, but the stability of ChatGPT-4 responses was moderate to high(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eWorkman et al. evaluated the applicability of ChatGPT based on the International Consensus Statement of Allergy and Rhinology: Rhinosinusitis (ICAR-RS). They reported that the ChatGPT performed satisfactorily, correctly answering more than 80% of the questions in all categories(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eYe F. et al. evaluated the accuracy of ChatGPT in answering 189 questions about allergic rhinitis (AR) and 249 questions about chronic rhinosinusitis (CRS). They found that the ChatGPT had a high rate of conveying accurate information for AR and CRS patients(\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eBellinger et al. asked questions about epistaxis, chronic sinusitis, sinus infection, allergic rhinitis, allergies and treatment of nasal polyps to ChatGPT, ChatGPT Plus, Google Bard and Microsoft Bing chatbots. They reported that ChatGPT Plus and Google Bard are currently the most patient-friendly chatbots for the treatment of common pathologies in rhinology(\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e). In our study, questions consisting of general information about rhinology and detailed information were directed to ChatGPT to evaluate whether ChatGPT can be a part of the education process and informing patients. If we look at the strengths of our study, it can be said that the accuracy and reproducibility of ChatGPT was evaluated in 130 questions, which reduced the margin of error. In addition, the fact that 3 different ENT specialists answered the answers independently of each other enabled us to get away from errors that may be caused by the person.\u003c/p\u003e \u003cp\u003eThe answers given by ChatGPT to the same question when asked for the first and second time were found to be 91.5% consistent. Kuşcu et al. also found 94.1% in their study and this result is compatible with our study(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e). And the reproducibility of ChatGPT about rhinology was found to be satisfactory. In 11 questions, there was a significant inconsistency between the two answers. In our study, general rhinology was seen as the subheading with the highest number of inconsistent answers with 3 inconsistent answers. When evaluated between the first and second inconsistent answers, the second answer was found to be more accurate than the first.\u003c/p\u003e \u003cp\u003eThe rates of 3(A mix of accurate and inaccurate/misleading) and 4(Completely incorrect/ irrelevant) answers of ChatGPT in rhinology were found to be 7.7%, 13.8% and 10%, respectively. In the light of this result, the inaccuracy of ChatGPT in patient information and education process is considered to be at an acceptable level and reliable. When the complete correctness of the answers was evaluated, the rate of 1(Completely correct) answers was found to be 76.2%, 62.3% and 61.5% in the controllers, respectively. Kuş\u0026ccedil;u et al. found a completely correct response rate of 86.4% in their study on head and neck cancers with ChatGPT(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e). The reason for the lower rate of correct answers in our study despite similar standardization may be attributed to the high number of questions requiring detailed information and based on interpretation in our study.\u003c/p\u003e \u003cp\u003eHoch et al. evaluated the accuracy of ChatGPT's answers to 2576 (479 multiple-choice and 2097 single-choice) questions in otolaryngology board certification exams. And as a result, they found that 57% of the total questions were answered correctly(\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e). Patel et al. ChatGPT4 was reported to be 86% correct and was found to be higher than our study(\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). Ye F. et al. found the accuracy of ChatGPT to be 80% in a study of 438 questions related to allergic rhinitis (AR) and chronic rhinosinusitis (CRS)(\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe reason for this difference may be related to the number of questions, but it may also be due to the fact that the answers were standardized in more detail in our study, while the answers were evaluated as true/false in other studies. We believe that evaluating the questions similar to the standardization we used in our study, rather than evaluating them as true/false, will give more accurate results and will be more accurate in terms of developing AI. At the same time, we think that the re-evaluation of questions that require clear information and questions that require interpretation after classification will contribute to the literature. Although the rate of giving correct information in ChatGPT answers is at a satisfactory level, it is seen that there are deficiencies in terms of relying on complete correctness. The rates of 1(Completely correct) and 2(partially correct) answers were 92.3%, 86.2% and 90%, respectively. With these results, the rate of ChatGPT answers containing correct information was found to be satisfactorily high. This information increases the possibility of being used as informative and educational material for patients under the control of experts. We think that it may be right to be cautious for now due to the possible risks of using it without expert control. As a result, we think that it is an important resource under expert team control.\u003c/p\u003e"},{"header":"Conclusıon","content":"\u003cp\u003eChatGPT has proven to be a reliable source of information for patients and clinicians by answering questions in the field of rhinology with a low margin of error and high reproducibility rate. It is indisputable that AI will become an increasingly reliable source with technological improvements and intensified studies on this subject. In addition to being a potential source of information for now, studies are needed to make it a useful tool in the decision-making process of clinicians in diagnosis and treatment in the future.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments:\u0026nbsp;\u003c/strong\u003eNone\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest:\u0026nbsp;\u003c/strong\u003eAll the authors declare that there is no conflict of interest related to this study.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u003c/strong\u003e Source of financial support or funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical approval and informed consent statements:\u0026nbsp;\u003c/strong\u003eThe study does not require ethics committee approval and informed consent.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability and material:\u0026nbsp;\u003c/strong\u003eThis article includes all of the data that were gathered or examined during this investigation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contribution statement:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePreparing, organizing and asking questions to ChatGPT: Aykut \u0026Ouml;ZDOĞAN\u003c/p\u003e\n\u003cp\u003eEvaluation of the answers: Bur\u0026ccedil;ay TELLİOĞLU, Oğuzhan KATAR, Serdar \u0026Ouml;ZER\u003c/p\u003e\n\u003cp\u003eAnalyzing and writing up the data: Aykut \u0026Ouml;ZDOĞAN, Serdar \u0026Ouml;ZER\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAykut \u0026Ouml;ZDOĞAN\u003csup\u003e1,*\u003c/sup\u003e, Bur\u0026ccedil;ay TELLİOĞLU\u003csup\u003e2\u003c/sup\u003e, Oğuzhan KATAR\u003csup\u003e3\u003c/sup\u003e, Serdar \u0026Ouml;ZER\u003csup\u003e4\u003c/sup\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e1\u003c/sup\u003e Department of Otorhinolaryngology, Ankara Etlik City Hospital, Ankara, T\u0026uuml;rkiye\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;2\u0026nbsp;\u003c/sup\u003eElbistan State Hospital, Kahramanmaraş, T\u0026uuml;rkiye\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e3 \u0026nbsp;\u003c/sup\u003ePolatlı Duatepe State Hospital, Ankara, T\u0026uuml;rkiye\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;4 \u0026nbsp;\u003c/sup\u003eDepartment of Otorhinolaryngology, Hacettepe University, Ankara, T\u0026uuml;rkiye\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eBiswas SS. Role of chat GPT in Public Health. Ann Biomed Eng 2023; 51:868\u0026ndash;869\u003c/li\u003e\n \u003cli\u003eMunoz-Zuluaga C, Zhao Z, Wang F, et al. Assessing the accuracy and clinical utility of chatGPT in laboratory medicine. Clin Chem 2023; 69:939\u0026ndash;40.\u003c/li\u003e\n \u003cli\u003eThe Lancet Digital H. ChatGPT: friend or foe? Lancet Digit Health (2023) 5(3): e102. doi:10.1016/S2589-7500(23)00023-7\u003c/li\u003e\n \u003cli\u003evan Dis EAM, Bollen J, Zuidema W, et al. ChatGPT: five priorities for research. Nature 2023; 614:224\u0026ndash;6.\u003c/li\u003e\n \u003cli\u003eOpenAI. ChatGPT. 2023. https:// openai.com/blog/chatgpt (Accessed March 28, 2023).\u003c/li\u003e\n \u003cli\u003eExploding Topics. Number of ChatGPT users 2023. https://explodingtopics.com/blog/ chatgpt-users (Accessed March 30, 2023).\u003c/li\u003e\n \u003cli\u003eElse H. Abstracts written by ChatGPT fool scientists. Nature 2023; 613:423.\u003c/li\u003e\n \u003cli\u003eHaupt CE, Marks M. AI-generated medical advice-GPT and beyond. JAMA 2023; 329:1349\u0026ndash;50.\u003c/li\u003e\n \u003cli\u003eJeblick K, Schachtner B, Dexl J, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol 2024;34:2817-2825.\u003c/li\u003e\n \u003cli\u003eAllahqoli L, Ghiasvand MM, Mazidimoradi A, et al. The diagnostic and management performance of the chatGPT in obstetrics and gynecology. Gynecol Obstet Invest 2023; 88:310\u0026ndash;3.\u003c/li\u003e\n \u003cli\u003eSamaan JS, Yeo YH, Rajeev N, et al. Assessing the accuracy of responses by the language model chatGPT to questions regarding bariatric surgery. Obes Surg 2023;33:1790\u0026ndash;6.\u003c/li\u003e\n \u003cli\u003eYeo YH, Samaan JS, Ng WH, et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. \u003cem\u003eClin Mol Hepatol\u003c/em\u003e. 2023;29:721-732.\u003c/li\u003e\n \u003cli\u003eDave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6:1169595.\u003c/li\u003e\n \u003cli\u003eDe Angelis L, Baglivo F, Arzilli G, et al. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front Public Health 2023;11:1166120\u003c/li\u003e\n \u003cli\u003eKuscu O, Pamuk AE, S\u0026uuml;tay S\u0026uuml;sl\u0026uuml; N, et al. Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer? Front. Oncol. 2023;13:1256459.\u003c/li\u003e\n \u003cli\u003ePark I, Joshi AS, Javan R. Potential role of ChatGPT in clinical otolaryngology explained by ChatGPT. Am J Otolaryngol 2023; 44:103873.\u003c/li\u003e\n \u003cli\u003eAyoub NF, Lee YJ, Grimm D, et al. Comparison between chatGPT and google search as sources of postoperative patient instructions. JAMA Otolaryngol Head Neck Surg 2023; 149:556\u0026ndash;8.\u003c/li\u003e\n \u003cli\u003eChiesa-Estomba CM, Lechien JR, Vaira LA, et al. Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol 2024;281:2777\u003c/li\u003e\n \u003cli\u003eHoch CC, Wollenberg B, Luers JC, et al. ChatGPT\u0026apos;s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol 2023;280:4271\u0026ndash;8.\u003c/li\u003e\n \u003cli\u003eRiestra-Ayora J, Vaduva C, Esteban-S\u0026aacute;nchez J, et al. ChatGPT as an information tool in rhinology. Can we trust each other today? Eur Arch Otorhinolaryngol. 2024;281:3253-3259.\u003c/li\u003e\n \u003cli\u003eManiaci A, Saibene AM, Calvo-Henriquez C, et al. Is generative pre-trained transformer artificial intelligence (Chat-GPT) a reliable tool for guidelines synthesis? A preliminary evaluation for biologic CRSwNP therapy. Eur Arch Otorhinolaryngol. 2024;281:2167-2173.\u003c/li\u003e\n \u003cli\u003ePatel EA, Fleischer L, Filip P, et al. Comparative Performance of ChatGPT 3.5 and GPT4 on Rhinology Standardized Board Examination Questions. OTO Open. 2024;8:e164.\u003c/li\u003e\n \u003cli\u003eRadulesco T, Saibene AM, Michel J, et al. ChatGPT-4 performance in rhinology: A clinical case series. Int Forum Allergy Rhinol. 2024;14:1123-1130.\u003c/li\u003e\n \u003cli\u003eWorkman AD, Rathi VK, Lerner DK, et al. Utility of a LangChain and OpenAI GPT-powered chatbot based on the international consensus statement on allergy and rhinology: Rhinosinusitis. Int Forum Allergy Rhinol. 2024;14:1101-1109.\u003c/li\u003e\n \u003cli\u003eYe F, Zhang H, Luo X, et al. Evaluating ChatGPT\u0026apos;s Performance in Answering Questions About Allergic Rhinitis and Chronic Rhinosinusitis. Otolaryngol Head Neck Surg. 2024;171:571-577.\u003c/li\u003e\n \u003cli\u003eBellinger JR, Kwak MW, Ramos GA, et al. Quantitative Comparison of Chatbots on Common Rhinology Pathologies. Laryngoscope. 2024;134(10):4225-4231.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"ChatGPT 4, rhinology, artificial intelligence, chatbot, machine learning, head and neck","lastPublishedDoi":"10.21203/rs.3.rs-5788648/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5788648/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground:\u003c/b\u003e\u003c/p\u003e \u003cp\u003eChatGPT is a new artificial intelligence model designed to create human-like chat. As a result of advancing knowledge and technological improvements, it is promising in the field of medicine, especially as a resource that patients and clinicians can apply.\u003c/p\u003e\u003cp\u003e\u003cb\u003eObjective:\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe aim of our study is to measure the accuracy and consistency of ChatGPT's answers to questions in the field of rhinology.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods:\u003c/b\u003e\u003c/p\u003e \u003cp\u003eIn March 2024, the ChatGPT (ChatGPT version 4) was presented with 130 questions in rhinology. Each question was asked to ChatGPT twice and the consistency/reproducibility of the answers was investigated. The answers were evaluated by three ENT physicians. The physicians followed a standardised 4-point format (1:Completely correct, 2:Partially correct, 3:A mix of accurate and inaccurate/misleading, 4:Completely incorrect/ irrelevant).\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults:\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe answers given by ChatGPT were consistent at a rate of 91.5%(119/130). Among the inconsistent answers, the second answer was found to be more correct in 10/11. Statistically, the second answer was found to be more correct (p: 0011). In 130 questions, as a result of the controllers' evaluation, the number of answers evaluated as completely correct was 99/81/80(76.2%/62.3%/61.5%) respectively. However, completely incorrect answers were 7/6/7(5.4%/4.6%/5.4%), respectively. Accordingly, it is seen that there is no statistical difference between the controllers(p:0.270).\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusion:\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe inaccuracy of ChatGPT in patient information and education process is considered to be at an acceptable level and reliable. However, it is also seen that ChatGPT answers are not completely correct and can give misleading answers to some questions. We believe that it would be safer and more accurate to use ChatGPT as an informative and educational material for patients with the control of experts.\u003c/p\u003e","manuscriptTitle":"Is ChatGPT's Knowledge on Rhinology Accurate? Can It Be Utilized in Medical Education and Patient Information?","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-20 05:59:03","doi":"10.21203/rs.3.rs-5788648/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"35dee431-0ab6-4257-bd05-811cdcde90b4","owner":[],"postedDate":"January 20th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-01-20T05:59:03+00:00","versionOfRecord":[],"versionCreatedAt":"2025-01-20 05:59:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5788648","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5788648","identity":"rs-5788648","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}