Assessment of the Ability of the ChatGPT-5 Model to Pass the Endocrinology Specialization Exam.

ChatGPT, developed by OpenAI, is an internet-based chatbot built on AI and powered by a large language model (LLM) [ 1 ]. Its origins date back to 2018 with the development of GPT-1, which served as the foundation for subsequent improvements leading to newer generations. A breakthrough occurred with GPT-3, which enabled direct interaction with ChatGPT through the ability to ask questions and receive detailed, practical responses. It also facilitated text generation for a variety of applications, as well as programming code generation. In March 2023, the next generation, ChatGPT-4, was released, featuring enhancements such as reduced likelihood of producing offensive content, improved alignment of responses, and increased factual accuracy [ 2 ]. In August 2025, the most recent model, ChatGPT-5, was launched in several versions, standard, mini, and nano, differing primarily in processing speed, available functionalities, and computational capacity. Data from 2025 indicate that ChatGPT is used by approximately 800 million to 1 billion individuals worldwide. In the United States, around 53% of adults report using the application, making it the most widely adopted AI tool among consumers. Interest continues to grow, as demonstrated by a fourfold increase in daily active users between 2024 and 2025. OpenAI continues to refine the product to further expand its user base [ 3 ]. Owing to its versatility, ChatGPT has gained increasing popularity globally, particularly in research environments, where it is employed for data processing and hypothesis generation [ 1 ]. It is also gaining traction in medicine, with potential applications in clinical research and decision support. One field in which AI technologies are especially impactful is radiology, which relies heavily on the interpretation of imaging studies and large diagnostic datasets [ 4 ]. Another area of interest is medical education, including the evaluation of AI’s ability to solve examination tasks from the National Medical Licensing Examination or specialty board examinations. In Poland, recent studies by Jaworski A et al. demonstrated that ChatGPT-4o successfully passed medical and dental final examinations, as well as specialty board examinations (Państwowy Egzamin Specjalizacyjny, PES), achieving scores of 78.3% in ophthalmology and 71.43% in infectious diseases [ 5 - 6 ]. Similar findings have been reported in other countries. Jungo K et al. showed that GPT-4 exceeded the passing threshold on the Japanese National Medical Licensing Examination, highlighting the potential of LLMs even in non-English testing contexts. Likewise, studies conducted in Israel indicated that ChatGPT and other LLMs achieved above-passing scores on the Family Medicine licensing exam, particularly on general multiple-choice questions, although their performance remained limited in tasks requiring complex clinical reasoning. These results point to the growing potential of LLMs in medical education and competency assessment [ 7 - 8 ]. Artificial intelligence is increasingly supporting endocrinology by facilitating diagnosis, treatment, and patient education. Studies have shown that AI algorithms can effectively analyze ultrasound images, detect thyroid nodules, and assess their characteristics with accuracy comparable to that of radiology experts [ 9 ]. AI is also used to predict the risk of other hormonal disorders, such as Cushing’s disease or hypothyroidism, through the analysis of large clinical datasets [ 10 ]. In clinical decision support systems, AI suggests optimal treatment strategies and helps determine appropriate doses of hormonal medications, thereby improving therapy personalization [ 11 ]. Moreover, AI supports education by creating personalized materials for patients and interactive training tools for physicians, enhancing both treatment effectiveness and patient engagement [ 12 ]. The aim of the present study was to assess whether the latest version, ChatGPT-5 Plus, is capable of passing the Polish endocrinology specialty board examination (PES). The passing threshold was set at 60% of the available points. Both answer accuracy and the model’s self-reported confidence ratings were analyzed. The evaluation was based on examination questions and the official answer key published by the Medical Examination Center (CEM) in Łódź, along with the responses generated by ChatGPT. Considering the growing role of AI in endocrinology, supporting diagnosis, optimizing treatment strategies, and providing educational tools for physicians and patients, assessing the ability of the most advanced model to solve standard examination questions represents an important step in evaluating its potential as both a decision-support and training tool for endocrinologists.

The ChatGPT-5 model achieved a score of 76.47%, corresponding to 91 correct and 28 incorrect answers (23.53%) (Figure 1 and Table 1 ). The table presents the model’s responses in relation to the correct answers, based on the official answer key published by the Medical Examination Center (CEM) in Łódź. For each question, the declared confidence level provided by the model is reported on a 1-5 scale, where 1 = no confidence, 2 = low confidence, 3 = moderate confidence, 4 = high confidence, and 5 = complete confidence. Table 2 presents a comparison of the model’s accuracy in responding to clinical and theoretical questions. For clinical questions, the proportion of correct answers was 68.97%, whereas for theoretical questions it reached 78.89%. However, this difference did not reach statistical significance (χ² = 1.200; p = 0.273). Figure 2 visualizes these results, showing the percentage of correct answers separately for clinical and theoretical questions. Data are presented as absolute numbers (N) and percentages (%). No statistically significant differences in response accuracy were observed between clinical and theoretical/other questions (χ² = 1.200, p = 0.273). The figure shows the percentage of correct answers in two categories of questions: clinical and theoretical. The data are based on the results presented in the table above, allowing a comparison of the model’s performance on practical (clinical) versus theoretical questions. Analysis of the confidence levels declared by the ChatGPT-5 model revealed significant differences between correct and incorrect responses. Figure 3 illustrates the distribution of confidence ratings (on a 1-5 scale) in both groups. For correct answers, the model more frequently reported higher confidence levels, whereas incorrect answers were associated with relatively lower ratings. This difference reached statistical significance (U = 950; p = 0.017). The figure illustrates the data obtained using the Mann-Whitney test. The computed test statistic was 950 (U = 950), with a significance level of 0.017 (p < 0.05), indicating statistical significance. To illustrate the interaction process, Appendices 1-2 present selected screenshots of ChatGPT’s responses to the exam questions. To preserve consistency with the content and wording of the official examination items, all interactions with the model were conducted in Polish. Consequently, the screenshots are displayed in the original language.

The model answered 91 questions correctly (76.47%) and 28 incorrectly (23.53%) out of 119 analyzed items, surpassing the passing threshold and demonstrating performance comparable to earlier ChatGPT versions assessed on other specialty examinations. For example, prior evaluations showed scores of 78.3% in ophthalmology and 71.43% in infectious diseases. These findings support the notion that the newest generations of large language models consistently achieve high performance in solving specialty examination questions across various fields of medicine. Statistical analysis revealed no significant differences in accuracy between clinical and theoretical questions (χ² = 1.200; p = 0.273). The model performed similarly well regardless of question type, whether based on clinical case scenarios or focused on purely theoretical knowledge. By contrast, analysis of declared confidence levels demonstrated significantly higher ratings for correct answers compared with incorrect ones (U = 950; p = 0.017). The results of the present study align with the findings of Jaworski A et al., who evaluated the performance of the ChatGPT-4o model on specialty examinations in ophthalmology and infectious diseases. As in those studies, no significant differences were observed in accuracy between clinical and theoretical items, while confidence levels were higher for correct responses. When compared with earlier models such as ChatGPT-3.5, which achieved only 52% in radiology and 52.54% in allergology specialty examinations, failing to reach the passing threshold [ 14 - 15 ], the superiority of newer models becomes evident. A study conducted in Poland by Laskowski M et al., which evaluated the performance of ChatGPT-3.5 and ChatGPT-4 on the neurosurgery PES examination administered in autumn 2017, similarly revealed a significantly higher accuracy rate for the newer model (35.3% vs. 60.3%) [ 16 ]. The findings are consistent with results obtained in international examinations, including the United States Medical Licensing Examination (USMLE). GPT-4 models achieved accuracy rates of up to 86%, clearly surpassing the passing threshold and demonstrating stable performance across diverse clinical domains [ 17 ]. Broader comparative analyses that incorporated the USMLE and other medical licensing examinations confirmed the superiority of GPT-4 and GPT-4o over earlier versions such as GPT-3.5, as well as over alternative large language models that scored substantially lower [ 18 ]. Other studies conducted in Chile showed that ChatGPT-4 and GPT-4V achieved scores of 79.32% and 78.83%, respectively, while ChatGPT-3.5 obtained 57.53%. In Brazil, ChatGPT-3.5 achieved a score of 68.4%, whereas the newer ChatGPT-4o version reached as high as 87.2% [ 19 - 20 ]. These findings suggest notable progress in AI development; however, confirmation of this hypothesis requires further studies using the latest AI models across different medical specialties. Although the present analysis focuses on GPT-4, and the examples discussed above likewise refer to this generation of large language models, it should be noted that GPT-5 has recently been released. At present, peer-reviewed evidence on the performance of GPT-5 in medical specialty or licensing examinations is lacking. The smaller-than-expected difference between GPT-5 and GPT-4o may be due to the fact that the test questions primarily assess factual knowledge, in which both models achieve similarly high scores. GPT-5 has also been optimized for safety and response consistency, which can lead to more cautious answers that do not necessarily increase the number of correct responses. Furthermore, at a high level of accuracy, further improvements translate into relatively small percentage gains. It is also possible that more diverse and challenging tests in the future will better reveal the advantages of GPT-5. Further studies are needed to determine whether the technical advances attributed to this model will translate into improved performance in medical examinations and medical education. Additionally, future studies should evaluate specialized medical models, which may outperform general-purpose LLMs in tasks requiring advanced clinical reasoning or in specific medical specialties. The present study is subject to several limitations. First, it relies exclusively on a single Polish specialty examination in endocrinology, and the lack of prior research on the application of language models in this field limits the ability to compare the obtained results with existing literature. Second, although the exam questions were originally formulated in Polish, the analytical tool operated primarily in English, which may have led to the loss of subtle terminological and contextual differences, affecting the model’s understanding, performance, and the accuracy of responses. Additionally, the probabilistic nature of the generated answers means that results may vary across repeated attempts, reducing reproducibility. Results may also be inflated if the model’s training data included materials overlapping with the examination questions. Another limitation concerns equity and accessibility, access to GPT-5 requires a Plus plan subscription, currently costing over 20 USD per month, which may limit availability for students, educators, or institutions in low-resource regions. This could exacerbate existing educational disparities. Future strategies could include the development of open-access language models, cheaper subscriptions for low-income regions, or institutional agreements providing shared access, helping to ensure a more equitable distribution of AI-assisted medical education benefits.

In this study, the ChatGPT-5 model achieved a score of 76.47%, surpassing the passing threshold of the endocrinology specialty examination (spring 2025 session). The result is comparable to previous studies evaluating the performance of ChatGPT-4o. However, when compared with ChatGPT-3.5, a clear improvement was observed, as the newer model generations consistently exceeded the passing threshold. The progression from version 3.5 to 4 represented a substantial increase in effectiveness, while ChatGPT-5 demonstrated performance similar to that of ChatGPT-4o. It should be emphasized that a more pronounced improvement had been anticipated with the release of the newest generation. Therefore, the outcome can be regarded as adequate but not fully satisfactory in the context of expected progress. Continued advancements in AI are anticipated, with the potential to achieve higher levels of accuracy, opening new opportunities and influencing both clinical practice and research across various medical specialties.

The study was conducted between August 11 and 13, 2025, using the ChatGPT-5 Plus model in the browser version. The endocrinology specialty board examination (spring 2025 session) was selected from the database of the CEM in Łódź [ 13 ]. Test questions from the specialization examination, together with the correct answers, have been published by CEM within seven days of the examination date since the spring session of 2023. These data are publicly accessible on the official CEM website. The exam initially consisted of 120 questions; however, one question was withdrawn by CEM after being classified as inconsistent with current medical knowledge and was excluded from statistical analyses and the calculation of total correct answers. Each question contained five answer options, only one of which was correct. The entire study was carried out in Polish. Questions were categorized into two groups. Clinical questions required identifying the correct answer based on a patient’s clinical presentation, which included case history, laboratory results, or imaging studies. The second category comprised theoretical/other questions, aimed at assessing knowledge of current treatment standards defined by professional medical societies, as well as principles of clinical management. Classification was performed by two independent researchers, with any disputed cases reviewed by a third researcher. Before attempting the examination, the ChatGPT-5 model was presented with the official exam regulations, the pool of questions, and the rule that only one correct answer was possible for each question. No changes were made to default settings, and the model was not reset between questions; each question was attempted only once, and the single response was used for analysis. All questions and model-generated answers were documented and compared with the official CEM answer key. For further analysis, after each response the model was additionally asked: “On a scale from one to five, how confident are you in this answer?” The possible confidence ratings were: 1 - no confidence, 2 - low confidence, 3 - moderate confidence, 4 - high confidence, and 5 - complete confidence. This allowed evaluation of the model’s self-reported confidence levels. Statistical analyses were performed using Microsoft Excel and GraphPad Prism 10. Two approaches were applied: the chi-square test to compare the number of correct versus incorrect answers between clinical and theoretical/other questions, and the Mann-Whitney U test to compare confidence levels between correct and incorrect responses. A p-value < 0.05 was considered statistically significant.