An Exploratory Evaluation of GPT-5 in Periodontitis Staging and Grading Using Published Clinical Cases

An Exploratory Evaluation of GPT-5 in Periodontitis Staging and Grading Using Published Clinical Cases | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An Exploratory Evaluation of GPT-5 in Periodontitis Staging and Grading Using Published Clinical Cases Ihunna Amugo, Katie L. Frederickson, Harshana Rajakaruna, Hua Xie, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8742868/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Periodontitis is a chronic gum disease affecting approximately 42% of adults aged 30 and older in the United States. Training dental students to accurately diagnose and manage periodontitis is a critical component of dental education and clinical care. Recent advances in large language models (LLMs) offer new opportunities to support both domains, yet their performance in periodontal diagnosis remains largely unexplored—particularly for newer models such as Generative Pre-trained Transformer 5 (GPT-5). Objective This study conducted an exploratory evaluation of GPT-5’s ability to stage and grade periodontitis. Methods Twenty-five publicly available clinical cases were identified through Google and PubMed searches. Each case description was entered into GPT-5 using a zero-shot prompting approach, and the model’s predictions were compared with the published reference diagnoses. Performance was measured using accuracy and Cohen’s kappa. Results Across these cases, GPT-5 showed marked class-dependent performance and a tendency to overestimate disease severity. Compared with prior models, it achieved comparable or improved performance, with accuracies of 68.0% for staging and 77.3% for grading and corresponding Cohen’s kappa values of 0.432 and 0.179, respectively. While staging performance showed fair agreement beyond chance, the low kappa for grading indicates poor agreement and limited reliability in distinguishing periodontal disease severity. Conclusions These findings suggest that although GPT-5 shows improvement over previous models, its current diagnostic performance, particularly for periodontitis grading, limits its utility in clinical assessment and educational training. Meaningful application in periodontal diagnosis and training will require substantial improvements in reliability and rigorous validation. The limitations of the study and implications for future development are also discussed. Dentistry Artificial Intelligence and Machine Learning large language model ChatGPT GPT-5 dental care dental education periodontitis periodontitis staging and grading Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction As the demand for accessible, accurate, and cost-effective resources to support dental care and education continues to grow, large language model (LLM)–based chatbots, such as Chat Generative Pre-trained Transformer (ChatGPT)[ 1 ], have emerged as promising tools. Although not originally developed for dental and healthcare applications, these systems can generate human-like responses with remarkable accuracy on many health-related topics [ 2 – 8 ], offering new opportunities for disease surveillance, biomedical research, and education. Compared with traditional resources, LLMs offer distinct advantages for education and diagnostic support, including lower cost, continuous availability without the need for appointments, good accuracy for many diseases and conditions, customizable interactions, and user-friendly interfaces. As a result, people increasingly turn to them for medication information, self-diagnosis, and disease-prevention guidance[ 9 – 12 ]. Clinicians, dental and medical students also use them to acquire knowledge and support clinical decision-making[ 13 , 14 ]. A growing body of research has examined LLMs’ utilities in dental care and education[ 5 , 15 – 18 ]. Several benchmark studies have demonstrated LLMs’ competitive performance on the American Academy of Periodontology (AAP) in-service examination[ 6 ], the United States Medical Licensing Examination (USMLE)[ 2 , 7 ], and other major assessments[ 1 , 3 , 8 ]. In an educational study involving 77 second-year dental students, those who used LLMs for learning assignments were found to perform better on knowledge examinations than peers relying on traditional methods[ 19 ]. Furthermore, Rahad et al. showed that ChatGPT excelled in recognizing and correcting specialized dental terminology and achieved 66.7% accuracy in extracting and synthesizing information from documents [ 5 ]. In the clinical context, Eroglu et al. evaluated ChatGPT-3.5 on 200 untreated periodontitis cases and reported moderate performance for staging and grading[ 4 ]. Despite these advances, critical gaps remain. Most prior studies used non-public datasets, limiting the reproducibility of their findings. Moreover, given the continual iteration and rapid improvement of LLMs, earlier assessments may not accurately reflect the capabilities of newer models such as Generative Pre-trained Transformer 5 (GPT-5), released in August 2025, whose performance in dentistry has not yet been systematically evaluated. Evaluation of LLMs in high-stakes dental clinical contexts is essential to establish quality-control mechanisms, mitigate risks of inaccurate or biased outputs, and guide their safe adoption into dental education and care. An important component of dental education and care is training students to diagnose and manage periodontitis, a chronic gum disease affecting approximately 42% of adults aged 30 and older in the US[ 20 ]. Periodontitis staging (I–IV) reflects disease severity and extent based on levels of destroyed tissues, including gingival attachment and alveolar bone, while grading (A–C) estimates the rate of progression and future risk[ 21 – 23 ]. Even with explicit and standardized criteria for staging and grading[ 21 – 23 ], clinical diagnosis of periodontitis remains challenging and context-dependent, requiring careful integration of radiographic evidence, periodontal charting, and patient-specific risk factors. An LLM capable of accurately analyzing, staging, and grading clinical periodontitis cases could serve as both a valuable diagnostic aid for clinicians and a useful educational resource for students. To address this need, we conducted an exploratory evaluation of the performance of the newly released GPT-5 in staging and grading periodontitis cases. Methods Case Identification and Data Collection We identified dental clinical cases by searching Google and PubMed in August and September 2025 using the keywords ‘periodontitis’, ‘staging’, and ‘grading.’ These searches targeted peer-reviewed articles, case reports, and publicly available teaching materials that explicitly described periodontitis staging and grading according to established clinical criteria. All records retrieved from the searches were screened manually. Review articles, duplicate records, and reports lacking complete staging or grading diagnoses, sufficient periodontal clinical descriptions, or adequate medical and dental histories were excluded. After screening, 25 cases were retained from a total of 52 identified records. The workflow for data collection and case selection is illustrated in Fig. 1 . Most of the public cases collected included panoramic and periapical radiographs, which, together with patients’ dental histories and periodontal charting, play a central role in diagnosis. In these cases, the radiographs had already been systematically evaluated by the original authors, and numerical measures of the bone loss were extracted and reported in their publications. We manually extracted these clinically meaningful data from the publications and used them directly in our assessments. Radiographic images themselves were not used in our analyses. It is important to note that publicly available and teaching-oriented cases, which are often unusually well documented and may not fully reflect the spectrum of routine clinical presentations, may introduce selection bias and limit generalizability. As a result, the GPT-5 performance observed in this exploratory study should be interpreted cautiously and may not reflect real-world diagnostic performance. These limitations are discussed in greater detail in the Discussion section. GPT-5 Prompting and Evaluation While GPT-5 can process multimodal data, we used only its interactive textual interface. All analyses were conducted between September 1st and 12th, 2025, using the free GPT-5 version, without any paid enhancements. GPT-5 was accessed through its publicly available interactive interface, which does not permit user control over system-level parameters such as temperature and system prompts. Therefore, all interactions were conducted using the default system configuration (temperature = 1.0). Given GPT-5’s demonstrated accuracy and reasoning improvements over earlier models [ 24 ], and because the criteria for periodontitis staging and grading outlined in the 2017 World Workshop are strictly guideline-based [ 21 – 23 ], we adopted a zero-shot prompting strategy. This approach evaluates GPT-5’s ability to apply explicit clinical thresholds and decision logic without influence from exemplar conditioning. In contrast to few-shot prompting or fine-tuning [ 25 – 27 ], which may introduce anchoring effects or label leakage and thereby inflate performance, zero-shot prompting provides a more conservative and transparent assessment of model reasoning. Using zero-shot prompting, case descriptions were submitted directly to the model without examples or prior instructions, and GPT-5 was asked to return predictions of periodontitis stage and grade. Before submission, case descriptions were lightly reformatted to correct line breaks, spacing, and formatting artifacts introduced during PDF extraction to improve readability for the model. No clinical content was added, removed, reworded, or reorganized, and the original meaning and structure of the source material were preserved. An example prompt for a clinical case and a detailed ChatGPT response are provided in the appendix. To assess response stability, each case was tested in multiple sessions using slight variations in prompt phrasing (e.g., “Can you help determine the periodontitis stage and grade of this patient?” or “Please stage and grade the periodontitis of this patient”). The model consistently generated identical predictions, regardless of prompt phrasing or session timing, indicating stable behavior at the decision level under these conditions. For the final analyses reported in this study, we used a standardized prompt, “Please stage and grade the periodontitis of this patient”, followed by the corresponding clinical case description. Data collection and analysis were conducted by four domain experts, all of whom are co-authors of this study. The team included a dental student, two professors from the School of Dentistry, and a professor of Data Science from the School of Medicine at Meharry Medical College. Their responsibilities included extracting clinical cases from publications, developing prompts for GPT-5, and verifying the model’s responses. Performance Metrics GPT-5 predictions were compared against published ground-truth diagnoses to calculate model accuracy and Cohen’s kappa. To characterize error patterns, confusion matrices were generated for both staging and grading to allow assessment of tendencies toward overestimation or underestimation. Given the limited dataset size (n = 25) and the exploratory nature of this study, results are presented as descriptive performance metrics, without formal hypothesis testing. All data analyses were conducted in RStudio (v2025.09.2), and visualizations were generated using the R package ggplot2 (v4.3.3). Results Description of Periodontitis Cases Of the 25 periodontitis cases collected for evaluating GPT-5, two were borderline cases with staging ambiguities between Stage III and Stage IV, and three provided only staging information in the original publications. Full case descriptions and corresponding sources are provided in the Appendix . Figure 2 summarizes the cohort. The median age of these patients is 45 years, with most cases (68%) occurring between 35 and 64 years (Fig. 2 A). Females comprised the majority (72%), while males accounted for 28% (Fig. 2 B). In terms of disease severity, Stage III periodontitis was most common (56%), followed by Stage IV (36%), Stage II and I (4%) (Fig. 2 C). For grading, most cases were classified as Grade C (77%), with smaller proportions in Grade B (23%) (Fig. 2 D). Workflow for evaluating GPT-5 Figure 3 . Framework for evaluating GPT-5 performance in periodontitis staging and grading. ( A ) A prompt was constructed from a publicly available case[ 28 ], with minor formatting adjustments (e.g., line breaks and spacing) to improve readability for the model while preserving the original content. ( B ) GPT-5 generated categorical outputs for periodontitis stage (I–IV) and grade (A–C). ( C ) Model predictions were compared with the clinical reference diagnoses reported in the original publication to assess accuracy. The prompt was then submitted to GPT-5, which was asked to determine the stage and grade of periodontitis. The model’s response was collected (Fig. 3 B), recording only the predicted stage and grade while disregarding the diagnostic reasoning. The output was compared against the clinical diagnosis to evaluate accuracy (Fig. 3 C). After all cases were processed, GPT-5’s predictions were aggregated, and performance metrics were calculated to summarize its diagnostic accuracy and Cohen’s kappa and assess its potential utility in real-world clinical settings. GPT-5 performance in periodontitis staging and grading Across the 25 periodontitis cases (including the two borderline cases), GPT-5 achieved 68.0% accuracy for staging (17/25 correct) and 77.3% accuracy for grading (17/22 correct, excluding three cases without specified grades). The corresponding Cohen’s kappa values were 0.432 for staging and 0.179 for grading, indicating fair agreement and poor agreement beyond chance, respectively. The confusion matrix in Fig. 4 A shows that all Stage I and II cases were classified correctly (recall = 100%), whereas recall was substantially lower for Stage III (57%) and Stage IV (75%). For grading, performance was markedly imbalanced, with high recall for Grade C (94%) but very low recall for Grade B (20%) (Fig. 4 B). This class-dependent performance indicates that GPT-5 performs well for early-stage periodontitis and advanced disease detection but struggles to reliably distinguish intermediate disease categories. Importantly, misclassifications exhibited nonrandom patterns: errors were confined to adjacent categories, with no Stage I or II cases misclassified as Stage III or IV, no Stage IV cases misclassified as Stage I or II (Fig. 4 A), and no “skipping” across nonadjacent categories for grading (Fig. 4 B). Because all errors were limited to adjacent stages and grades and no catastrophic misclassifications were observed, these patterns are clinically and educationally relevant. Among the misclassified cases, 43% of Stage III cases were predicted as Stage IV, while 25% of Stage IV cases were predicted as Stage III (Fig. 4 A). For grading (Fig. 4 B), GPT-5 correctly classified 94% of Grade C cases, with most Grade B cases misclassified as Grade C. These results suggest that GPT-5 tends to assign higher severity for both periodontitis stage and grade. GPT-5’s predictions for the 25 cases are provided in the Appendix . Table 1 and Fig. 5 compare our findings with two prior assessments. Eroglu et al. evaluated GPT-3.5, an earlier ChatGPT version, on 200 untreated patients and reported 59.5% accuracy in staging and 50.5% accuracy in grading[ 4 ]—both notably lower than the performance achieved by GPT-5 on our dataset. The low kappa values observed for both models (0.284 for GPT-3.5 and 0.179 for GPT-5) in Table 1 underscore ChatGPT’s limited discriminatory ability in grading periodontitis beyond chance agreement. Table 1 Recent studies of periodontitis staging and grading using textual input. Dataset + Model Accuracy Cohen’s kappa Study ID Stage Grade Stage Grade 309 periodontal charts and clinical notes BERT 77.0% 75.0% NA NA Ameli et al. 200 untreated periodontitis patients GPT-3.5 59.5% 50.5% 0.447 0.284 Eroglu et al. 25 dental clinical cases from public sources GPT-5 68.0% 77.3% 0.432 0.179 Our result + The findings from the first two studies were extracted from published papers, while the details of our own assessment results are provided in the Appendix . Another study summarized in Table 1 and Fig. 5 , conducted by Ameli et al. [ 29 ], fine-tuned a Bidirectional Encoder Representations from Transformers (BERT) model using 309 anonymized periodontal charts and corresponding clinician notes. The model was trained on 70% of the data and tested on 32 holdout cases. The fine-tuned BERT model achieved 77.0% accuracy in staging and 75.0% in grading[ 29 ]. Although it outperformed GPT-5 in staging, it was slightly inferior in grading despite being specifically optimized for periodontitis. This comparison should be interpreted with caution, however, as BERT was evaluated on periodontal charts and clinician notes, whereas GPT-5 and GPT-3.5 were assessed using standardized textual case descriptions that included patient age, sex, and numerical measures of periodontitis-related parameters. Discussion Limitations Our exploratory evaluation of GPT-5 relied solely on published cases, which represent a relatively limited sample. Moreover, both the small sample size and the gender imbalance within the data may disproportionately reflect more severe or well-documented presentations, potentially inflating GPT-5’s performance. In addition, publicly available case reports are often curated to highlight clear diagnostic features and may not accurately reflect the full clinical heterogeneity or noise encountered in real-world practice. As a result, such cases may underrepresent diverse patient populations and disease presentations commonly seen in routine clinical settings. Expanding the dataset in future studies to include larger, more diverse, and non-published clinical data will therefore be essential to broaden the scope of evaluation, strengthen generalizability, and support meaningful clinical translation. Additional methodological constraints include our use of GPT-5’s interactive interface, which does not allow modification of underlying system parameters such as temperature, and hence may restrict reproducibility at the system level. Moreover, this study did not evaluate model stability across alternative prompting strategies, which are known to influence LLM behavior. Furthermore, only two borderline or equivocal cases were included, which are insufficient to assess GPT-5’s performance in diagnostically challenging scenarios where clinician disagreement is common and clinical judgment plays a critical role. Future studies should therefore incorporate more diagnostically ambiguous cases and systematically compare prompting strategies to better evaluate model reliability and clinical relevance. Although this study focused narrowly on GPT-5’s performance in periodontitis staging and grading, the potential applications of LLMs extend more broadly to both clinical diagnostics and dental education. In clinical practice, LLMs could assist practitioners by consistently applying standardized staging and grading criteria, integrating charting and radiographic data, and generating preliminary assessments. In education, LLMs can function as personalized learning assistants, offering structured feedback on case analyses and helping students navigate diagnostic complexity. Beyond these applications, LLM-based chatbots hold promise for reducing gaps in dental educational resources, particularly in under-resourced institutions, thereby strengthening their capacity to deliver high-quality dental education and care. However, the observed diagnostic agreement, particularly for periodontitis grading, highlights an important limitation. The low Cohen’s kappa (κ = 0.179) for grading indicates poor agreement beyond chance, suggesting that GPT-5 currently lacks sufficient reliability to accurately distinguish between periodontitis grades. Consequently, GPT-5 is not yet suitable for independent clinical grading, and its outputs should be interpreted with caution. In addition, in the absence of established minimum clinically important difference (MCID) thresholds for AI-assisted periodontitis staging/grading, κ = 0.432 for staging should be interpreted cautiously and not as evidence of clinical readiness. Looking forward, GPT-5 and other LLMs are likely to continue to improve diagnostic performance. Nevertheless, its meaningful clinical translation will hinge on overcoming current deficiencies in reliability and consistency. Beyond advancements in model development, a potential pathway toward high-stakes clinical and educational applications likely lies in the integration of LLMs with validated AI tools optimized specifically for clinical use. Such hybrid systems, which combine the precision of specialized diagnostic models with the reasoning, interpretability, and interactivity of LLMs, may provide more robust support for complex, multimodal clinical decision-making in dental care and education. Finally, this exploratory study was conducted with consideration of established AI reporting guidelines, including DECIDE-AI and STROBE-AI [ 30 – 32 ]. While the exploratory design and reliance on publicly available clinical cases, rather than real-time clinical data, precluded full adherence to all framework components, key principles such as transparency in data sources, model usage, limitations, and reproducibility were followed. Future prospective studies using large-scale, real-world clinical data will be better positioned to fully implement these reporting standards. Conclusions With the growing use of LLMs by dental and medical students, clinicians, and the general public, it is important to evaluate their performance in high-stakes diagnostic and educational settings to inform safety protocols and guide responsible applications. In this study, we assessed GPT-5’s ability to stage and grade periodontitis—tasks central to periodontal diagnosis and student training. Compared with prior models, GPT-5 demonstrated comparable or improved performance, achieving 68.0% staging accuracy and 77.3% grading accuracy, with a staging kappa of 0.432, indicating fair agreement beyond chance. However, despite these performance gains, the low kappa for grading (0.179) underscores the very limited discriminatory capacity in distinguishing periodontitis grades, indicating that GPT-5 is not yet suitable for independent clinical applications. Additionally, our results revealed a consistent tendency for GPT-5 to overestimate disease severity. Therefore, inappropriate reliance on model outputs could increase the risk of overtreatment or unnecessary escalation of care. This underscores the importance of human oversight and the need for future evaluations of uncertainty reporting, refusal behavior, and decision-level safeguards before any clinical integration is considered. In conclusion, while GPT-5 demonstrated potential as a supportive tool for education and clinical exploration, it is not yet ready for autonomous use. Meaningful application in periodontal diagnosis and training will depend on substantial improvements in reliability and rigorous validation. Abbreviations BERT GPT LLM Bidirectional Encoder Representations from Transformers Generative Pre-trained Transformer Large language model Declarations Data Availability The sources of our dental clinical cases are provided in the Appendix. Multimedia Appendix 1 GPT-5’s performance in periodontitis staging and grading on the textual input of 25 clinical cases. Acknowledgments We thank SeTonia Cook and Jacqueline Harding for helping with grant management. Funding This research was funded by the National Institute of Minority Health Disparities (NIMHD) under grant number U54MD007586, National Institute of Dental and Craniofacial Research (NIDCR) under grant number U01DE033241, National Institute of General Medical Sciences (NIGMS) under grant number R16GM149359, National Human Genome Research Institute (NHGRI) under award number UG3HG013248, National Institutes of Health (NIH) under Agreement Number 1OT2OD032581, Meharry's American Cancer Society (ACS) under grant number DICRIDG-21-071-01-DICRIDG, and Chan Zuckerberg Initiative (CZI) grant under award number CZIF2022-007043. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the NIH, CZI, and Meharry Medical College. Institutional Review Board (IRB) Statement IRB approval was not applicable, as the dental clinical cases used for assessment were obtained from publicly available sources. Informed Consent Statement Informed consent was not required, as this study used only publicly available data and did not involve human participants. Disclosure of Generative AI Use During the preparation of this manuscript, the authors used GPT-5 for minor editing and language polishing to improve the clarity of the English writing. All content was subsequently reviewed and revised by the authors, who take full responsibility for the final version of the publication. Conflicts of Interest None declared. References OpenAI GPT-5 System Card2025. Available from: https://cdn.openai.com/gpt-5-system-card.pdf Brin D, Sorin V, Vaid A, Soroush A, Glicksberg BS, Charney AW et al (2023) Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep 13(1):16492 Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW et al (2023) Large language models encode clinical knowledge. Nature 620(7972):172–180 Eroglu ZT, Babayigit O, Sen DO, Yarkac FU (2024) Performance of ChatGPT in classifying periodontitis according to the 2018 classification of periodontal diseases. Clin Oral Invest 28(7):407 Rahad K, Martin K, Amugo I, Ferguson S, Curtis A, Davis A et al (2024) ChatGPT to Enhance Learning in Dental Education at a Historically Black Medical College. Dent Res Oral Health 7(1):8–14 Ahmad B, Saleh K, Alharbi S, Alqaderi H, Jeong YN Artificial Intelligence in Periodontology: Performance Evaluation of ChatGPT, Claude, and Gemini on the In-service Examination. medRxiv [Internet]. 2024. Available from: https://dx.doi.org/10.1101/2024.05.29.24308155 Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C et al (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health 2(2):e0000198 Katz U, Cohen E, Shachar E, Somer J, Fink A, Morse E et al (2024) GPT versus Resident Physicians — A Benchmark Based on Official Board Scores. NEJM AI. ;1(5) Shahsavar Y, Choudhury A (2023) User Intentions to Use ChatGPT for Self-Diagnosis and Health-Related Purposes: Cross-sectional Survey Study. JMIR Hum Factors 10:e47564 Marley Presiado AM, Lopes L, Hamel L (2024) KFF Health Misinformation Tracking Poll: Artificial Intelligence and Health Information https://www.kff.org: The Henry J. Kaiser Family Foundation; [Available from: https://www.kff.org/health-information-and-trust/poll-finding/kff-health-misinformation-tracking-poll-artificial-intelligence-and-health-information/ Kuroiwa T, Sarcon A, Ibara T, Yamada E, Yamamoto A, Tsukamoto K et al (2023) The Potential of ChatGPT as a Self-Diagnostic Tool in Common Orthopedic Diseases: Exploratory Study. J Med Internet Res 25:e47621 Du D, Paluch R, Stevens G, Müller C Exploring patient trust in clinical advice from AI-driven LLMs like ChatGPT for self-diagnosis. arXiv [Internet]. 2024 2/2/2024. Available from: https://dx.doi.org/10.48550/arxiv.2402.07920 Kisvarday S, Yan A, Yarahuan J, Kats DJ, Ray M, Kim E et al (2024) ChatGPT Use Among Pediatric Health Care Providers: Cross-Sectional Survey Study. JMIR Formative Res 8:e56797 Ozkan E, Tekin A, Ozkan MC, Cabrera D, Niven A, Dong Y (2025) Global Health care Professionals’ Perceptions of Large Language Model Use In Practice: Cross-Sectional Survey Study. JMIR Med Educ 11:e58801–e Shifai N, Van Doorn R, Malvehy J, Sangers TE (2024) Can ChatGPT vision diagnose melanoma? An exploratory diagnostic accuracy study. J Am Acad Dermatol 90(5):1057–1059 Sattler SS, Chetla N, Chen M, Hage TR, Chang J, Guo WY et al (2025) Evaluating the Diagnostic Accuracy of ChatGPT-4 Omni and ChatGPT-4 Turbo in Identifying Melanoma: Comparative Study. JMIR Dermatology 8:e67551–e Cirone K, Akrout M, Abid L, Oakley A (2024) Assessing the Utility of Multimodal Large Language Models (GPT-4 Vision and Large Language and Vision Assistant) in Identifying Melanoma Across Different Skin Tones. JMIR Dermatology 7:e55508 Perlmutter JW, Milkovich J, Fremont S, Datta S, Mosa A (2025) Beyond the Surface: Assessing GPT-4's Accuracy in Detecting Melanoma and Suspicious Skin Lesions From Dermoscopic Images. Plast Surg Kavadella A, Dias Da Silva MA, Kaklamanos EG, Stamatopoulos V, Giannakopoulos K (2024) Evaluation of ChatGPT’s Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study. JMIR Med Educ 10:e51344 Eke PI, Thornton-Evans GO, Wei L, Borgnakke WS, Dye BA, Genco RJ (2018) Periodontitis in US Adults: National Health and Nutrition Examination Survey 2009–2014. J Am Dent Association 149(7):576– – 88.e6 Tonetti MS, Greenwell H, Kornman KS (2018) Staging and grading of periodontitis: Framework and proposal of a new classification and case definition. J Clin Periodontol 45(S20):S149–S61 Caton JG, Armitage G, Berglundh T, Chapple ILC, Jepsen S, Kornman KS et al (2018) A new classification scheme for periodontal and peri-implant diseases and conditions – Introduction and key changes from the 1999 classification. J Periodontol 89(S1):S1–S8 Chapple ILC, Mealey BL, Van Dyke TE, Bartold PM, Dommisch H, Eickholz P et al (2018) Periodontal health and gingival diseases and conditions on an intact and a reduced periodontium: Consensus report of workgroup 1 of the 2017 World Workshop on the Classification of Periodontal and Peri-Implant Diseases and Conditions. J Periodontol 89(S1):S74–S84 Wang Q, Amugo I, Rajakaruna H, Irudayam MJ, Xie H, Shanker A et al (2025) Evaluating GPT-5 for Melanoma Detection Using Dermoscopic Images. Diagnostics 15(23):3052 Liu Y, Deng G, Xu Z, Li Y, Zheng Y, Zhang Y et al (eds) (2024) 7/152024 A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering. The 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Physical Systems/Internet of Things; ; Porto de Galinhas Brazil: ACM Pranab Sahoo AKS, Saha S, Jain V, Mondal S (2024) Aman Chadha. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv Xiangyu Qi YZ, Xie T, Chen P-Y, Jia R, Mittal P, Henderson P (2023) Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! arxiv Meyerson J (2025) Perio Classification for the INBDE: Bootcamp.com; [Available from: https://bootcamp.com/blog/bootcamp-coms-perio-classification-for-the-inbde Ameli N, Firoozi T, Gibson M, Lai H (2024) Classification of periodontitis stage and grade using natural language processing techniques. PLOS Digit Health 3(12):e0000692 Vasey B, Novak A, Ather S, Ibrahim M, McCulloch P (2023) DECIDE-AI: a new reporting guideline and its relevance to artificial intelligence studies in radiology. Clin Radiol 78(2):130–136 DECIDE-AI (2021) new reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence. Nat Med 27(2):186–187 Von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP (2008) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol 61(4):344–349 Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8742868","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":583171894,"identity":"eec63023-4d29-4264-9f77-947565dc8db0","order_by":0,"name":"Ihunna Amugo","email":"","orcid":"https://orcid.org/0009-0006-4087-7415","institution":"Department of ODS \u0026 Research, School of Dentistry, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Ihunna","middleName":"","lastName":"Amugo","suffix":""},{"id":583171895,"identity":"09b68c0c-be45-4049-8160-8ef1fbf2c0a0","order_by":1,"name":"Katie L. Frederickson","email":"","orcid":"https://orcid.org/0009-0007-9318-6535","institution":"Department of Biochemistry, Cancer Biology, Neurosciences and Pharmacology, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Katie","middleName":"L.","lastName":"Frederickson","suffix":""},{"id":583171896,"identity":"4ed4f77e-cfca-475f-b2cf-659896eab1a2","order_by":2,"name":"Harshana Rajakaruna","email":"","orcid":"https://orcid.org/0009-0001-5264-2780","institution":"The Office for Research and Innovation, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Harshana","middleName":"","lastName":"Rajakaruna","suffix":""},{"id":583171897,"identity":"dbe9dcfd-2741-491a-b233-1d95e9b8f1f4","order_by":3,"name":"Hua Xie","email":"","orcid":"https://orcid.org/0000-0002-1712-3256","institution":"Department of ODS \u0026 Research, School of Dentistry, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Hua","middleName":"","lastName":"Xie","suffix":""},{"id":583171898,"identity":"b4a5cc2c-0a55-4c00-9954-ff7e8d00b8e0","order_by":4,"name":"Pandu Gangula","email":"","orcid":"https://orcid.org/0000-0001-7259-570X","institution":"Department of ODS \u0026 Research, School of Dentistry, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Pandu","middleName":"","lastName":"Gangula","suffix":""},{"id":583171899,"identity":"8dd57130-9181-48a5-8ecd-3ad30e8df591","order_by":5,"name":"Anil Shanker","email":"","orcid":"https://orcid.org/0000-0001-6372-3669","institution":"The Office for Research and Innovation, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":false,"prefix":"","firstName":"Anil","middleName":"","lastName":"Shanker","suffix":""},{"id":583171900,"identity":"de50f2ae-d6dc-4fe4-99d8-457822a1621a","order_by":6,"name":"Qingguo Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAArUlEQVRIiWNgGAWjYDACZjBpA+HwkKAljRQtEHCYBC18x3kMPxf8Oh+t236A8cHbNiK0SB7mMZae2Xc7d9uZBGbDucRoMTjMu0Gatweo5QYDmzQvkVo2/+btOQfSwv6bWC3bpHl+HADbwkyUFsnD/N+seRuSgX5JbJacc44ILXznjyXf5vljl7vt+OGDH96UEaGF4QAQM4Ldw9hAjHqoFoY/RCoeBaNgFIyCkQkATcQ5W8aIABkAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-5125-3724","institution":"Department of Biochemistry, Cancer Biology, Neurosciences and Pharmacology, Meharry Medical College, Nashville, TN 37208, USA","correspondingAuthor":true,"prefix":"","firstName":"Qingguo","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2026-01-30 15:47:10","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8742868/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8742868/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101789970,"identity":"9870cf26-e93d-4444-8632-ca93bbe6ff1b","added_by":"auto","created_at":"2026-02-03 16:02:50","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":5675,"visible":true,"origin":"","legend":"\u003cp\u003eData collection and case selection workflow.\u003c/p\u003e","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/de288bd8d17411427822ea2c.png"},{"id":101789971,"identity":"e37f2ef4-171e-42fb-83c2-82a6d2e2d47a","added_by":"auto","created_at":"2026-02-03 16:02:51","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":166618,"visible":true,"origin":"","legend":"\u003cp\u003eDemographic and clinical characteristics of 25 patients with periodontitis.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/9c1e07ac62cc8e40c9c78df4.png"},{"id":101881529,"identity":"87d9bdb4-5f2c-42ee-a127-9ff9ae140f97","added_by":"auto","created_at":"2026-02-04 15:13:00","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":233307,"visible":true,"origin":"","legend":"\u003cp\u003eFramework for evaluating GPT-5 performance in periodontitis staging and grading. (\u003cstrong\u003eA\u003c/strong\u003e) A prompt was constructed from a publicly available case[28], with minor formatting adjustments (e.g., line breaks and spacing) to improve readability for the model while preserving the original content. (\u003cstrong\u003eB\u003c/strong\u003e) GPT-5 generated categorical outputs for periodontitis stage (I–IV) and grade (A–C). (\u003cstrong\u003eC\u003c/strong\u003e) Model predictions were compared with the clinical reference diagnoses reported in the original publication to assess accuracy.\u003c/p\u003e","description":"","filename":"floatimage328.png","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/a5c91d0e7a567c414923e2ee.png"},{"id":101789972,"identity":"d96517ea-fda6-42f0-b195-711f86631ac4","added_by":"auto","created_at":"2026-02-03 16:02:51","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":102938,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrices of GPT-5 predictions for periodontitis staging and grading.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/04a4ec7d95e23c1f8b655a36.png"},{"id":101789973,"identity":"4e1115cd-15da-4acd-9792-653fc1e19842","added_by":"auto","created_at":"2026-02-03 16:02:51","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":116181,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of BERT, GPT-3.5, and GPT-5 performance in periodontitis staging and grading. Data are derived from Table 1.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/8fa4327333e4dcdfc412f2da.png"},{"id":101884109,"identity":"1efb58d2-60d7-4db6-afa1-c1e7f39d2fb9","added_by":"auto","created_at":"2026-02-04 15:30:27","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1103724,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8742868/v1/81547d40-f7ef-431c-b280-d6976ff8ed68.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eAn Exploratory Evaluation of GPT-5 in Periodontitis Staging and Grading Using Published Clinical Cases\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eAs the demand for accessible, accurate, and cost-effective resources to support dental care and education continues to grow, large language model (LLM)\u0026ndash;based chatbots, such as Chat Generative Pre-trained Transformer (ChatGPT)[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e], have emerged as promising tools. Although not originally developed for dental and healthcare applications, these systems can generate human-like responses with remarkable accuracy on many health-related topics [\u003cspan additionalcitationids=\"CR3 CR4 CR5 CR6 CR7\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], offering new opportunities for disease surveillance, biomedical research, and education.\u003c/p\u003e \u003cp\u003eCompared with traditional resources, LLMs offer distinct advantages for education and diagnostic support, including lower cost, continuous availability without the need for appointments, good accuracy for many diseases and conditions, customizable interactions, and user-friendly interfaces. As a result, people increasingly turn to them for medication information, self-diagnosis, and disease-prevention guidance[\u003cspan additionalcitationids=\"CR10 CR11\" citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Clinicians, dental and medical students also use them to acquire knowledge and support clinical decision-making[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eA growing body of research has examined LLMs\u0026rsquo; utilities in dental care and education[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan additionalcitationids=\"CR16 CR17\" citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. Several benchmark studies have demonstrated LLMs\u0026rsquo; competitive performance on the American Academy of Periodontology (AAP) in-service examination[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], the United States Medical Licensing Examination (USMLE)[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e], and other major assessments[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. In an educational study involving 77 second-year dental students, those who used LLMs for learning assignments were found to perform better on knowledge examinations than peers relying on traditional methods[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. Furthermore, Rahad \u003cem\u003eet al.\u003c/em\u003e showed that ChatGPT excelled in recognizing and correcting specialized dental terminology and achieved 66.7% accuracy in extracting and synthesizing information from documents [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. In the clinical context, Eroglu \u003cem\u003eet al.\u003c/em\u003e evaluated ChatGPT-3.5 on 200 untreated periodontitis cases and reported moderate performance for staging and grading[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eDespite these advances, critical gaps remain. Most prior studies used non-public datasets, limiting the reproducibility of their findings. Moreover, given the continual iteration and rapid improvement of LLMs, earlier assessments may not accurately reflect the capabilities of newer models such as Generative Pre-trained Transformer 5 (GPT-5), released in August 2025, whose performance in dentistry has not yet been systematically evaluated. Evaluation of LLMs in high-stakes dental clinical contexts is essential to establish quality-control mechanisms, mitigate risks of inaccurate or biased outputs, and guide their safe adoption into dental education and care.\u003c/p\u003e \u003cp\u003eAn important component of dental education and care is training students to diagnose and manage periodontitis, a chronic gum disease affecting approximately 42% of adults aged 30 and older in the US[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Periodontitis staging (I\u0026ndash;IV) reflects disease severity and extent based on levels of destroyed tissues, including gingival attachment and alveolar bone, while grading (A\u0026ndash;C) estimates the rate of progression and future risk[\u003cspan additionalcitationids=\"CR22\" citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. Even with explicit and standardized criteria for staging and grading[\u003cspan additionalcitationids=\"CR22\" citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], clinical diagnosis of periodontitis remains challenging and context-dependent, requiring careful integration of radiographic evidence, periodontal charting, and patient-specific risk factors. An LLM capable of accurately analyzing, staging, and grading clinical periodontitis cases could serve as both a valuable diagnostic aid for clinicians and a useful educational resource for students. To address this need, we conducted an exploratory evaluation of the performance of the newly released GPT-5 in staging and grading periodontitis cases.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eCase Identification and Data Collection\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eWe identified dental clinical cases by searching Google and PubMed in August and September 2025 using the keywords \u0026lsquo;periodontitis\u0026rsquo;, \u0026lsquo;staging\u0026rsquo;, and \u0026lsquo;grading.\u0026rsquo; These searches targeted peer-reviewed articles, case reports, and publicly available teaching materials that explicitly described periodontitis staging and grading according to established clinical criteria.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eAll records retrieved from the searches were screened manually. Review articles, duplicate records, and reports lacking complete staging or grading diagnoses, sufficient periodontal clinical descriptions, or adequate medical and dental histories were excluded. After screening, 25 cases were retained from a total of 52 identified records. The workflow for data collection and case selection is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eMost of the public cases collected included panoramic and periapical radiographs, which, together with patients\u0026rsquo; dental histories and periodontal charting, play a central role in diagnosis. In these cases, the radiographs had already been systematically evaluated by the original authors, and numerical measures of the bone loss were extracted and reported in their publications. We manually extracted these clinically meaningful data from the publications and used them directly in our assessments. Radiographic images themselves were not used in our analyses.\u003c/p\u003e \u003cp\u003eIt is important to note that publicly available and teaching-oriented cases, which are often unusually well documented and may not fully reflect the spectrum of routine clinical presentations, may introduce selection bias and limit generalizability. As a result, the GPT-5 performance observed in this exploratory study should be interpreted cautiously and may not reflect real-world diagnostic performance. These limitations are discussed in greater detail in the Discussion section.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eGPT-5 Prompting and Evaluation\u003c/h3\u003e\n\u003cp\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003eWhile GPT-5 can process multimodal data, we used only its interactive textual interface. All analyses were conducted between September 1st and 12th, 2025, using the free GPT-5 version, without any paid enhancements. GPT-5 was accessed through its publicly available interactive interface, which does not permit user control over system-level parameters such as temperature and system prompts. Therefore, all interactions were conducted using the default system configuration (temperature\u0026thinsp;=\u0026thinsp;1.0).\u003c/p\u003e\u003cp\u003eGiven GPT-5\u0026rsquo;s demonstrated accuracy and reasoning improvements over earlier models [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], and because the criteria for periodontitis staging and grading outlined in the 2017 World Workshop are strictly guideline-based [\u003cspan additionalcitationids=\"CR22\" citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], we adopted a zero-shot prompting strategy. This approach evaluates GPT-5\u0026rsquo;s ability to apply explicit clinical thresholds and decision logic without influence from exemplar conditioning. In contrast to few-shot prompting or fine-tuning [\u003cspan additionalcitationids=\"CR26\" citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e], which may introduce anchoring effects or label leakage and thereby inflate performance, zero-shot prompting provides a more conservative and transparent assessment of model reasoning.\u003c/p\u003e\u003cp\u003eUsing zero-shot prompting, case descriptions were submitted directly to the model without examples or prior instructions, and GPT-5 was asked to return predictions of periodontitis stage and grade. Before submission, case descriptions were lightly reformatted to correct line breaks, spacing, and formatting artifacts introduced during PDF extraction to improve readability for the model. No clinical content was added, removed, reworded, or reorganized, and the original meaning and structure of the source material were preserved. An example prompt for a clinical case and a detailed ChatGPT response are provided in the appendix.\u003c/p\u003e\u003cp\u003eTo assess response stability, each case was tested in multiple sessions using slight variations in prompt phrasing (e.g., \u0026ldquo;Can you help determine the periodontitis stage and grade of this patient?\u0026rdquo; or \u0026ldquo;Please stage and grade the periodontitis of this patient\u0026rdquo;). The model consistently generated identical predictions, regardless of prompt phrasing or session timing, indicating stable behavior at the decision level under these conditions. For the final analyses reported in this study, we used a standardized prompt, \u0026ldquo;Please stage and grade the periodontitis of this patient\u0026rdquo;, followed by the corresponding clinical case description.\u003c/p\u003e\u003cp\u003eData collection and analysis were conducted by four domain experts, all of whom are co-authors of this study. The team included a dental student, two professors from the School of Dentistry, and a professor of Data Science from the School of Medicine at Meharry Medical College. Their responsibilities included extracting clinical cases from publications, developing prompts for GPT-5, and verifying the model\u0026rsquo;s responses.\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\n\u003ch3\u003ePerformance Metrics\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eGPT-5 predictions were compared against published ground-truth diagnoses to calculate model accuracy and Cohen\u0026rsquo;s kappa. To characterize error patterns, confusion matrices were generated for both staging and grading to allow assessment of tendencies toward overestimation or underestimation. Given the limited dataset size (n\u0026thinsp;=\u0026thinsp;25) and the exploratory nature of this study, results are presented as descriptive performance metrics, without formal hypothesis testing. All data analyses were conducted in RStudio (v2025.09.2), and visualizations were generated using the R package ggplot2 (v4.3.3).\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eDescription of Periodontitis Cases\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eOf the 25 periodontitis cases collected for evaluating GPT-5, two were borderline cases with staging ambiguities between Stage III and Stage IV, and three provided only staging information in the original publications. Full case descriptions and corresponding sources are provided in the \u003cem\u003eAppendix\u003c/em\u003e.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e summarizes the cohort. The median age of these patients is 45 years, with most cases (68%) occurring between 35 and 64 years (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). Females comprised the majority (72%), while males accounted for 28% (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eB). In terms of disease severity, Stage III periodontitis was most common (56%), followed by Stage IV (36%), Stage II and I (4%) (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eC). For grading, most cases were classified as Grade C (77%), with smaller proportions in Grade B (23%) (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eD).\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eWorkflow for evaluating GPT-5\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. Framework for evaluating GPT-5 performance in periodontitis staging and grading. (\u003cb\u003eA\u003c/b\u003e) A prompt was constructed from a publicly available case[\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], with minor formatting adjustments (e.g., line breaks and spacing) to improve readability for the model while preserving the original content. (\u003cb\u003eB\u003c/b\u003e) GPT-5 generated categorical outputs for periodontitis stage (I\u0026ndash;IV) and grade (A\u0026ndash;C). (\u003cb\u003eC\u003c/b\u003e) Model predictions were compared with the clinical reference diagnoses reported in the original publication to assess accuracy.\u003c/p\u003e \u003cp\u003eThe prompt was then submitted to GPT-5, which was asked to determine the stage and grade of periodontitis. The model\u0026rsquo;s response was collected (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eB), recording only the predicted stage and grade while disregarding the diagnostic reasoning. The output was compared against the clinical diagnosis to evaluate accuracy (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eC).\u003c/p\u003e \u003cp\u003eAfter all cases were processed, GPT-5\u0026rsquo;s predictions were aggregated, and performance metrics were calculated to summarize its diagnostic accuracy and Cohen\u0026rsquo;s kappa and assess its potential utility in real-world clinical settings.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eGPT-5 performance in periodontitis staging and grading\u003c/h3\u003e\n\u003cp\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003eAcross the 25 periodontitis cases (including the two borderline cases), GPT-5 achieved 68.0% accuracy for staging (17/25 correct) and 77.3% accuracy for grading (17/22 correct, excluding three cases without specified grades). The corresponding Cohen\u0026rsquo;s kappa values were 0.432 for staging and 0.179 for grading, indicating fair agreement and poor agreement beyond chance, respectively.\u003c/p\u003e\u003cp\u003eThe confusion matrix in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eA shows that all Stage I and II cases were classified correctly (recall\u0026thinsp;=\u0026thinsp;100%), whereas recall was substantially lower for Stage III (57%) and Stage IV (75%). For grading, performance was markedly imbalanced, with high recall for Grade C (94%) but very low recall for Grade B (20%) (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eB). This class-dependent performance indicates that GPT-5 performs well for early-stage periodontitis and advanced disease detection but struggles to reliably distinguish intermediate disease categories.\u003c/p\u003e\u003cp\u003eImportantly, misclassifications exhibited nonrandom patterns: errors were confined to adjacent categories, with no Stage I or II cases misclassified as Stage III or IV, no Stage IV cases misclassified as Stage I or II (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eA), and no \u0026ldquo;skipping\u0026rdquo; across nonadjacent categories for grading (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eB). Because all errors were limited to adjacent stages and grades and no catastrophic misclassifications were observed, these patterns are clinically and educationally relevant.\u003c/p\u003e\u003cp\u003eAmong the misclassified cases, 43% of Stage III cases were predicted as Stage IV, while 25% of Stage IV cases were predicted as Stage III (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eA). For grading (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eB), GPT-5 correctly classified 94% of Grade C cases, with most Grade B cases misclassified as Grade C. These results suggest that GPT-5 tends to assign higher severity for both periodontitis stage and grade. GPT-5\u0026rsquo;s predictions for the 25 cases are provided in the \u003cem\u003eAppendix\u003c/em\u003e.\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e compare our findings with two prior assessments. Eroglu \u003cem\u003eet al.\u003c/em\u003e evaluated GPT-3.5, an earlier ChatGPT version, on 200 untreated patients and reported 59.5% accuracy in staging and 50.5% accuracy in grading[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]\u0026mdash;both notably lower than the performance achieved by GPT-5 on our dataset. The low kappa values observed for both models (0.284 for GPT-3.5 and 0.179 for GPT-5) in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e underscore ChatGPT\u0026rsquo;s limited discriminatory ability in grading periodontitis beyond chance agreement.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRecent studies of periodontitis staging and grading using textual input.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eDataset \u003csup\u003e+\u003c/sup\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c4\" namest=\"c3\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eCohen\u0026rsquo;s kappa\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eStudy ID\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eStage\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eGrade\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eStage\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eGrade\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e309 periodontal charts and clinical notes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBERT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e77.0%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e75.0%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNA\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNA\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAmeli \u003cem\u003eet al.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e200 untreated periodontitis patients\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGPT-3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e59.5%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e50.5%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.447\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.284\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eEroglu \u003cem\u003eet al.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e25 dental clinical cases from public sources\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGPT-5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e68.0%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e77.3%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.432\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.179\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eOur result\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003e \u003csup\u003e+\u003c/sup\u003e The findings from the first two studies were extracted from published papers, while the details of our own assessment results are provided in the \u003cem\u003eAppendix\u003c/em\u003e.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eAnother study summarized in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, conducted by Ameli \u003cem\u003eet al.\u003c/em\u003e [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e], fine-tuned a Bidirectional Encoder Representations from Transformers (BERT) model using 309 anonymized periodontal charts and corresponding clinician notes. The model was trained on 70% of the data and tested on 32 holdout cases. The fine-tuned BERT model achieved 77.0% accuracy in staging and 75.0% in grading[\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Although it outperformed GPT-5 in staging, it was slightly inferior in grading despite being specifically optimized for periodontitis. This comparison should be interpreted with caution, however, as BERT was evaluated on periodontal charts and clinician notes, whereas GPT-5 and GPT-3.5 were assessed using standardized textual case descriptions that included patient age, sex, and numerical measures of periodontitis-related parameters.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eLimitations\u003c/h2\u003e \u003cp\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003eOur exploratory evaluation of GPT-5 relied solely on published cases, which represent a relatively limited sample. Moreover, both the small sample size and the gender imbalance within the data may disproportionately reflect more severe or well-documented presentations, potentially inflating GPT-5\u0026rsquo;s performance. In addition, publicly available case reports are often curated to highlight clear diagnostic features and may not accurately reflect the full clinical heterogeneity or noise encountered in real-world practice. As a result, such cases may underrepresent diverse patient populations and disease presentations commonly seen in routine clinical settings. Expanding the dataset in future studies to include larger, more diverse, and non-published clinical data will therefore be essential to broaden the scope of evaluation, strengthen generalizability, and support meaningful clinical translation.\u003c/p\u003e\u003cp\u003eAdditional methodological constraints include our use of GPT-5\u0026rsquo;s interactive interface, which does not allow modification of underlying system parameters such as temperature, and hence may restrict reproducibility at the system level. Moreover, this study did not evaluate model stability across alternative prompting strategies, which are known to influence LLM behavior. Furthermore, only two borderline or equivocal cases were included, which are insufficient to assess GPT-5\u0026rsquo;s performance in diagnostically challenging scenarios where clinician disagreement is common and clinical judgment plays a critical role. Future studies should therefore incorporate more diagnostically ambiguous cases and systematically compare prompting strategies to better evaluate model reliability and clinical relevance.\u003c/p\u003e\u003cp\u003eAlthough this study focused narrowly on GPT-5\u0026rsquo;s performance in periodontitis staging and grading, the potential applications of LLMs extend more broadly to both clinical diagnostics and dental education. In clinical practice, LLMs could assist practitioners by consistently applying standardized staging and grading criteria, integrating charting and radiographic data, and generating preliminary assessments. In education, LLMs can function as personalized learning assistants, offering structured feedback on case analyses and helping students navigate diagnostic complexity. Beyond these applications, LLM-based chatbots hold promise for reducing gaps in dental educational resources, particularly in under-resourced institutions, thereby strengthening their capacity to deliver high-quality dental education and care.\u003c/p\u003e\u003cp\u003eHowever, the observed diagnostic agreement, particularly for periodontitis grading, highlights an important limitation. The low Cohen\u0026rsquo;s kappa (κ\u0026thinsp;=\u0026thinsp;0.179) for grading indicates poor agreement beyond chance, suggesting that GPT-5 currently lacks sufficient reliability to accurately distinguish between periodontitis grades. Consequently, GPT-5 is not yet suitable for independent clinical grading, and its outputs should be interpreted with caution. In addition, in the absence of established minimum clinically important difference (MCID) thresholds for AI-assisted periodontitis staging/grading, κ\u0026thinsp;=\u0026thinsp;0.432 for staging should be interpreted cautiously and not as evidence of clinical readiness.\u003c/p\u003e\u003cp\u003eLooking forward, GPT-5 and other LLMs are likely to continue to improve diagnostic performance. Nevertheless, its meaningful clinical translation will hinge on overcoming current deficiencies in reliability and consistency. Beyond advancements in model development, a potential pathway toward high-stakes clinical and educational applications likely lies in the integration of LLMs with validated AI tools optimized specifically for clinical use. Such hybrid systems, which combine the precision of specialized diagnostic models with the reasoning, interpretability, and interactivity of LLMs, may provide more robust support for complex, multimodal clinical decision-making in dental care and education.\u003c/p\u003e\u003cp\u003eFinally, this exploratory study was conducted with consideration of established AI reporting guidelines, including DECIDE-AI and STROBE-AI [\u003cspan additionalcitationids=\"CR31\" citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]. While the exploratory design and reliance on publicly available clinical cases, rather than real-time clinical data, precluded full adherence to all framework components, key principles such as transparency in data sources, model usage, limitations, and reproducibility were followed. Future prospective studies using large-scale, real-world clinical data will be better positioned to fully implement these reporting standards.\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusions","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eWith the growing use of LLMs by dental and medical students, clinicians, and the general public, it is important to evaluate their performance in high-stakes diagnostic and educational settings to inform safety protocols and guide responsible applications. In this study, we assessed GPT-5\u0026rsquo;s ability to stage and grade periodontitis\u0026mdash;tasks central to periodontal diagnosis and student training. Compared with prior models, GPT-5 demonstrated comparable or improved performance, achieving 68.0% staging accuracy and 77.3% grading accuracy, with a staging kappa of 0.432, indicating fair agreement beyond chance.\u003c/p\u003e \u003cp\u003eHowever, despite these performance gains, the low kappa for grading (0.179) underscores the very limited discriminatory capacity in distinguishing periodontitis grades, indicating that GPT-5 is not yet suitable for independent clinical applications. Additionally, our results revealed a consistent tendency for GPT-5 to overestimate disease severity. Therefore, inappropriate reliance on model outputs could increase the risk of overtreatment or unnecessary escalation of care. This underscores the importance of human oversight and the need for future evaluations of uncertainty reporting, refusal behavior, and decision-level safeguards before any clinical integration is considered.\u003c/p\u003e \u003cp\u003eIn conclusion, while GPT-5 demonstrated potential as a supportive tool for education and clinical exploration, it is not yet ready for autonomous use. Meaningful application in periodontal diagnosis and training will depend on substantial improvements in reliability and rigorous validation.\u003c/p\u003e \u003c/div\u003e "},{"header":"Abbreviations","content":"\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"524\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003eBERT\u003c/p\u003e\n \u003cp\u003eGPT\u003c/p\u003e\n \u003cp\u003eLLM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 452px;\"\u003e\n \u003cp\u003eBidirectional Encoder Representations from Transformers\u003c/p\u003e\n \u003cp\u003eGenerative Pre-trained Transformer\u003c/p\u003e\n \u003cp\u003eLarge language model\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe sources of our dental clinical cases are provided in the Appendix.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMultimedia Appendix 1\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eGPT-5\u0026rsquo;s performance in periodontitis staging and grading on the textual input of 25 clinical cases.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgments \u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe thank SeTonia Cook and Jacqueline Harding for helping with grant management.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis research was funded by the National Institute of Minority Health Disparities (NIMHD) under grant number U54MD007586, National Institute of Dental and Craniofacial Research (NIDCR) under grant number U01DE033241, National Institute of General Medical Sciences (NIGMS) under grant number R16GM149359, National Human Genome Research Institute (NHGRI) under award number UG3HG013248, National Institutes of Health (NIH) under Agreement Number 1OT2OD032581, Meharry\u0026apos;s American Cancer Society (ACS) under grant number DICRIDG-21-071-01-DICRIDG, and Chan Zuckerberg Initiative (CZI) grant under award number CZIF2022-007043. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the NIH, CZI, and Meharry Medical College.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInstitutional Review Board (IRB) Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIRB approval was not applicable, as the dental clinical cases used for assessment were obtained from publicly available sources.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInformed Consent Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eInformed consent was not required, as this study used only publicly available data and did not involve human participants.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDisclosure of Generative AI Use\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDuring the preparation of this manuscript, the authors used GPT-5 for minor editing and language polishing to improve the clarity of the English writing. All content was subsequently reviewed and revised by the authors, who take full responsibility for the final version of the publication.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of Interest\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNone declared.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eOpenAI GPT-5 System Card2025. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cdn.openai.com/gpt-5-system-card.pdf\u003c/span\u003e\u003cspan address=\"https://cdn.openai.com/gpt-5-system-card.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrin D, Sorin V, Vaid A, Soroush A, Glicksberg BS, Charney AW et al (2023) Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep 13(1):16492\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSinghal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW et al (2023) Large language models encode clinical knowledge. Nature 620(7972):172\u0026ndash;180\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEroglu ZT, Babayigit O, Sen DO, Yarkac FU (2024) Performance of ChatGPT in classifying periodontitis according to the 2018 classification of periodontal diseases. Clin Oral Invest 28(7):407\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRahad K, Martin K, Amugo I, Ferguson S, Curtis A, Davis A et al (2024) ChatGPT to Enhance Learning in Dental Education at a Historically Black Medical College. Dent Res Oral Health 7(1):8\u0026ndash;14\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmad B, Saleh K, Alharbi S, Alqaderi H, Jeong YN Artificial Intelligence in Periodontology: Performance Evaluation of ChatGPT, Claude, and Gemini on the In-service Examination. medRxiv [Internet]. 2024. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://dx.doi.org/10.1101/2024.05.29.24308155\u003c/span\u003e\u003cspan address=\"10.1101/2024.05.29.24308155\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepa\u0026ntilde;o C et al (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health 2(2):e0000198\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKatz U, Cohen E, Shachar E, Somer J, Fink A, Morse E et al (2024) GPT versus Resident Physicians \u0026mdash; A Benchmark Based on Official Board Scores. NEJM AI. ;1(5)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShahsavar Y, Choudhury A (2023) User Intentions to Use ChatGPT for Self-Diagnosis and Health-Related Purposes: Cross-sectional Survey Study. JMIR Hum Factors 10:e47564\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarley Presiado AM, Lopes L, Hamel L (2024) KFF Health Misinformation Tracking Poll: Artificial Intelligence and Health Information https://www.kff.org: The Henry J. Kaiser Family Foundation; [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.kff.org/health-information-and-trust/poll-finding/kff-health-misinformation-tracking-poll-artificial-intelligence-and-health-information/\u003c/span\u003e\u003cspan address=\"https://www.kff.org/health-information-and-trust/poll-finding/kff-health-misinformation-tracking-poll-artificial-intelligence-and-health-information/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKuroiwa T, Sarcon A, Ibara T, Yamada E, Yamamoto A, Tsukamoto K et al (2023) The Potential of ChatGPT as a Self-Diagnostic Tool in Common Orthopedic Diseases: Exploratory Study. J Med Internet Res 25:e47621\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDu D, Paluch R, Stevens G, M\u0026uuml;ller C Exploring patient trust in clinical advice from AI-driven LLMs like ChatGPT for self-diagnosis. arXiv [Internet]. 2024 2/2/2024. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://dx.doi.org/10.48550/arxiv.2402.07920\u003c/span\u003e\u003cspan address=\"10.48550/arxiv.2402.07920\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKisvarday S, Yan A, Yarahuan J, Kats DJ, Ray M, Kim E et al (2024) ChatGPT Use Among Pediatric Health Care Providers: Cross-Sectional Survey Study. JMIR Formative Res 8:e56797\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOzkan E, Tekin A, Ozkan MC, Cabrera D, Niven A, Dong Y (2025) Global Health care Professionals\u0026rsquo; Perceptions of Large Language Model Use In Practice: Cross-Sectional Survey Study. JMIR Med Educ 11:e58801\u0026ndash;e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShifai N, Van Doorn R, Malvehy J, Sangers TE (2024) Can ChatGPT vision diagnose melanoma? An exploratory diagnostic accuracy study. J Am Acad Dermatol 90(5):1057\u0026ndash;1059\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSattler SS, Chetla N, Chen M, Hage TR, Chang J, Guo WY et al (2025) Evaluating the Diagnostic Accuracy of ChatGPT-4 Omni and ChatGPT-4 Turbo in Identifying Melanoma: Comparative Study. JMIR Dermatology 8:e67551\u0026ndash;e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCirone K, Akrout M, Abid L, Oakley A (2024) Assessing the Utility of Multimodal Large Language Models (GPT-4 Vision and Large Language and Vision Assistant) in Identifying Melanoma Across Different Skin Tones. JMIR Dermatology 7:e55508\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePerlmutter JW, Milkovich J, Fremont S, Datta S, Mosa A (2025) Beyond the Surface: Assessing GPT-4's Accuracy in Detecting Melanoma and Suspicious Skin Lesions From Dermoscopic Images. Plast Surg\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKavadella A, Dias Da Silva MA, Kaklamanos EG, Stamatopoulos V, Giannakopoulos K (2024) Evaluation of ChatGPT\u0026rsquo;s Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study. JMIR Med Educ 10:e51344\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEke PI, Thornton-Evans GO, Wei L, Borgnakke WS, Dye BA, Genco RJ (2018) Periodontitis in US Adults: National Health and Nutrition Examination Survey 2009\u0026ndash;2014. J Am Dent Association 149(7):576\u0026ndash; \u0026ndash;\u0026thinsp;88.e6\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTonetti MS, Greenwell H, Kornman KS (2018) Staging and grading of periodontitis: Framework and proposal of a new classification and case definition. J Clin Periodontol 45(S20):S149\u0026ndash;S61\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCaton JG, Armitage G, Berglundh T, Chapple ILC, Jepsen S, Kornman KS et al (2018) A new classification scheme for periodontal and peri-implant diseases and conditions \u0026ndash; Introduction and key changes from the 1999 classification. J Periodontol 89(S1):S1\u0026ndash;S8\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChapple ILC, Mealey BL, Van Dyke TE, Bartold PM, Dommisch H, Eickholz P et al (2018) Periodontal health and gingival diseases and conditions on an intact and a reduced periodontium: Consensus report of workgroup 1 of the 2017 World Workshop on the Classification of Periodontal and Peri-Implant Diseases and Conditions. J Periodontol 89(S1):S74\u0026ndash;S84\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang Q, Amugo I, Rajakaruna H, Irudayam MJ, Xie H, Shanker A et al (2025) Evaluating GPT-5 for Melanoma Detection Using Dermoscopic Images. Diagnostics 15(23):3052\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu Y, Deng G, Xu Z, Li Y, Zheng Y, Zhang Y et al (eds) (2024) 7/152024 A Hitchhiker\u0026rsquo;s Guide to Jailbreaking ChatGPT via Prompt Engineering. The 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Physical Systems/Internet of Things; ; Porto de Galinhas Brazil: ACM\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePranab Sahoo AKS, Saha S, Jain V, Mondal S (2024) Aman Chadha. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXiangyu Qi YZ, Xie T, Chen P-Y, Jia R, Mittal P, Henderson P (2023) Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! arxiv\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMeyerson J (2025) Perio Classification for the INBDE: Bootcamp.com; [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://bootcamp.com/blog/bootcamp-coms-perio-classification-for-the-inbde\u003c/span\u003e\u003cspan address=\"https://bootcamp.com/blog/bootcamp-coms-perio-classification-for-the-inbde\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAmeli N, Firoozi T, Gibson M, Lai H (2024) Classification of periodontitis stage and grade using natural language processing techniques. PLOS Digit Health 3(12):e0000692\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVasey B, Novak A, Ather S, Ibrahim M, McCulloch P (2023) DECIDE-AI: a new reporting guideline and its relevance to artificial intelligence studies in radiology. Clin Radiol 78(2):130\u0026ndash;136\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDECIDE-AI (2021) new reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence. Nat Med 27(2):186\u0026ndash;187\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVon Elm E, Altman DG, Egger M, Pocock SJ, G\u0026oslash;tzsche PC, Vandenbroucke JP (2008) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol 61(4):344\u0026ndash;349\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[{"identity":"0ad8ea9f-a68f-4b56-86fc-2ea398357282","identifier":"10.13039/100006545","name":"National Institute on Minority Health and Health Disparities","awardNumber":"U54MD007586","order_by":0},{"identity":"77eac708-1b86-4e3d-ba43-bbc2e5614959","identifier":"10.13039/100000072","name":"National Institute of Dental and Craniofacial Research","awardNumber":"U01DE033241","order_by":1},{"identity":"369d1639-7fdc-4210-a3a4-2da0e0a122b7","identifier":"10.13039/100000057","name":"National Institute of General Medical Sciences","awardNumber":"R16GM149359","order_by":2},{"identity":"97085df4-5d6b-4b49-8e69-d08f94e5cb26","identifier":"10.13039/100000051","name":"National Human Genome Research Institute","awardNumber":"UG3HG013248","order_by":3},{"identity":"7ee0c0c8-4cd9-4981-a6eb-c9aab83b1f3d","identifier":"10.13039/100000002","name":"National Institutes of Health","awardNumber":"1OT2OD032581","order_by":4}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Meharry Medical College","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"large language model, ChatGPT, GPT-5, dental care, dental education, periodontitis, periodontitis staging and grading","lastPublishedDoi":"10.21203/rs.3.rs-8742868/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8742868/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003ePeriodontitis is a chronic gum disease affecting approximately 42% of adults aged 30 and older in the United States. Training dental students to accurately diagnose and manage periodontitis is a critical component of dental education and clinical care. Recent advances in large language models (LLMs) offer new opportunities to support both domains, yet their performance in periodontal diagnosis remains largely unexplored\u0026mdash;particularly for newer models such as Generative Pre-trained Transformer 5 (GPT-5).\u003c/p\u003e\u003ch2\u003eObjective\u003c/h2\u003e \u003cp\u003eThis study conducted an exploratory evaluation of GPT-5\u0026rsquo;s ability to stage and grade periodontitis.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eTwenty-five publicly available clinical cases were identified through Google and PubMed searches. Each case description was entered into GPT-5 using a zero-shot prompting approach, and the model\u0026rsquo;s predictions were compared with the published reference diagnoses. Performance was measured using accuracy and Cohen\u0026rsquo;s kappa.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eAcross these cases, GPT-5 showed marked class-dependent performance and a tendency to overestimate disease severity. Compared with prior models, it achieved comparable or improved performance, with accuracies of 68.0% for staging and 77.3% for grading and corresponding Cohen\u0026rsquo;s kappa values of 0.432 and 0.179, respectively. While staging performance showed fair agreement beyond chance, the low kappa for grading indicates poor agreement and limited reliability in distinguishing periodontal disease severity.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThese findings suggest that although GPT-5 shows improvement over previous models, its current diagnostic performance, particularly for periodontitis grading, limits its utility in clinical assessment and educational training. Meaningful application in periodontal diagnosis and training will require substantial improvements in reliability and rigorous validation. The limitations of the study and implications for future development are also discussed.\u003c/p\u003e","manuscriptTitle":"An Exploratory Evaluation of GPT-5 in Periodontitis Staging and Grading Using Published Clinical Cases","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-03 16:02:46","doi":"10.21203/rs.3.rs-8742868/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2dfb3d19-7350-4f37-9aba-b4b6281e1dbb","owner":[],"postedDate":"February 3rd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62048127,"name":"Dentistry"},{"id":62048128,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2026-02-03T16:02:46+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-03 16:02:46","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8742868","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8742868","identity":"rs-8742868","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}