Expert-Based Evaluation of ChatGPT for Removable Partial Denture Design: Accuracy and Reliability Analysis

Expert-Based Evaluation of ChatGPT for Removable Partial Denture Design: Accuracy and Reliability Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Expert-Based Evaluation of ChatGPT for Removable Partial Denture Design: Accuracy and Reliability Analysis Ebru ARSLAN, Simge Alıcı, Busemin Kesgin, Selim Erkut, Caner İncekaş This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8487824/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Objective This study evaluated the accuracy and reproducibility of ChatGPT-5 in designing removable partial dentures (RPDs) for partially edentulous arches and investigated whether the inclusion of Kennedy classification improves the clinical validity of the generated treatment plans. Materials and Methods Twenty standardized partially edentulous scenarios (10 maxillary and 10 mandibular) were presented to ChatGPT-5 under two prompting conditions: ( 1 ) describing only the dental chart (tooth configuration), and ( 2 ) including the specific Kennedy classification. To assess reproducibility, prompts were submitted across three separate sessions (N = 240 responses). Two experienced prosthodontists evaluated the outputs (major connector and direct/indirect retainers) using a 3-point Likert scale. Inter-rater reliability was calculated using Gwet’s AC1 and percent agreement. Results A total of 240 evaluations were analyzed by averaging the scores of two prosthodontic experts across all case scenarios. Intra-rater reliability demonstrated moderate to substantial agreement for both experts, with Gwet’s AC1 values ranging from 0.637 to 0.733 and percent agreement between 0.733 and 0.800. Without Kennedy classification, correct response rates in the mandible ranged from 41.7% to 60.0%, whereas higher accuracy was observed in the maxilla (45.0%–76.7%). With Kennedy classification, mandibular accuracy increased across all components (major connectors: 55.0%; direct retainers: 76.7%; indirect retainers: 66.7%), while maxillary accuracy remained stable or decreased (50.0%–68.3%). Conclusions While ChatGPT shows promise as a supportive educational tool, its performance is highly sensitive to prompt engineering and anatomical context. The inclusion of Kennedy classification improves precision in mandibular cases but may introduce conflicting constraints in maxillary planning. Therefore, AI-generated plans currently require strict expert validation before clinical application. Artificial intelligence ChatGPT removable partial denture Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8487824","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":578254376,"identity":"bad2215b-a179-4858-9aaa-aaafc377ff46","order_by":0,"name":"Ebru ARSLAN","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABSklEQVRIie3RMUvDQBTA8RcDr0ti1gtR8xUSCpaCpV8lIXAuWVwkg2hAeJ2qa8D2O3R0rATOpThX6BCXuLSgi1SMYqodTC3F0eH+Sx4cv9xxByCT/cfY8rsNqGQA3mJGWA4A2gaCgKpTJfhN2AaC7E/EsLq5pZ1MbKwJHr1ec7vRGWI2L/hew46VbEZw2ogrxOzd7luayF3SuLjvjkK3N/JqbpfCepNAdfsEbGdYIc6Yo6VhqhAc0linSEnAQ6bHkT8QgJZekpWTtb/IR9om45GO3ilqJ0aGZlFEZyWpva0hDuOLX6U+MS5UnUI/Yd5i39Bzyl3UNYRNhNrsX+QBsTywdokHCXvo1MvBHQj/3OzdMTNZubErUsbTl0nr0uDu84yCVmIE4mFaBLaTpjdP0+MD49fDAGxpULkTJf45rAHlyrxKZDKZTFbtEw7YbxqaClAuAAAAAElFTkSuQmCC","orcid":"","institution":"Başkent University","correspondingAuthor":true,"prefix":"","firstName":"Ebru","middleName":"","lastName":"ARSLAN","suffix":""},{"id":578254377,"identity":"160d95b8-f249-4ede-a56e-90a81df1647d","order_by":1,"name":"Simge Alıcı","email":"","orcid":"","institution":"Başkent University","correspondingAuthor":false,"prefix":"","firstName":"Simge","middleName":"","lastName":"Alıcı","suffix":""},{"id":578254378,"identity":"90c4ab22-7b0b-4a4b-befd-5c20450a83ad","order_by":2,"name":"Busemin Kesgin","email":"","orcid":"","institution":"Başkent University","correspondingAuthor":false,"prefix":"","firstName":"Busemin","middleName":"","lastName":"Kesgin","suffix":""},{"id":578254379,"identity":"17a895de-3183-475e-8085-6aa9b9e09ce3","order_by":3,"name":"Selim Erkut","email":"","orcid":"","institution":"Başkent University","correspondingAuthor":false,"prefix":"","firstName":"Selim","middleName":"","lastName":"Erkut","suffix":""},{"id":578254380,"identity":"7a7dcf29-c602-4ff1-96c1-3d5540336915","order_by":4,"name":"Caner İncekaş","email":"","orcid":"","institution":"Başkent University","correspondingAuthor":false,"prefix":"","firstName":"Caner","middleName":"","lastName":"İncekaş","suffix":""}],"badges":[],"createdAt":"2025-12-31 08:38:50","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8487824/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8487824/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101006084,"identity":"5dfee752-6de2-4178-9724-8f0d9ddb91fa","added_by":"auto","created_at":"2026-01-23 17:51:28","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":129405,"visible":true,"origin":"","legend":"","description":"","filename":"ChatgptHbpson2.docx","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1/439ccc67d2b6868242cb3fc6.docx"},{"id":101006081,"identity":"7f70a61f-408a-46d3-a822-47361b6cf4cf","added_by":"auto","created_at":"2026-01-23 17:51:28","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":7170,"visible":true,"origin":"","legend":"","description":"","filename":"7baef6e4968646acbace361aeefe753b.json","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1/41618be98e445a91eab8ebd5.json"},{"id":101204306,"identity":"e49d2c4c-158a-453f-bf3b-1804aa8cb2e8","added_by":"auto","created_at":"2026-01-27 09:42:36","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":71022,"visible":true,"origin":"","legend":"","description":"","filename":"7baef6e4968646acbace361aeefe753b1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1/f21e7f8fd533d2a42d52390e.xml"},{"id":101006083,"identity":"afd9746c-368e-4cbb-b50c-8403d898d251","added_by":"auto","created_at":"2026-01-23 17:51:28","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":69551,"visible":true,"origin":"","legend":"","description":"","filename":"7baef6e4968646acbace361aeefe753b1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1/854d4245505f999499a78b36.xml"},{"id":101006085,"identity":"2a5b00a3-0118-4861-ad11-b8d0b6b723ba","added_by":"auto","created_at":"2026-01-23 17:51:28","extension":"html","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":80665,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1/a58377ae2d01790159dae8d8.html"},{"id":109488280,"identity":"ae734ccb-e9bd-4f83-a2c4-eea97d9e161b","added_by":"auto","created_at":"2026-05-18 16:55:52","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":311492,"visible":true,"origin":"","legend":"","description":"","filename":"ChatgptHbpson2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8487824/v1_covered_1ca31aac-2bcb-4fce-b038-01f7ab6225c7.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Expert-Based Evaluation of ChatGPT for Removable Partial Denture Design: Accuracy and Reliability Analysis","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Artificial intelligence, ChatGPT, removable partial denture","lastPublishedDoi":"10.21203/rs.3.rs-8487824/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8487824/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eObjective\u003c/h2\u003e \u003cp\u003eThis study evaluated the accuracy and reproducibility of ChatGPT-5 in designing removable partial dentures (RPDs) for partially edentulous arches and investigated whether the inclusion of Kennedy classification improves the clinical validity of the generated treatment plans.\u003c/p\u003e\u003ch2\u003eMaterials and Methods\u003c/h2\u003e \u003cp\u003eTwenty standardized partially edentulous scenarios (10 maxillary and 10 mandibular) were presented to ChatGPT-5 under two prompting conditions: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) describing only the dental chart (tooth configuration), and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) including the specific Kennedy classification. To assess reproducibility, prompts were submitted across three separate sessions (N\u0026thinsp;=\u0026thinsp;240 responses). Two experienced prosthodontists evaluated the outputs (major connector and direct/indirect retainers) using a 3-point Likert scale. Inter-rater reliability was calculated using Gwet\u0026rsquo;s AC1 and percent agreement.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eA total of 240 evaluations were analyzed by averaging the scores of two prosthodontic experts across all case scenarios. Intra-rater reliability demonstrated moderate to substantial agreement for both experts, with Gwet\u0026rsquo;s AC1 values ranging from 0.637 to 0.733 and percent agreement between 0.733 and 0.800. Without Kennedy classification, correct response rates in the mandible ranged from 41.7% to 60.0%, whereas higher accuracy was observed in the maxilla (45.0%\u0026ndash;76.7%). With Kennedy classification, mandibular accuracy increased across all components (major connectors: 55.0%; direct retainers: 76.7%; indirect retainers: 66.7%), while maxillary accuracy remained stable or decreased (50.0%\u0026ndash;68.3%).\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eWhile ChatGPT shows promise as a supportive educational tool, its performance is highly sensitive to prompt engineering and anatomical context. The inclusion of Kennedy classification improves precision in mandibular cases but may introduce conflicting constraints in maxillary planning. Therefore, AI-generated plans currently require strict expert validation before clinical application.\u003c/p\u003e","manuscriptTitle":"Expert-Based Evaluation of ChatGPT for Removable Partial Denture Design: Accuracy and Reliability Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-23 17:51:23","doi":"10.21203/rs.3.rs-8487824/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d7b962d2-25a8-4d86-a420-1f233115173d","owner":[],"postedDate":"January 23rd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-18T16:55:44+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-23 17:51:23","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8487824","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8487824","identity":"rs-8487824","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}