Assessment of the credibility of generative artificial intelligence large language models on nutritional guidelines for orthodontic patients

Assessment of the credibility of generative artificial intelligence large language models on nutritional guidelines for orthodontic patients | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Assessment of the credibility of generative artificial intelligence large language models on nutritional guidelines for orthodontic patients Tevhide Sokmen, İrem Kar, Cumhur Tuncer, Burcu Balos Tuncer This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6792273/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 26 Nov, 2025 Read the published version in BMC Oral Health → Version 1 posted 12 You are reading this latest preprint version Abstract Background To evaluate the credibility of large language models (LLMs) compared to American Association of Orthodontists (AAO) and British Orthodontic Society (BOS) guides regarding nutritional guidelines for orthodontic patients. Methods The responses offered by ChatGPT 4.0, Copilot and Gemini were assessed for information credibility about tooth decay, food, beverages, oral care, and further assistance as; compatible, compatible but insufficient, partially incompatible and incompatible compared to the guidelines. Reliability was analyzed by the DISCERN tool. The readability of the sources was assessed using the Flesch Reading Ease Score and the Flesch–Kincaid Grade Level. The Friedman test was conducted to compare DISCERN scores. Results Responses of LLMs were understandable and compatible with the guidelines, but detailed information on tooth decay, dental plaque and risks of acidic environments were inadequate. ChatGPT 4.0, Copilot and Gemini provided detailed lists of foods to avoid and include. Only AAO suggested being aware of extreme temperatures and usage of sugar-free gums. All sources mentioned the necessity of good oral hygiene, but oral hygiene tools were not mentioned in Copilot. All, except ChatGPT 4.0, recommended orthodontist consultation for personalized advice. BOS leaflet had the highest mean DISCERN score (4.70 ± 0.27), followed by Gemini (4.54 ± 1.03), AAO web-source (4.45 ± 0.75), Copilot (3.87 ± 1.64) and ChatGPT 4.0 (3.08 ± 1.65), revealing no significant difference. BOS and AAO were more readable than the LLMs. ChatGPT 4.0 was more readable among the LLMs but was still found to be difficult for the readers. Conclusion Guidelines have a superior narrative in terms of their detailed content and especially the justifications for the recommendations. AI-supported LLMs provided understandable, simple and accurate information, despite lack of some details on certain topics. The readability of the responses from LLMs was difficult. Overall, patients should be advised that pre-trained algorithms should be used with caution as a source of information and that they should receive individual information from their orthodontists. Orthodontics nutrition artificial intelligence large language models (LLMs) ChatGPT Copilot Gemini Leaflet Figures Figure 1 Introduction The cooperation of patients is important during fixed orthodontic treatment in terms of maintaining oral hygiene, achieving treatment targets, and adhering to the recommended nutritional changes [ 1 ]. Orthodontists advise certain dietary restrictions/modifications at the beginning of treatment. According to Carter et al. [ 2 ] patients reported limited food choice due to dietary advice given by their orthodontist, and fear of breakage. Choi et al. [ 3 ] revealed that orthodontic treatment changed patients eating behaviors, and may cause eating problems. The patients are advised to minimize consumption of sugary and sticky food to lower the risk of caries and white spot lesions [ 4 – 6 ]. Again, the patients are instructed to prefer softer food in order not to harm the appliances [ 7 ]. The British Orthodontic Society (BOS) [ 8 ], and American Association of Orthodontists (AAO) [ 9 ] have prepared guidelines to assist orthodontic patients with their food choices. Technological advancements changed lifestyles, demands, and habits of information seeking, mainly among younger generations. This has led to the search for information turning primarily to internet browsing for almost every information. Crispino et al. [ 10 ] investigated the relevance of web-based information for orthodontic patients, and reported that, most use Internet as a source for orthodontic information. They also reported that patients who perceived web-sourced information as useful did not easily ignore this information, even if it was contrary to the orthodontist's opinions. By the end of 2022, a revolutionary advancement in artificial intelligence (AI), known as Chat Generative Pre-Trained Transformer (ChatGPT), was developed by OpenAI Inc. (San Francisco, CA, USA). Within the initial three months post-launch, it gained over 100 million new users [ 11 ]. In February 2023, Microsoft Corporation (Redmond, WA, USA) released the Bing Chat AI chatbot utilizing the GPT-4 language model, which is now being renamed as Copilot, following the success of Chat GPT. In March 2023, Google (Google Ireland Limited, Dublin, Ireland) introduced the Google Bard language model, first utilizing LaMDA (Language Model for Dialogue Applications) and subsequently driven by the PaLM 2 LLM (Pathways Language Model 2), which is presently being reintroduced as Gemini [ 12 ]. These models and their corresponding updates have various distinguishing characteristics. ChatGPT's responses derive from pre-existing training data, whereas Gemini utilizes real-time Internet connection to integrate current information in its response generation [ 13 ]. The daily use of AI chatbots is gaining popularity among healthcare professionals and patients as a useful tool for medical and dental information, due to advances in technology [ 14 , 15 ]. AI chatbots provide patients with the ability to access information at any time, as well as rapid responses to questions and the reduction of the need for appointments [ 16 ]. However, concerns about the credibility of the information of AI chatbots have been highlighted [ 17 ]. Recently, the need for further model development was emphasized to increase ChatGPT’s accuracy for medical queries [ 18 ]. Similarly, ChatGPT answers were declared as comprehensive and understandable in assessment of obesity in type 2 diabetes, but the need of patient-centered approach verified by up-to-date and reliable sources was reported [ 19 ]. A recent study in the medical field assessing the responses of ChatGPT-3.5, Gemini, and Copilot regarding a specific topic in orthopedic surgery found that the answers from the language models were highly complex, and no single resource emerged to be superior [ 20 ]. The evaluation of the reliability of ChatGPT responses related to orthodontics showed that the responses were not scientific due to the lack of specified peer-reviewed references [ 17 ]. Abu Arqub et al. [ 21 ] demonstrated that overall accuracy of ChatGPT-3.5 responses to inquiries about clear aligners is insufficient. In a study assessing the quality of information provided in four different LLMs about clear aligners declared that, more evidence-based information is needed and readability should be improved [ 14 ]. Moreover, a recent study in orthodontics concluded that current LLMs are not a consistently reliable source of evidence for either patients or clinicians seeking information online [ 12 ]. With the emphasis on nutritional issues at the beginning of treatment, patients who seek orthodontic treatment can search for information on the internet, so it is important to assess the credibility of online information sources. To the best of our knowledge, there is no publication examining this topic in the literature. Therefore, this study intended to be the first to evaluate the credibility of the different LLMs compared to the current nutritional guidelines for orthodontic patients. The null hypothesis of the study was that there was no difference in the reliability and readability of information conveyed by three different LLM resources relative to AAO and BOS guides in terms of nutritional emphasis for orthodontic patients. Materials and Methods Ethical approval was not relevant, as this study required no human/animal materials. The credibility of ChatGPT 4.0, Gemini and Copilot responses were assessed regarding AAO web-source [ 9 ] and the BOS leaflet [ 8 ], to provide an objective evaluation. ChatGPT 4.0 responses were achieved on December 23, 2024. Additionally, responses from Gemini and Copilot were obtained both on March 21, 2025. By using "New Chat" function, “W hat are the nutritional guidelines for orthodontic patients? ” was asked. Three orthodontists (TS, CT, BBT) evaluated information credibility about tooth decay, food, beverages, oral care, and further assistance through the following classification [ 19 ]: 1. Compatible with the guidelines: contains information from the guideline on the subject, with no missing or contradictory information. 2. Compatible with the guidelines but insufficient: contains information from the guideline on the subject, with some missing information but no contradictory information. 3. Partially incompatible with the guidelines: contains information from the guideline on the subject, with some contradictory information. 4. Incompatible with the guidelines: contains information on the subject contradictory to the guidelines. The reasons for insufficient responses were noted. Further, scores for reliability were assessed by the same researchers (TS, CT, BBT) using the modified DISCERN instrument, which is a valid tool for assessing the quality of health-related written or online-information ( www.discern.org.uk ). The instrument composed of 3 parts and 16 questions using a 1–5 rating scale, where 1 indicates total rejection and 5 indicates absolute agreement [ 22 ]. In this study, the questions in the first part regarding reliability (questions 1–8) were rated in accordance with the instructions of the DISCERN tool. The readability of each source was assessed using the Flesch Reading Ease instrument (FRES) [ 23 ], and Flesch-Kincaid Grade Level (FKGL) [ 24 ]. The clarity of a document is represented by a numerical value ranging from 0 to 100 in the Flesch Reading Ease score. Scores near “100” show that the material is highly simple to read, while scores near “0” indicate it is quite complex and challenging to comprehend [ 23 ]. The complete text from the AI chatbots and websites were copied into a Word (Microsoft Office, Version 16.78.0) document. The text was subsequently loaded into an online tool “ https://readable.com ” and tested for readability of sources. The software's scores were analyzed using the data previously supplied in research [ 17 ] and shown in Table 1 . Table 1 The evaluation of the reading levels according to Flesch Reading Ease Score. Flesch Reading Ease Score Reading Level Estimated reading grade level 0–29 Very difficult Graduated college 30–49 Difficult Attended college 50–59 Fairly difficult High school (10th to 12th) 60–69 Standard and/or plain Eighth to ninth 70–79 Fairly easy Seventh grade 80–89 Easy Fifth to sixth grade 90–100 Very easy Fourth to fifth grade Statistical analysis Five resources as AAO web-source, BOS leaflet, ChatGPT-4.0, Copilot, and Gemini were evaluated based on their responses to eight structured items. Each response was rated on a 1 to 5 Likert scale by three independent raters. Given the ordinal nature of the data and the small sample size ( n = 8 per system), nonparametric statistical methods were employed. Inter-rater reliability was assessed for each system. High agreement was observed among raters for ChatGPT-4, Copilot, and Gemini. Moderate agreement was found for AAO, while inter-rater reliability could not be calculated for BOS due to the restricted range of scores (only ratings of 4 and 5 were assigned). Nonetheless, visual inspection suggested consistent scoring across raters for BOS. For each system, the mean score across the three raters was calculated. A Friedman test was then performed to assess whether there were statistically significant differences in mean ratings among the five systems. In the presence of overall significance, pairwise comparisons were conducted using Wilcoxon signed-rank tests with Bonferroni correction to control for the risk of type I error. All analyses were conducted using IBM SPSS Statistics version 30. Statistical significance was set at a threshold of p < 0.05. Results Responses of all LLM resources were understandable, and compatible with no contradictory information. The credibility of information resources is shown in Table 2 (Table 2 was placed at the end of the document text file). General recommendations on nutritional guidelines for orthodontic patients were mentioned, but some details were missing as follows: Table 2 Assessment of the responses of the guidelines and language models on nutritional guidelines of orthodontic patients. Resources Information on tooth decay Information on food Information on beverages Information on oral care Information on further assistance AAO Swish and rinse ; after eating swishing water around your mouth is beneficial to dislodge any food particles stuck in or around your braces. This prevents plaque build-up and staining. Stay hydrated; Drinking water throughout the day keeps you hydrated and helps keep your mouth clean. It assists in rinsing away food debris and acidic residues. Food choices with braces First few days with braces Embracing soft foods; consume soft, gentle foods for first days with braces: oatmeal, scrambled eggs, soups, pasta, seafood, soft vegetables, mashed potatoes, soft cheeses, bananas, yogurt, ice cream (cold comforts and relieves some pressure of the wires) Foods to approach with caution initially; Be careful with chewy breads, rolls, meat Foods to avoid ; hard-sticky candies, biting whole apples, corn on the cob, popcorn, nuts, crunchy veggies-fruits, pizza crust, bagels or hard rolls, ice, snack chips, pretzels (may cause loosing braces, wires and retard treatment) Essential tips for eating with braces (cooking/steaming/slicing for softer foods, preferring small pieces, careful chewing, oral care tools, attention to temperature, sugar-free gum to handle soreness), avoid extreme temperature (to prevent tooth sensitivity especially at early days of braces) Avoid extreme temperatures; Be wary of foods or drinks that are too hot or cold (extreme temperatures can heighten tooth sensitivity, especially during the early days of braces) Stay hydrated; Drinking water throughout the day keeps you hydrated and helps your mouth clean Swish and rinse; After eating, swishing water around mouth is beneficial to dislodge any food particles around braces, that prevents plaque build-up Invest in the right tools; Carry travel toothbrush, or water flosser. Brushing, flossing (especially after meals) Stay hydrated; Helps keeping mouth clean and rinse away food debris and acidic residues Listen to your orthodontist; Your orthodontist will provide specific guidelines about avoiding foods and ways to take care of your braces, their advice stems from years of experience and knowledge, so it is essential to follow their recommendations Take the next step; find a dedicated AAO orthodontist near you and get the expert care you deserve, visit our orthodontist locator today BOS A brief definition is provided answering “What is tooth decay?” When you eat or drink particularly sugary items, the bacteria (plaque) in your mouth produce acid. This acid can make holes in your teeth (tooth decay). Your teeth will be under attack from this plaque acid for up to one hour after eating food or drinking sugary drinks. Retained food and bacteria (plaque) may lead to permanent and unsightly marks on teeth and tooth decay, especially sugary items produce acid for up to one hour after eating food or drinking sugary drinks, which can make holes in teeth (tooth decay). Fruit juices should be kept to mealtimes only. Diet and braces; eating wrong foods can break your braces/damage your teeth, treatment will be lengthened Do I need to change my eating habits; choose softer foods when braces are fitted/adjusted for few days: soup, pasta dishes, yoghurts, cheese; cut up rather than bite Can I eat snacks? Eat 3 meals as healthier What food and drinks can I eat between meals? Select healthy foods as whole meal bread, plain cheese, dry crackers, breadsticks, fruits, vegetables, plain milk, plain water Food and drinks to avoid; hard, sticky, crunchy foods (toffee, caramel, lollies; consume buy cutting apples, raw carrot, pizza crust, nuts, biscuits, celery, chicken wings, ribs; chewing them with the back teeth), fizzy, acidic/sugary foods and drinks Fruit juices only at mealtimes using a straw, minimum chocolates/ sweets (only at meal times) Drinks to avoid (fizzy, soft, acidic drinks can cause damage) Select plain milk or plain water (not fizzy) Limit fruit juice to mealtimes, drink with a straw Brush your teeth; for 2 minutes at least twice a day with a fluoride toothpaste If you have any further questions; please ask your orthodontist, dentist or hygienist. Free links : British Dental Health Foundation website www.dentalhealth.org.uk NHS Oral Health Specialist Library website www.library.nhs.uk/oralhealth/ For more information about Orthodontics, please visit: www.bos.org.uk ChatGPT 4.0 Sugary foods and drinks increase the risk of cavities and decalcification around brackets (soda, candy, cookies, sugary juices) Drink water frequently , helps wash away food debris and neutralize acids. Minimize snacking, reduces the number of times teeth are exposed to sugars and acids. Foods to avoid; hard foods (nuts, hard candies, popcorn, raw carrots, ice), sticky foods (chewing gum, caramel, taffy, gummy candies), sugary foods/drinks (soda, candy, cookies, sugary juices), chewy foods (bagels, tough bread, jerky), foods to bite into (cut into smaller pieces apples, corn on the cob, whole pears) Foods to include; soft foods (mashed potatoes, yogurt, pasta, scrambled eggs, soups), cut/cooked vegetables (steamed broccoli, roasted zucchini, thinly sliced cucumbers), dairy products (cheese, milk, yogurt), protein-rich foods (soft-cooked chicken, fish, tofu, eggs), fruits (bananas, berries, applesauce, melons), whole grains (oatmeal, soft bread, rice) Avoid sugary drinks; soda, sugary juices (risk of cavities, decalcification) Drink water frequently (helps wash away food debris and neutralize acids) Oral care tips Brush and floss after every meal; food particles can easily get trapped in braces Use orthodontic tools; interdental brushes and water flossers help cleaning around braces Drink water frequently ; helps wash away food debris and neutralize acids Minimize snacking; reduces the number of times teeth are exposed to sugars and acids Not existed ChatGPT 4.0 credibility Compatible but insufficient : did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment Compatible Compatible Compatible Incompatible Copilot Stay hydrated with water; Drinking plenty of water not only aids digestion but also maintain saliva flow, essential for neutralizing acids that can cause tooth decay. Limit sugary and acidic foods and beverages; Sugar and acid can increase the risk of cavities and since braces make cleaning teeth more challenging, minimizing these foods is crucial. Emphasize soft foods after adjustments; soring may occur for a few days, eat soft, easily chewable foods (mashed potatoes, soups, yogurts, smoothies, scrambled eggs) Prioritize a balanced diet rich in essential nutrients; calcium-vitamin D (milk, cheese, yogurt, leafy greens, fortified foods), lean proteins (tender chicken, tofu, fish, eggs), fruits-vegetables (bananas, melons, berries, steamed carrots, zucchini, broccoli), whole grains (oatmeal, soft pasta, rice) Avoid hard-sticky-chewy foods ; nuts, hard candy, gum, caramel, raw crunchy vegetables/fruits (cut into smaller pieces, cook/chop) Cut foods into bite-sized pieces reduces the strain on your braces and chewing muscles Limit sugary and acidic foods/beverages Stay hydrated with water; Drinking plenty of water Limit sugary and acidic foods and beverages; Sugar and acid can increase the risk of cavities and since braces make cleaning teeth more challenging, minimizing these foods is crucial. Stay hydrated with water; Water helps rinse away small food particles that might get trapped in your braces Limit sugary and acidic foods and beverages; Being mindful of consumption and practicing through dental hygiene after sugary and acidic foods and beverages is key Orthodontists and nutritionists can offer personalized advice as your treatment progresses, so be sure to share any concerns you have during your visits Copilot credibility Compatible but insufficient : did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment Compatible Compatible Compatible but insufficient Oral hygiene tools were not mentioned Compatible Gemini Foods to Avoid Sugary and Acidic Foods; Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices. Expand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. www.colgate.com ) Balanced Diet; A well-rounded diet rich in essential vitamins and minerals is crucial for overall oral health and supports the effectiveness of orthodontic treatment. Expand/Google Search found similar content as follows: (A balanced diet with adequate nutrients is essential for a healthy mouth and in turn, a healthy mouth supports nutritional well-being. nutritionsource.hsph.harvard.edu) Focus on fruits, vegetables, lean proteins, and dairy products. Expand (Consuming a diet high in fruits, vegetables, dairy products, and lean proteins gives you the vitamins and minerals your teeth need to be healthy. www.batemanorthodontics.com ) Foods to Enjoy Soft Foods; Yogurt, smoothies, mashed potatoes, soups, and soft fruits are gentle on braces and aligners. Expand (Go Soft: opt for softer foods right after tightening your braces. Think mashed potatoes, yogurts, and smoothies. aaoinfo.org) Soft cooked vegetables. Expand (Soft food such as pasta, rice, soft bread, mashed potatoes, apple sauce, bananas, soft cooked chicken and seafood, meatballs, lunch meat, cooked vegetables. www.marksortho.com ) Nutrient-Rich Foods; Dairy products (milk, cheese) provide calcium and vitamin D, essential for strong teeth and bones. Expand (Sources of calcium include: milk, cheese and other dairy foods www.nhs.uk ) Lean proteins (fish, chicken) support overall health. Expand (Lean animal proteins include white-fleshed fish, skinless poultry, and cuts of red meat such as loin and round. www.healthline.com ) Fruits, cut into small portions. Foods to Avoid Hard Foods : Nuts, hard candies, and crunchy vegetables can damage brackets and wires. Expand (Although nuts are a good option for your health, they are a terrible option for your braces. www.lakeminnetonkaortho.com ) Sticky and Chewy Foods : Caramel, chewing gum, and taffy can get lodged in braces and are difficult to remove. Expand (Avoid chewy candies, gum, caramel, and taffy. These foods can pull on wires and brackets, weakening the braces’ structure. epicortho.com) Sugary and Acidic Foods : Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices. Expand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. www.colgate.com ) Tough Meats : Tough or stringy meats can also cause damage to orthodontic appliances. Expand (Your braces are precise appliances that can be damaged by eating hard foods. Some soft and sticky foods can cause tooth decay and loosen… www.burlingamesmilestudio.com ) Hydration : Water : Drinking plenty of water is essential for maintaining oral hygiene and preventing dry mouth. Expand (By drinking enough water, you help prevent dry mouth and ensure that your saliva is produced at an optimal rate. dentistry.uic.edu) Water also helps to rinse away food particles. Expand (Water, however, cleans your mouth with every sip. It washes away leftover food and residue that cavity-causing bacteria are looking for. www.mouthhealthy.org ) Sugary and Acidic Foods : Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices. Expand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. www.colgate.com ) Oral Hygiene; Maintaining excellent oral hygiene is paramount. This includes thorough brushing and flossing to remove food particles that can become trapped in orthodontic appliances. Expand (Proper oral hygiene is essential to minimize complications related to orthodontic miniscrews. pmc.ncbi.nlm.nih.gov) (Careful brushing and flossing regularly can help remove any food particles or bacteria trapped in the orthodontic appliance or between the teeth. theorthodonticplace.com.au) Regular dental check-ups are also essential. Expand (These check-ups aren’t just routine. They’re an important part of making sure your orthodontic treatment is effective and your oral health is in good shape. southmiamiorthodontics.com) Key considerations; Orthodontic appliances create more places for food to become trapped, so diligent oral hygiene is essential. Expand (Maintaining a diligent daily oral hygiene routine is critical when you have braces or aligners. www.smilesaversortho.com ) Key considerations; It is very important to consult with your orthodontist, they can provide personalized dietary advice based on your specific treatment plan. Expand (Patients receiving orthodontic treatment must have open lines of communication with their dentist and orthodontist on food preferences and any oral health issues. www.batemanorthodontics.com ) Gemini credibility Compatible but insufficient : did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment Compatible Compatible Compatible Compatible Credibility of information resources Information credibility about tooth decay AAO web-source mentioned the need for swishing and rinsing with water, drinking water to prevent dental plaque to rinse away food debris, and acidic residues. The BOS leaflet contained detailed information about the effect of retained food, plaque, and sugary items on creating acidic environment which further causes tooth decays. ChatGPT 4.0 contained recommendations regarding sticky and sugary foods mentioning risks of decalcification around the brackets, and cleaning difficulties. Oral care tips included minimizing snacks to neutralize acids, and to drink water frequently. Copilot emphasized the importance of drinking water emphasizing its benefits in increasing saliva flow and reducing acids that can cause tooth decay. Tooth decay was referred through avoiding sugary and acidic foods in Gemini, compatible with the guidelines. The responses of LLMs were compatible but insufficient, since neither of them gave a detailed explanation as in BOS, regarding sugary items effects on the bacterial plaque which further produce an acidic environment leading to tooth decay. Information credibility about food Both AAO and BOS gave detailed information on the types of foods to be preferred and avoided, especially for the initial days after the braces are inserted. Details included hard/crunchy food types, and reasons on why to pay attention to the requested foods. Both guidelines suggested consumption of soft foods requiring little or no chewing on first days. Only AAO web-source attracted attention on extreme temperatures to avoid tooth sensitivity, and provided essential tips during eating. AAO web-source also suggested sugar-free gums, to overcome soreness. Consistent information in ChatGPT-4.0 was evident for avoiding hard, sticky, sugary foods, excessive chewing, consuming calcium-rich foods, vitamins, minerals and healthy fats including information for foods that patients can eat and prepare more comfortably because of the braces. The response was compatible. More details were apparent regarding the need for soft foods especially for few days, essential nutrients referring to special fruits, vegetables, vitamins, proteins, whole grains, the need for avoiding hard, sticky foods in Copilot, which all provided a compatible response. In general, similar information was given in Gemini, declaring the need for a balanced diet rich in vitamins and minerals, fruits, vegetables, lean proteins and dairy products, as well as recommendations for consuming soft foods and soft cooked vegetables, and eating with small portions. The information for foods to avoid was detailed. Overall, the response was compatible. Information credibility about beverages All resources emphasized the need to drink plenty of water. AAO web-source recommended avoiding drinks with extreme temperatures to overcome tooth sensitivity especially during the initial stages of treatment. The BOS leaflet recommended consuming milk and plain water, avoiding acidic drinks, limiting fruit juices only at mealtimes, and using a straw while consuming such beverages. ChatGPT 4.0 included examples of sugary drinks, like soda and juices, as drinks that should not be preferred. The effect of sugary drinks on the risk of cavities and decalcification were mentioned, and drinking water frequently was advised mentioning its’ benefits on cleaning food debris and neutralize acids around the braces. The response was compatible. Copilot provided an explanatory response due to challenges of sugary and acidic beverages pointing out the risk of dental cavities, declaring a compatible response. Given as a separate heading, Gemini recommended to drink water to prevent dry mouth and contribute to oral hygiene, together with the warning for sugary drinks and acidic foods with a short explanation of tooth decay and enamel erosion risks. The response was compatible. Information credibility about oral care All resources mentioned the need for good oral hygiene. AAO web-source advised swishing water for rinsing away food debris and acidic residues after eating. Oral hygiene tools were given in detail, carrying travel toothbrush, or water flosser were mentioned. BOS leaflet suggested brushing for 2 minutes at least twice a day using a fluoride toothpaste. Oral care tips are explained through recommendations as brushing and flossing after every meal, using interdental brushes and water flossers, drinking water frequently and minimizing snacking with ChatGPT 4.0. The responses were compatible. Rinsing away small food particles around the braces, practice of dental hygiene after consuming sugary and acidic foods or beverages were advised in Copilot. The response was in line, but the oral hygiene tools were not explained in detail. Gemini mentioned that orthodontic appliances create more places for food to become trapped, so diligent oral hygiene is essential, revealing the needs for brushing and flossing to remove food particles, and provided links for detailed information. So, the response was compatible. Information credibility about further assistance AAO web-source emphasized that specific guidelines could be best provided by orthodontists, as their advices relied on years of experience and knowledge, further provided a referral opportunity to find the nearest orthodontist. BOS leaflet asked patients to refer to either an orthodontist, dentist or hygienist, in case of information needs. Besides, free links for patients to access further information were included. ChatGPT 4.0 did not contain any information for the need of consultation from orthodontists, which was incompatible with the guidelines. Copilot suggested patients to share their concerns with their orthodontists and nutritionists, and noted that they could receive personalized advice from them. The response was compatible with the guidelines. Gemini emphasized the importance of seeking personalized advice from orthodontists, which was found to be consistent with the guidelines. DISCERN scores The DISCERN scores were calculated as mean, standard deviation, median, minimum and maximum, and presented in Table 3 . BOS leaflet had the highest mean DISCERN score (4.70 ± 0.27), followed by Gemini (4.54 ± 1.03), AAO web-source (4.45 ± 0.75), Copilot (3.87 ± 1.64) and ChatGPT 4.0 (3.08 ± 1.65). The results of the Friedman test indicated a statistically significant difference among the five groups (χ² (4) = 10.387, p = .034). Following pairwise comparisons using the Wilcoxon signed-rank tests with Bonferroni correction indicating that none of the pairwise comparisons remained statistically significant after adjustment (p > 0.05). The closest comparisons to significance were between ChatGPT 4.0 and BOS (adjusted p = 0.398) and between ChatGPT 4.0 and Gemini (adjusted p = 0.091). mean scores by resource by examiners are given in Fig. 1 . Table 3 Descriptive statistics for DISCERN scores, and pairwise comparison among resources. Resources DISCERN Scores Mean ± SD Median (Min-Max) P value AAO 4.45 ± 0.75 4.83 (3.00–5.00) 0.034* BOS 4.70 ± 0.28 4.67 (4.33-5.00) ChatGPT 4.0 3.08 ± 1.65 3.50 (1.00–5.00) Copilot 3.87±1.64 4.83 (1.00–5.00) Gemini 4.54±1.03 5.00 (2.00–5.00) SD, standard deviation, Min, minimum, Max, maximum. * Friedman test Readability scores : Table 4 demonstrated the results for FRES and FKGL levels for each source. AAO web-source readability was standard and/or plain with FRES score of 61.4, and the BOS leaflet was fairly easy to read with a 74.4 FRES score. The FRES levels of both guidelines revealed higher scores than the LLMs, showing that the responses were easier to read. The grades of FKGL revealed higher scores for Copilot and Gemini (Table 4 ). Table 4 Flesch Reading Ease Scores and Flesch-Kincaid Grade Levels of each resource. Flesch Reading Ease Flesch-Kincaid Grade Level BOS 74.4 6 AAO 61.4 7.6 ChatGPT 4.0 55.5 7.8 Copilot 47.1 11.9 Gemini 42.6 9.8 Discussion AI chatbots connect with users through natural language processing and machine learning methodologies, aiming to improve access to healthcare services, deliver fast medical information, and reduce the burden on medical systems [ 16 ]. In this regard, it’s crucial that clinicians examine criteria such as precision, reliability, quality, and readability of chatbot responses in the medical area [ 14 ]. In this study, responses of different LLMs in terms of nutritional guidelines of orthodontic patients were compared with the AAO web-source and BOS leaflet. Present results showed that the responses of AI-based LLMs were understandable and compatible with the guidelines. In general, the null hypothesis was accepted, but several details especially regarding information for tooth decay were lacking in LLMs. The most detailed information about tooth decay was included in the BOS leaflet with a brief explanation of what tooth decay means. In other sources, there was not enough explanation about the mechanism of tooth decay risk caused by inadequate oral care due to orthodontic braces and the bacterial plaque. Eventually, the increased acidic environment and the effects of nutrition on these mechanisms were not fully mentioned. Avoiding hard, sticky, chewy and sugary foods were specified in all LLMs similar to the guidelines. Essential tips were provided in AAO web-source to avoid extreme temperatures to prevent tooth sensitivity especially at early days of braces, different than the other resources. Information regarding foods/beverages to avoid and prefer, and the balanced diet contents were compatible with the guidelines. Only the Copilot's response lacked information regarding oral hygiene tools. Concerning the necessity of referral to orthodontists, apart from ChatGPT 4.0, responses were sufficient in all resources. Orthodontists' clinical experience, up-to-date knowledge and relationship with the patients enable them to convey more detailed and individualized information. Healthcare professionals are the most reliable source of information, but nowadays it is a fact that patients use the internet as a source of information [ 25 ]. Patients seeking orthodontic treatment try to collect as much information as possible to facilitate their decision-making process [ 26 ]. Therefore, the use of internet as an information source is becoming increasingly widespread. It has even become the first source of information for some patients before consulting any specialist. A previous study reported that more than half of the individuals used social media and websites as a source of information for orthodontic treatment [ 27 ]. Information-seeking through the internet depends also on patients' expectations from orthodontic treatment. Previously, differences were found between the expectations of patients and their parents in terms of expectations for treatment duration, demand for braces fitted at the initial visit, and dietary restrictions due to orthodontic treatment. The authors also reported that different ethnic groups exhibit different expectations regarding the initial orthodontic visit, and the impact of orthodontic treatment on nutrition [ 28 ]. According to Firestone et al. [ 29 ] patients underestimated the required dietary changes as a result of pain associated with orthodontic treatment. Children expected significantly greater restrictions for food types that they could consume, than their parents, since they have learned the necessity of low-sugar and low-acid diet by consulting their friends who had previously received orthodontic treatment [ 28 ]. This shows that patients seek various information before receiving treatment. AAO web-source provided information for the variety of healthy foods, how to consume them at the initial stages of treatment, with a detailed, empathetic and a conversational language. The importance of temperature of foods/drinks, possible relief with ice cream/sugar-free gums, and detailed useful tips for patients were also highlighted. The BOS leaflet put emphasis on avoiding sugary/hard foods and advised selecting healthy foods and beverages (advised using straw with fruit juices). All LLMs suggested consumption of calcium-rich foods for orthodontic patients. Similarly, Al Jawad et al. [ 7 ] recommended dairy products (milk, cheese, ice-cream), as they are soft and supportive in bone remodeling during tooth movement. Ozdemir et al. [ 30 ] noted a non-significant decrease in calcium and vitamin A intake in the third month of orthodontic treatment, compared to pretreatment values in adolescents. Additionally, the study found significantly decreased levels of vitamin C, vitamin E, and fiber intake, especially in the first weeks of orthodontic treatment, and declared that intake of these nutrients did not return to their initial levels even in the 12th week of orthodontic treatment. Only AAO web-source suggested sugar-free gums for reducing pain during fixed orthodontic treatment. A recent systematic review approved chewing gums to be effective for reducing orthodontic pain after 24 hours of initial wire placement in orthodontic patients [ 31 ]. The need for good oral care was mentioned in all resources, with more details in the AAO web-source, such as the need for carrying travel toothbrush, or water flosser, and brushing and flossing especially after meals. Yeo et al. [ 32 ] noted that, unless explicitly stated, ChatGPT cannot distinguish specific guidelines relevant to a particular country. It can therefore be confusing if patients do not receive advice specific to their country, age, or existing problem. To avoid this, it is necessary to evaluate AI models according to limits and guidelines established by regulatory authorities or professional bodies [ 19 ]. The need to consult an orthodontist to receive personalized advice and be able to ask further questions was suggested by all resources except for ChatGPT 4.0. In order to increase the satisfaction levels of patients, and meet their expectations, orthodontists and patients need to establish good communication [ 33 ]. This relationship starts with sharing information in initial orthodontic consultation. A study declared that good communication has significant effects on the success of orthodontic treatment [ 34 ]. As mentioned in a previous report, ChatGPT cannot ask questions to clarify the questions being asked [ 35 ]. Thus, the doctor-patient relationship remains the most important factor contributing to patient satisfaction [ 34 ]. The Copilot provided examples and answers to questions that patients may have about nutrition, such as “What happens if I don’t follow these guidelines?”, “What are the signs of potential damage to my braces?” and “What specific foods should I include in my diet?” The Gemini provided an expand option for the information given, by showing the link or the literature from which the information was obtained. Although these features provide superiority, they cannot replace the individual recommendations that can be given by orthodontists. Despite the fast and easily achievable response of LLMs, it would be useful to evaluate the consistency of their responses to different clinical scenario questions in further studies. Previously, most of the ChatGPT 4.0 responses about eye health were found to be appropriate, but it was emphasized that these platforms are not a source of factual information in their current form. Finally, it was stated that patients should limit their search for health-related information online [ 36 ]. In Stevens et al.'s [ 37 ] study, 8% of the patients preferred to obtain online information, which was attributed to their concerns about the reliability of the information. Recommendations of the guidelines of various professional societies may vary. Additionally, the fact that information seeking behaviors may differ depending on different social and cultural expectations, reveals the importance of receiving face-to-face information, especially on medical issues. Sources of information, citations, and references are crucial in establishing the reliability of health information, and these factors were assessed in our study using the modified DISCERN tool. With respect to DISCERN scores, higher score was achieved in BOS leaflet (4.70 ± 0.27) followed by Gemini (4.54 ± 1.03) and AAO web-source (4.45 ± 0.75). Copilot (3.87 ± 1.64) and ChatGPT 4.0 (3.08 ± 1.65) had lower scores. However, pairwise comparisons did not yield statistically significant results. This may be attributed to the small sample size, which limits statistical power, and the conservative nature of the Bonferroni correction. Moreover, the limited score variability and possible similarity across systems may have reduced the ability to detect meaningful differences. A study compared four LLMs about clear aligners by DISCERN tool and Copilot was found to be the most reliable, followed by Gemini, ChatGPT 4.0 and ChatGPT 3.5. Researchers reported that Copilot utilized 87 references in all inquiries. Gemini used only four references in four questions. The research also indicated that the majority of Copilot's citations and all of Gemini's citations were from websites [ 14 ]. In contrast, in our study only Gemini utilized references in all presented information. However, the references were mostly from dental clinic websites and firms, instead of scientific sources/articles in this study. A significant limitation for chatbots such as ChatGPT 3.5 and ChatGPT 4.0 has been noted as their inability to provide references or citations for the information they produce [ 38 ]. This is evident from ChatGPT 4.0’s low scores and Gemini’s high scores on the DISCERN question ‘specifying the information sources used in the creation of the content’. Another study assessing responses to clinical orthodontics inquiries indicated that, Copilot's answers achieved the greatest ratings, followed by ChatGPT 4.0, Gemini, and lastly ChatGPT 3.5. Nevertheless, according to researchers, all models occasionally generated responses that lacked comprehensiveness, scientific accuracy, clarity, and relevance [ 12 ]. Conversely Daraqel et al. [ 39 ] stated that, ChatGPT 3.5 and Google Bard models have generally produced reasonably accurate and complete responses to posed general orthodontic inquiries asked by patients. The observed difference is believed to be associated with the content of the questions. Makrygiannakis et al. [ 12 ] posed detailed and technical inquiries regarding clinical orthodontics, whereas Daraqel et al. [ 39 ] included questions commonly raised by patients concerning general orthodontics. Health literacy is the skill of being able to read and understand knowledge about health, which helps individuals make decisions and control their treatment processes [ 17 ]. Readability has been emphasized in the literature as a critical component of health literacy, facilitating comprehension of documents. Young people and adolescents who require information at their level, both in-person and online, represent the majority of orthodontic populations worldwide [ 40 ]. To improve health literacy, the US Department of Health and Human Services has advised that patient-targeted materials should not be more complex than a sixth-grade reader [ 41 ]. The FRES, introduced in 1948, is used to assess text readability, which indicates the estimated educational level required for an individual to understand a specific text with ease. The clarity of a document is represented by a numerical value ranging from 0 to 100, and scores near “100” show that the material is highly simple to read, while scores near “0” indicate the complexity to comprehend [ 23 ]. The FKGL is a metric based on average sentence length and word complexity revealing the educational level required for an individual to understand a specific text [ 24 ]. According to our findings, AAO web-source and BOS leaflet were simpler to read than the LLMs. Similarly, a research evaluating the quality of leaflets from two international orthodontic societies for orthodontic patients reported a median FRES of 70 [ 42 ]. Although ChatGPT 4.0 had the highest FRES score among the LLMs in this study, it was found that all currently assessed AI chatbots were below 60 points declaring that they are difficult to read. A study evaluating the readability of information by different LLMs utilizing FRES, reported similar findings [ 14 ]. In terms of orthodontics, another study revealed that general responses by ChatGPT were classified as very difficult to read, and treatment-related questions were classified as difficult to read [ 17 ]. With respect to the written content's readability, it was suggested that the United States and the United Kingdom should ideally aim at eighth graders and fifth to sixth graders, respectively [ 43 , 44 ]. In accordance, the mean FKGL for AAO web-source was 7.6 and 6 for BOS leaflet. Leaflets from two international orthodontic societies found similar FKGL scores with 7.2 [ 42 ]. Copilot and Gemini demonstrated the highest FKGL scores. It implies that the responses of the Copilot and Gemini are less comprehensible than those of the other sources. Kılınç et al. [ 17 ] demonstrated that the mean FKGL for general orthodontic inquiries received from ChatGPT was 14.52 ± 1.48, whereas the score for treatment-related questions was 11.90 ± 2.08. It is believed that the general public's capacity to use chatbots will be limited by their readability issues. Future studies may aim to assess the correlation between complexity and readability of written texts. Websites created by experts may improve the performance of language models trained on diverse internet data by providing high-quality, clear, and comprehensible health information [ 14 ]. The advantage of this study can be considered as examining the same topic at different times and with different AI-based language models. One of the most important shortcomings was that the information sources in ChatGPT and Copilot were unclear. It would be advisable to use more objective categories to evaluate the LLMs' responses to ensure consistency. Additionally, this study was conducted only in English, where other languages and the intelligibility of these models by individuals with different educational levels could be the subject of future studies. Overall, further studies are still needed to determine which topics patients seek information about most regarding orthodontic treatment, and to make comprehensive evaluations by including newly developed models. Conclusion The credibility of LLMs was found to be adequate in terms of basic information on dietary recommendations, but details on the relationship between nutrition and tooth decay mechanisms were lacking. BOS leaflet had the highest mean DISCERN score followed by Gemini and AAO web-source. Average reliability was found for Copilot and ChatGPT 4.0 responses when compared to the guidelines according to the DISCERN tool. The responses of LLMs had a difficult reading level according to the FRES and FKGL. Patients' dietary habits and treatment requirements vary greatly, so despite the general and understandable responses of LLMs, it is ideal for orthodontists to provide detailed information about treatment content, dietary changes or oral care recommendations in a patient and treatment-specific manner. Abbreviations BOS The British Orthodontic Society AAO Association of Orthodontists AI artificial intelligence ChatGPT Chat Generative Pre-Trained Transformer LLMs Large Language Models LaMDA Language Model for Dialogue Applications PaLM 2 LLM Pathways Language Model 2 Declarations Ethics approval and consent to participate Ethical approval was not relevant, as this study required no human/animal materials. Consent for publication Not applicable. Availability of data and materials The data used and/or analyzed during the current study are available from the corresponding author on reasonable request ( [email protected] ). Competing interests The authors declare that they have no competing interests. Funding Not applicable. Author contributions Conceptualization and Investigation, BBT, TS; Supervision, BBT, CT; Original draft, BBT, TS; Review and editing, BBT; Analyzing the data, BBT, CT, TS; Statistical Analysis, İK. Acknowledgements Not applicable. References Tervonen MM, Pirttiniemi P, Lahti S. Development of a measure for orthodontists to evaluate patient compliance. Am J Orthod Dentofac Orthop. 2011;139(6):791–6. Carter LA, Geldenhuys M, Moynihan PJ, Slater DR, Exley CE, Rolland SL. The impact of orthodontic appliances on eating young people’s views and experiences. J Orthodont. 2015;42(2):114–22. Choi SH, Cha JY, Lee KJ, Yu HS, Hwang CJ. Changes in psychological health, subjective food intake ability and oral health-related quality of life during orthodontic treatment. J Rehabil. 2017;44(11):860–9. Marshall TA. Chairside diet assessment of caries risk. J Am Dent Association. 2009;140(6):670–4. Sudjalim T, Woods M, Manton D. Prevention of white spot lesions in orthodontic practice: a contemporary review. Aust Dent J. 2006;51(4):284–9. Aljohani SR, Alsaggaf DH. Adherence to dietary advice and oral hygiene practices among orthodontic patients. Patient Prefer Adherence. 2020;1991–2000. Abed Al Jawad F, Cunningham SJ, Croft N, Johal A. A qualitative study of the early effects of fixed orthodontic treatment on dietary intake and behaviour in adolescent patients. Eur J Orthod. 2012;34(4):432–6. British Orthodontic Society. Patient information leaflet: teeth and brace-friendly food and drink. https://www.bos.org.uk/wp-content/uploads/2022/03/British-Orthodontic-Society-Food-and-DrinkMarch2019.pdf (2019) Accessed 19 Feb 2024. American Association of Orthodontists. What Can I Eat with Braces? A Comprehensive Guide. https://aaoinfo.org/blog/what-can-i-eat-with-braces/ (2023) Accessed 19 Feb 2024. Crispino R, Mannocci A, Dilena IA, Sides J, Forchini F, Asif Alherawi WM, Frazier-Bowers SA, Grippaudo C. Orthodontic patients and the information found on the web: a cross-sectional study. BMC Oral Health. 2023;23(1):860. Eggmann F, Blatz MB. ChatGPT: Chances and Challenges for Dentistry. Compendium Continuing Educ Dentistry (15488578). 2023;44(4). Makrygiannakis MA, Giannakopoulos K, Kaklamanos EG. Evidence-based potential of generative artificial intelligence large language models in orthodontics: A comparative study of ChatGPT, Google Bard, and Microsoft Bing. Eur J Orthod. 2024;cjae017. Rahaman MS, Ahsan M, Anjum N, Rahman MM, Rahman MN. The AI race is on! Google's Bard and OpenAI's ChatGPT head to head: an opinion article. Mizanur and Rahman, Md Nafizur,. ; 2023. The AI Race is on. Dursun D, Bilici Geçer R. Can artificial intelligence models serve as patient information consultants in orthodontics? BMC Med Inf Decis Mak. 2024;24(1):211. Xue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Translational Med. 2023;13(3):e1216. Acar AH. Can natural language processing serve as a consultant in oral surgery? J Stomatology Oral Maxillofacial Surg. 2024;125(3):101724. Kılınç DD, Mansız D. Examination of the reliability and readability of Chatbot Generative Pretrained Transformer’s (ChatGPT) responses to questions about orthodontics and the evolution of these responses in an updated version. Am J Orthod Dentofac Orthop. 2024;165(5):546–55. Goodman RS, Patrinely JR, Stone CA, Zimmerman E, Donald RR, Chang SS, Berkowitz ST, Finn AP, Jahangir E, Scoville EA. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open. 2023;6(10):e2336483–2336483. Barlas T, Altinova AE, Akturk M, Toruner FB. Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines. Int J Obes. 2024;48(2):271–5. Gupta S, Tarapore R, Haislup B, Fillar A. Microsoft copilot provides more accurate and reliable information about anterior cruciate ligament injury and repair than ChatGPT and Google Gemini; however, no resource was overall the best. Arthrosc Sports Med Rehabilitation. 2025;7(2):101043. Abu Arqub S, Al-Moghrabi D, Allareddy V, Upadhyay M, Vaid N, Yadav S. Content analysis of AI-generated (ChatGPT) responses concerning orthodontic clear aligners. Angle Orthod. 2024;94(3):263–72. Charnock D, Shepperd S. Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res. 2004;19(4):440–6. Flesch R. A new readability yardstick. J Appl Psychol. 1948;32(3):221. Kincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975. Hu J, Luo E, Song E, Xu X, Tan H, Zhao Y, Wang Y, Li Z. Patients’ attitudes towards online dental information and a web-based virtual reality program for clinical dentistry: A pilot investigation in China. Int J Med Informatics. 2009;78(3):208–15. Wahab RMA, Hasan SK, Yamin NEM, Ibrahim Z. Awareness of fake braces usage among Y-generations. J Int Dent Med Res. 2019;12(2):663–6. Kamarozaman DMS, Kamarudin Y, Hamzah TNNT, Nor NAM. Self-perceived Orthodontic Need, Information Seeking Behaviour and Knowledge on Orthodontic Treatment Among Young Malaysian Adults. J Health Translational Med (JUMMEC). 2020;23(2):8–15. Sayers M, Newton J. Patients’ expectations of orthodontic treatment: Part 2—findings from a questionnaire survey. J Orthodont. 2007;34(1):25–35. Firestone AR, Scheurer PA, Bürgin WB. Patients' anticipation of pain and pain-related side effects, and their perception of pain as a result of orthodontic treatment with fixed appliances. Eur J Orthod. 1999;21(4):387–96. Ozdemir M, Ilhan A, Gorucu-Coskuner H, Taner T, Bilgic P. Assessment of food consumption changes in adolescents during orthodontic treatment. Am J Orthod Dentofac Orthop. 2021;159(5):604–12. Mando M, Talaat S, Bourauel C. The efficacy of chewing gum in the reduction of orthodontic pain at its peak intensity: a systematic review and meta-analysis. Angle Orthod. 2023;93(5):580–90. Yeo YH, Samaan JS, Ng WH, Ting P-S, Trivedi H, Vipani A, Ayoub W, Yang JD, Liran O, Spiegel B. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol. 2023;29(3):721. Chiang YC, Wu F, Ko S-H. Effective patient–dentist communication with a Simulation System for Orthodontics. Healthc MDPI. 2023;11(10):1433. Keles F, Bos A. Satisfaction with orthodontic treatment. Angle Orthod. 2013;83(3):507–11. Howard A, Hope W, Gerada A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect Dis. 2023;23(4):405–6. Momenaei B, Wakabayashi T, Shahlaee A, Durrani AF, Pandit SA, Wang K, Mansour HA, Abishek RM, Xu D, Sridhar J. Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases. Ophthalmol Retina. 2023;7(10):862–8. Stephens R, Ryan FS, Cunningham SJ. Information-seeking behavior of adolescent orthodontic patients. Am J Orthod Dentofac Orthop. 2013;143(3):303–9. Mohammad-Rahimi H, Ourang SA, Pourhoseingholi MA, Dianat O, Dummer PMH, Nosrat A. Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. Int Endod J. 2024;57(3):305–14. Daraqel B, Wafaie K, Mohammed H, Cao L, Mheissen S, Liu Y, Zheng L. The performance of artificial intelligence models in generating responses to general orthodontic questions: ChatGPT vs Google Bard. Am J Orthod Dentofac Orthop. 2024;165(6):652–62. Oey CG, Livas C. The informative value and design of orthodontic practice websites in The Netherlands. Prog Orthodont. 2020;21:1–7. Meade MJ, Dreyer CW. Orthodontic treatment consent forms: A readability analysis. J Orthodont. 2022;49(1):32–8. Seehra J, Cockerham L, Pandis N. A quality assessment of orthodontic patient information leaflets. Prog Orthodont. 2016;17:1–6. Williamson JML, Martin AG. Analysis of patient information leaflets provided by a district general hospital by the Flesch and Flesch–Kincaid method. Int J Clin Pract. 2010;64(13):1824–31. Kenny T, Wilson R, Purves I, Clark Sr J, Newton L, Newton D, Moseley D. A PIL for every ill? Patient information leaflets (PILs): a review of past, present and future use. Fam Pract. 1998;15(5):471–9. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 26 Nov, 2025 Read the published version in BMC Oral Health → Version 1 posted Editorial decision: Revision requested 26 Aug, 2025 Reviewers agreed at journal 25 Aug, 2025 Reviews received at journal 24 Aug, 2025 Reviewers agreed at journal 23 Aug, 2025 Reviewers agreed at journal 23 Aug, 2025 Reviews received at journal 06 Aug, 2025 Reviewers agreed at journal 29 Jul, 2025 Reviewers invited by journal 23 Jun, 2025 Editor invited by journal 04 Jun, 2025 Editor assigned by journal 02 Jun, 2025 Submission checks completed at journal 02 Jun, 2025 First submitted to journal 31 May, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6792273","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":475532443,"identity":"6a49c933-a5af-4ace-9b32-ec3bdf221ee2","order_by":0,"name":"Tevhide Sokmen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIiWNgGAWjYHACNoYENiDFA2JXADEzjIMD8KBqOYPQIoFXCwNMC2MbQhinFnv2HrMHD8ps7Pl5Tic+LpxXKy/fzsD44G0bQ515Aw5beM6YGyScS0uc2du72XjmtuOGjc0MzIZz2xgkZA7g0CKRliaR2HY4weA87zZp3m3HGJuZGdikeYFacLmMR/4ZSMt/e6CW7b955xyzb2NmYP+NV4sE8zGglgOMG872bmPmbahJ7AHawoxXy5nkYxIJ55ITZ/ac3SzNc+xA8gxmxmbJOeckJGfg0MLefrBN8keZHTDEcjd+5qmps53ff/jghzdlNvw4QxkNHAZixgYGPNGCAeqIVjkKRsEoGAUjBwAASgtPrFhWE9UAAAAASUVORK5CYII=","orcid":"","institution":"University of Gazi","correspondingAuthor":true,"prefix":"","firstName":"Tevhide","middleName":"","lastName":"Sokmen","suffix":""},{"id":475532444,"identity":"63c75452-e903-4239-8614-93124741045a","order_by":1,"name":"İrem Kar","email":"","orcid":"","institution":"University of Ankara","correspondingAuthor":false,"prefix":"","firstName":"İrem","middleName":"","lastName":"Kar","suffix":""},{"id":475532445,"identity":"9ac8ea60-a75d-48c6-b3c4-0277a576dfca","order_by":2,"name":"Cumhur Tuncer","email":"","orcid":"","institution":"University of Gazi","correspondingAuthor":false,"prefix":"","firstName":"Cumhur","middleName":"","lastName":"Tuncer","suffix":""},{"id":475532446,"identity":"685c05f1-27ca-4a3d-8c7f-70722c630656","order_by":3,"name":"Burcu Balos Tuncer","email":"","orcid":"","institution":"University of Gazi","correspondingAuthor":false,"prefix":"","firstName":"Burcu","middleName":"Balos","lastName":"Tuncer","suffix":""}],"badges":[],"createdAt":"2025-05-31 17:23:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6792273/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6792273/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12903-025-07153-1","type":"published","date":"2025-11-26T15:57:52+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":85643088,"identity":"a86fe62b-dcd8-43ff-a340-78471994ee47","added_by":"auto","created_at":"2025-06-30 08:06:21","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":642046,"visible":true,"origin":"","legend":"\u003cp\u003eLegend not included with this version\u003c/p\u003e","description":"","filename":"Figure110.png","url":"https://assets-eu.researchsquare.com/files/rs-6792273/v1/b52167391ffe6719db79998b.png"},{"id":97178385,"identity":"10eac881-7642-46aa-9e65-88d2d87d39ef","added_by":"auto","created_at":"2025-12-01 16:09:28","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2444856,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6792273/v1/2af49bc0-9295-4fa1-87b0-859ae0add732.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Assessment of the credibility of generative artificial intelligence large language models on nutritional guidelines for orthodontic patients","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe cooperation of patients is important during fixed orthodontic treatment in terms of maintaining oral hygiene, achieving treatment targets, and adhering to the recommended nutritional changes [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Orthodontists advise certain dietary restrictions/modifications at the beginning of treatment. According to Carter et al. [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] patients reported limited food choice due to dietary advice given by their orthodontist, and fear of breakage. Choi et al. [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] revealed that orthodontic treatment changed patients eating behaviors, and may cause eating problems. The patients are advised to minimize consumption of sugary and sticky food to lower the risk of caries and white spot lesions [\u003cspan additionalcitationids=\"CR5\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Again, the patients are instructed to prefer softer food in order not to harm the appliances [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. The British Orthodontic Society (BOS) [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], and American Association of Orthodontists (AAO) [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] have prepared guidelines to assist orthodontic patients with their food choices.\u003c/p\u003e \u003cp\u003eTechnological advancements changed lifestyles, demands, and habits of information seeking, mainly among younger generations. This has led to the search for information turning primarily to internet browsing for almost every information. Crispino et al. [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] investigated the relevance of web-based information for orthodontic patients, and reported that, most use Internet as a source for orthodontic information. They also reported that patients who perceived web-sourced information as useful did not easily ignore this information, even if it was contrary to the orthodontist's opinions. By the end of 2022, a revolutionary advancement in artificial intelligence (AI), known as Chat Generative Pre-Trained Transformer (ChatGPT), was developed by OpenAI Inc. (San Francisco, CA, USA). Within the initial three months post-launch, it gained over 100\u0026nbsp;million new users [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. In February 2023, Microsoft Corporation (Redmond, WA, USA) released the Bing Chat AI chatbot utilizing the GPT-4 language model, which is now being renamed as Copilot, following the success of Chat GPT. In March 2023, Google (Google Ireland Limited, Dublin, Ireland) introduced the Google Bard language model, first utilizing LaMDA (Language Model for Dialogue Applications) and subsequently driven by the PaLM 2 LLM (Pathways Language Model 2), which is presently being reintroduced as Gemini [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. These models and their corresponding updates have various distinguishing characteristics. ChatGPT's responses derive from pre-existing training data, whereas Gemini utilizes real-time Internet connection to integrate current information in its response generation [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The daily use of AI chatbots is gaining popularity among healthcare professionals and patients as a useful tool for medical and dental information, due to advances in technology [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. AI chatbots provide patients with the ability to access information at any time, as well as rapid responses to questions and the reduction of the need for appointments [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. However, concerns about the credibility of the information of AI chatbots have been highlighted [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Recently, the need for further model development was emphasized to increase ChatGPT\u0026rsquo;s accuracy for medical queries [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. Similarly, ChatGPT answers were declared as comprehensive and understandable in assessment of obesity in type 2 diabetes, but the need of patient-centered approach verified by up-to-date and reliable sources was reported [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. A recent study in the medical field assessing the responses of ChatGPT-3.5, Gemini, and Copilot regarding a specific topic in orthopedic surgery found that the answers from the language models were highly complex, and no single resource emerged to be superior [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe evaluation of the reliability of ChatGPT responses related to orthodontics showed that the responses were not scientific due to the lack of specified peer-reviewed references [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Abu Arqub et al. [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] demonstrated that overall accuracy of ChatGPT-3.5 responses to inquiries about clear aligners is insufficient. In a study assessing the quality of information provided in four different LLMs about clear aligners declared that, more evidence-based information is needed and readability should be improved [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Moreover, a recent study in orthodontics concluded that current LLMs are not a consistently reliable source of evidence for either patients or clinicians seeking information online [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. With the emphasis on nutritional issues at the beginning of treatment, patients who seek orthodontic treatment can search for information on the internet, so it is important to assess the credibility of online information sources. To the best of our knowledge, there is no publication examining this topic in the literature. Therefore, this study intended to be the first to evaluate the credibility of the different LLMs compared to the current nutritional guidelines for orthodontic patients. The null hypothesis of the study was that there was no difference in the reliability and readability of information conveyed by three different LLM resources relative to AAO and BOS guides in terms of nutritional emphasis for orthodontic patients.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003eEthical approval was not relevant, as this study required no human/animal materials. The credibility of ChatGPT 4.0, Gemini and Copilot responses were assessed regarding AAO web-source [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] and the BOS leaflet [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], to provide an objective evaluation. ChatGPT 4.0 responses were achieved on December 23, 2024. Additionally, responses from Gemini and Copilot were obtained both on March 21, 2025. By using \"New Chat\" function, \u0026ldquo;W\u003cem\u003ehat are the nutritional guidelines for orthodontic patients?\u003c/em\u003e\u0026rdquo; was asked. Three orthodontists (TS, CT, BBT) evaluated information credibility about tooth decay, food, beverages, oral care, and further assistance through the following classification [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]:\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cp\u003e \u003cb\u003e1.\u003c/b\u003e Compatible with the guidelines: contains information from the guideline on the subject, with no missing or contradictory information.\u003c/p\u003e \u003cp\u003e \u003cb\u003e2.\u003c/b\u003e Compatible with the guidelines but insufficient: contains information from the guideline on the subject, with some missing information but no contradictory information.\u003c/p\u003e \u003cp\u003e \u003cb\u003e3.\u003c/b\u003e Partially incompatible with the guidelines: contains information from the guideline on the subject, with some contradictory information.\u003c/p\u003e \u003cp\u003e \u003cb\u003e4.\u003c/b\u003e Incompatible with the guidelines: contains information on the subject contradictory to the guidelines. The reasons for insufficient responses were noted.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eFurther, scores for reliability were assessed by the same researchers (TS, CT, BBT) using the modified DISCERN instrument, which is a valid tool for assessing the quality of health-related written or online-information (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.discern.org.uk\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.discern.org.uk\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e). The instrument composed of 3 parts and 16 questions using a 1\u0026ndash;5 rating scale, where 1 indicates total rejection and 5 indicates absolute agreement [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. In this study, the questions in the first part regarding reliability (questions 1\u0026ndash;8) were rated in accordance with the instructions of the DISCERN tool.\u003c/p\u003e \u003cp\u003eThe readability of each source was assessed using the Flesch Reading Ease instrument (FRES) [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], and Flesch-Kincaid Grade Level (FKGL) [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. The clarity of a document is represented by a numerical value ranging from 0 to 100 in the Flesch Reading Ease score. Scores near \u0026ldquo;100\u0026rdquo; show that the material is highly simple to read, while scores near \u0026ldquo;0\u0026rdquo; indicate it is quite complex and challenging to comprehend [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. The complete text from the AI chatbots and websites were copied into a Word (Microsoft Office, Version 16.78.0) document. The text was subsequently loaded into an online tool \u0026ldquo;\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://readable.com\u003c/span\u003e\u003cspan address=\"https://readable.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u0026rdquo; and tested for readability of sources. The software's scores were analyzed using the data previously supplied in research [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] and shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eThe evaluation of the reading levels according to Flesch Reading Ease Score.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFlesch Reading\u003c/p\u003e \u003cp\u003eEase Score\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eReading Level\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eEstimated reading grade level\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e0\u0026ndash;29\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVery difficult\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eGraduated college\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e30\u0026ndash;49\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDifficult\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAttended college\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e50\u0026ndash;59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFairly difficult\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eHigh school (10th to 12th)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e60\u0026ndash;69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStandard and/or plain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eEighth to ninth\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e70\u0026ndash;79\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFairly easy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSeventh grade\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e80\u0026ndash;89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEasy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFifth to sixth grade\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e90\u0026ndash;100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVery easy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFourth to fifth grade\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStatistical analysis\u003c/h2\u003e \u003cp\u003eFive resources as AAO web-source, BOS leaflet, ChatGPT-4.0, Copilot, and Gemini were evaluated based on their responses to eight structured items. Each response was rated on a 1 to 5 Likert scale by three independent raters. Given the ordinal nature of the data and the small sample size (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;8 per system), nonparametric statistical methods were employed.\u003c/p\u003e \u003cp\u003eInter-rater reliability was assessed for each system. High agreement was observed among raters for ChatGPT-4, Copilot, and Gemini. Moderate agreement was found for AAO, while inter-rater reliability could not be calculated for BOS due to the restricted range of scores (only ratings of 4 and 5 were assigned). Nonetheless, visual inspection suggested consistent scoring across raters for BOS.\u003c/p\u003e \u003cp\u003eFor each system, the mean score across the three raters was calculated. A Friedman test was then performed to assess whether there were statistically significant differences in mean ratings among the five systems. In the presence of overall significance, pairwise comparisons were conducted using Wilcoxon signed-rank tests with Bonferroni correction to control for the risk of type I error. All analyses were conducted using IBM SPSS Statistics version 30. Statistical significance was set at a threshold of p\u0026thinsp;\u0026lt;\u0026thinsp;0.05.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eResponses of all LLM resources were understandable, and compatible with no contradictory information. The credibility of information resources is shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e was placed at the end of the document text file). General recommendations on nutritional guidelines for orthodontic patients were mentioned, but some details were missing as follows:\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAssessment of the responses of the guidelines and language models on nutritional guidelines of orthodontic patients.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResources\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eInformation on tooth\u003c/p\u003e \u003cp\u003edecay\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eInformation on food\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eInformation on beverages\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eInformation on oral care\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eInformation on further assistance\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAAO\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eSwish and rinse\u003c/b\u003e; after eating swishing water around your mouth is beneficial to dislodge any food particles stuck in or around your braces. This prevents plaque build-up and staining.\u003c/p\u003e \u003cp\u003e\u003cb\u003eStay hydrated;\u003c/b\u003e Drinking water throughout the day keeps you hydrated and helps keep your mouth clean. It assists in rinsing away food debris and acidic residues.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eFood choices with braces\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eFirst few days with braces\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eEmbracing soft foods;\u003c/b\u003e consume soft, gentle foods for first days with braces: oatmeal, scrambled eggs, soups, pasta, seafood, soft vegetables, mashed potatoes, soft cheeses, bananas, yogurt, ice cream (cold comforts and relieves some pressure of the wires) \u003cb\u003eFoods to approach with caution initially;\u003c/b\u003e Be careful with chewy breads, rolls, meat \u003cb\u003eFoods to avoid\u003c/b\u003e; hard-sticky candies, biting whole apples, corn on the cob, popcorn, nuts, crunchy veggies-fruits, pizza crust, bagels or hard rolls, ice, snack chips, pretzels (may cause loosing braces, wires and retard treatment)\u003c/p\u003e \u003cp\u003e\u003cb\u003eEssential tips for eating with braces\u003c/b\u003e (cooking/steaming/slicing for softer foods, preferring small pieces, careful chewing, oral care tools, attention to temperature, sugar-free gum to handle soreness), avoid extreme temperature (to prevent tooth sensitivity especially at early days of braces)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eAvoid extreme temperatures;\u003c/b\u003e Be wary of foods or drinks that are too hot or cold (extreme temperatures can heighten tooth sensitivity, especially during the early days of braces)\u003c/p\u003e \u003cp\u003e\u003cb\u003eStay hydrated;\u003c/b\u003e Drinking water throughout the day keeps you hydrated and helps your mouth clean\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eSwish and rinse;\u003c/b\u003e\u003c/p\u003e \u003cp\u003eAfter eating, swishing water around mouth is beneficial to dislodge any food particles around braces, that prevents plaque build-up\u003c/p\u003e \u003cp\u003e\u003cb\u003eInvest in the right tools;\u003c/b\u003e Carry travel toothbrush, or water flosser. Brushing, flossing (especially after meals)\u003c/p\u003e \u003cp\u003e\u003cb\u003eStay hydrated;\u003c/b\u003e Helps keeping mouth clean and rinse away food debris and acidic residues\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eListen to your orthodontist;\u003c/b\u003e Your orthodontist will provide specific guidelines about avoiding foods and ways to take care of your braces, their advice stems from years of experience and knowledge, so it is essential to follow their recommendations\u003c/p\u003e \u003cp\u003e\u003cb\u003eTake the next step;\u003c/b\u003e find a dedicated AAO orthodontist near you and get the expert care you deserve, visit our orthodontist locator today\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eBOS\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eA brief definition is provided answering \u0026ldquo;What is tooth decay?\u0026rdquo;\u003c/b\u003e When you eat or drink particularly sugary items, the bacteria (plaque) in your mouth produce acid. This acid can make holes in your teeth (tooth decay). Your teeth will be under attack from this plaque acid for up to one hour after eating food or drinking sugary drinks. Retained food and bacteria (plaque) may lead to permanent and unsightly marks on teeth and tooth decay, especially sugary items produce acid for up to one hour after eating food or drinking sugary drinks, which can make holes in teeth (tooth decay). Fruit juices should be kept to mealtimes only.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eDiet and braces;\u003c/b\u003e eating wrong foods can break your braces/damage your teeth, treatment will be lengthened\u003c/p\u003e \u003cp\u003e\u003cb\u003eDo I need to change my eating habits;\u003c/b\u003e choose softer foods when braces are fitted/adjusted for few days: soup, pasta dishes, yoghurts, cheese; cut up rather than bite\u003c/p\u003e \u003cp\u003e\u003cb\u003eCan I eat snacks?\u003c/b\u003e Eat 3 meals as healthier\u003c/p\u003e \u003cp\u003e\u003cb\u003eWhat food and drinks can I eat between meals?\u003c/b\u003e Select healthy foods as whole meal bread, plain cheese, dry crackers, breadsticks, fruits, vegetables, plain milk, plain water\u003c/p\u003e \u003cp\u003e\u003cb\u003eFood and drinks to avoid;\u003c/b\u003e hard, sticky, crunchy foods (toffee, caramel, lollies; consume buy cutting apples, raw carrot, pizza crust, nuts, biscuits, celery, chicken wings, ribs; chewing them with the back teeth), fizzy, acidic/sugary foods and drinks\u003c/p\u003e \u003cp\u003eFruit juices only at mealtimes using a straw, minimum chocolates/ sweets (only at meal times)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eDrinks to avoid\u003c/b\u003e (fizzy, soft, acidic drinks can cause damage)\u003c/p\u003e \u003cp\u003e\u003cb\u003eSelect plain milk or plain water (not fizzy)\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eLimit fruit juice to mealtimes, drink with a straw\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eBrush your teeth; for\u003c/b\u003e 2 minutes at least twice a day with a fluoride toothpaste\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eIf you have any further questions;\u003c/b\u003e please ask your orthodontist, dentist or hygienist. \u003cb\u003eFree links\u003c/b\u003e: British Dental Health Foundation website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.dentalhealth.org.uk\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.dentalhealth.org.uk\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003eNHS Oral Health Specialist Library website \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.library.nhs.uk/oralhealth/\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.library.nhs.uk/oralhealth/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003eFor more information about Orthodontics, please visit: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.bos.org.uk\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.bos.org.uk\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eChatGPT 4.0\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eSugary foods and drinks increase the risk of cavities and decalcification around brackets\u003c/b\u003e (soda, candy, cookies, sugary juices)\u003c/p\u003e \u003cp\u003e\u003cb\u003eDrink water frequently\u003c/b\u003e, helps wash away food debris and neutralize acids.\u003c/p\u003e \u003cp\u003eMinimize snacking, reduces the number of times teeth are exposed to sugars and acids.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eFoods to avoid;\u003c/b\u003e hard foods (nuts, hard candies, popcorn, raw carrots, ice), sticky foods (chewing gum, caramel, taffy, gummy candies), sugary foods/drinks (soda, candy, cookies, sugary juices), chewy foods (bagels, tough bread, jerky), foods to bite into (cut into smaller pieces apples, corn on the cob, whole pears)\u003c/p\u003e \u003cp\u003e\u003cb\u003eFoods to include;\u003c/b\u003e soft foods (mashed potatoes, yogurt, pasta, scrambled eggs, soups), cut/cooked vegetables (steamed broccoli, roasted zucchini, thinly sliced cucumbers), dairy products (cheese, milk, yogurt), protein-rich foods (soft-cooked chicken, fish, tofu, eggs), fruits (bananas, berries, applesauce, melons), whole grains (oatmeal, soft bread, rice)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eAvoid sugary drinks;\u003c/b\u003e soda, sugary juices (risk of cavities, decalcification)\u003c/p\u003e \u003cp\u003e\u003cb\u003eDrink water frequently\u003c/b\u003e (helps wash away food debris and neutralize acids)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eOral care tips Brush and floss after every meal;\u003c/b\u003e food particles can easily get trapped in braces\u003c/p\u003e \u003cp\u003e\u003cb\u003eUse orthodontic tools;\u003c/b\u003e interdental brushes and water flossers help cleaning around braces\u003c/p\u003e \u003cp\u003e\u003cb\u003eDrink water frequently\u003c/b\u003e; helps wash away food debris and neutralize acids\u003c/p\u003e \u003cp\u003e\u003cb\u003eMinimize snacking;\u003c/b\u003e reduces the number of times teeth are exposed to sugars and acids\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eNot existed\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eChatGPT 4.0\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003ecredibility\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003eCompatible but insufficient\u003c/b\u003e: did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eIncompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCopilot\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eStay hydrated with water;\u003c/b\u003e Drinking plenty of water not only aids digestion but also maintain saliva flow, essential for neutralizing acids that can cause tooth decay.\u003c/p\u003e \u003cp\u003e\u003cb\u003eLimit sugary and acidic foods and beverages;\u003c/b\u003e Sugar and acid can increase the risk of cavities and since braces make cleaning teeth more challenging, minimizing these foods is crucial.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eEmphasize soft foods after adjustments;\u003c/b\u003e soring may occur for a few days, eat soft, easily chewable foods (mashed potatoes, soups, yogurts, smoothies, scrambled eggs)\u003c/p\u003e \u003cp\u003e\u003cb\u003ePrioritize a balanced diet rich in essential nutrients;\u003c/b\u003e calcium-vitamin D (milk, cheese, yogurt, leafy greens, fortified foods), lean proteins (tender chicken, tofu, fish, eggs), fruits-vegetables (bananas, melons, berries, steamed carrots, zucchini, broccoli), whole grains (oatmeal, soft pasta, rice)\u003c/p\u003e \u003cp\u003e\u003cb\u003eAvoid hard-sticky-chewy foods\u003c/b\u003e; nuts, hard candy, gum, caramel, raw crunchy vegetables/fruits (cut into smaller pieces, cook/chop)\u003c/p\u003e \u003cp\u003e\u003cb\u003eCut foods into bite-sized pieces\u003c/b\u003e reduces the strain on your braces and chewing muscles\u003c/p\u003e \u003cp\u003e\u003cb\u003eLimit sugary and acidic foods/beverages\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eStay hydrated with water;\u003c/b\u003e Drinking plenty of water\u003c/p\u003e \u003cp\u003e\u003cb\u003eLimit sugary and acidic foods and beverages;\u003c/b\u003e Sugar and acid can increase the risk of cavities and since braces make cleaning teeth more challenging, minimizing these foods is crucial.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eStay hydrated with water;\u003c/b\u003e Water helps rinse away small food particles that might get trapped in your braces\u003c/p\u003e \u003cp\u003e\u003cb\u003eLimit sugary and acidic foods and beverages;\u003c/b\u003e Being mindful of consumption and practicing through dental hygiene after sugary and acidic foods and beverages is key\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eOrthodontists and nutritionists can offer personalized advice\u003c/b\u003e as your treatment progresses, so be sure to share any concerns you have during your visits\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCopilot\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003ecredibility\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eCompatible but insufficient\u003c/b\u003e: did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eCompatible but insufficient\u003c/b\u003e\u003c/p\u003e \u003cp\u003eOral hygiene tools were not mentioned\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eGemini\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eFoods to Avoid\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eSugary and Acidic Foods;\u003c/b\u003e Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices. Expand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.colgate.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.colgate.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eBalanced Diet;\u003c/b\u003e A well-rounded diet rich in essential vitamins and minerals is crucial for overall oral health and supports the effectiveness of orthodontic treatment. Expand/Google Search found similar content as follows: (A balanced diet with adequate nutrients is essential for a healthy mouth and in turn, a healthy mouth supports nutritional well-being.\u0026nbsp;nutritionsource.hsph.harvard.edu)\u003c/p\u003e \u003cp\u003e\u003cb\u003eFocus on fruits, vegetables, lean proteins, and dairy products.\u003c/b\u003e Expand (Consuming a diet high in fruits, vegetables, dairy products, and lean proteins gives you the vitamins and minerals your teeth need to be healthy.\u0026nbsp;\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.batemanorthodontics.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.batemanorthodontics.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eFoods to Enjoy\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eSoft Foods;\u003c/b\u003e Yogurt, smoothies, mashed potatoes, soups, and soft fruits are gentle on braces and aligners. Expand (Go Soft: opt for softer foods right after tightening your braces. Think mashed potatoes, yogurts, and smoothies. aaoinfo.org)\u003c/p\u003e \u003cp\u003e\u003cb\u003eSoft cooked vegetables.\u003c/b\u003e Expand (Soft food such as pasta, rice, soft bread, mashed potatoes, apple sauce, bananas, soft cooked chicken and seafood, meatballs, lunch meat, cooked vegetables. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.marksortho.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.marksortho.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eNutrient-Rich Foods; Dairy products\u003c/b\u003e (milk, cheese) provide calcium and vitamin D, essential for strong teeth and bones. Expand (Sources of calcium include: milk, cheese and other dairy foods \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.nhs.uk\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.nhs.uk\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eLean proteins\u003c/b\u003e (fish, chicken) support overall health. Expand (Lean animal proteins include white-fleshed fish, skinless poultry, and cuts of red meat such as loin and round. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.healthline.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.healthline.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003eFruits, cut into small portions.\u003c/p\u003e \u003cp\u003e\u003cb\u003eFoods to Avoid\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003eHard Foods\u003c/b\u003e: Nuts, hard candies, and crunchy vegetables can damage brackets and wires. Expand (Although nuts are a good option for your health, they are a terrible option for your braces. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.lakeminnetonkaortho.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.lakeminnetonkaortho.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eSticky and Chewy Foods\u003c/b\u003e: Caramel, chewing gum, and taffy can get lodged in braces and are difficult to remove. Expand (Avoid chewy candies, gum, caramel, and taffy. These foods can pull on wires and brackets, weakening the braces\u0026rsquo; structure. epicortho.com)\u003c/p\u003e \u003cp\u003e\u003cb\u003eSugary and Acidic Foods\u003c/b\u003e: Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices. Expand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.colgate.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.colgate.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eTough Meats\u003c/b\u003e: Tough or stringy meats can also cause damage to orthodontic appliances.\u003c/p\u003e \u003cp\u003eExpand (Your braces are precise appliances that can be damaged by eating hard foods. Some soft and sticky foods can cause tooth decay and loosen\u0026hellip; \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.burlingamesmilestudio.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.burlingamesmilestudio.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eHydration\u003c/b\u003e:\u003c/p\u003e \u003cp\u003e\u003cb\u003eWater\u003c/b\u003e: Drinking plenty of water is essential for maintaining oral hygiene and preventing dry mouth.\u003c/p\u003e \u003cp\u003eExpand (By drinking enough water, you help prevent dry mouth and ensure that your saliva is produced at an optimal rate. dentistry.uic.edu)\u003c/p\u003e \u003cp\u003eWater also helps to rinse away food particles.\u003c/p\u003e \u003cp\u003eExpand (Water, however, cleans your mouth with every sip. It washes away leftover food and residue that cavity-causing bacteria are looking for. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.mouthhealthy.org\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.mouthhealthy.org\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e\u003cb\u003eSugary and Acidic Foods\u003c/b\u003e: Sugary drinks and acidic foods can contribute to tooth decay and enamel erosion, which is a greater risk while wearing orthodontic devices.\u003c/p\u003e \u003cp\u003eExpand (As the enamel weakens, the likelihood of developing cavities becomes greater. Sugary drinks are known as one of the most common dietary causes of tooth decay. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.colgate.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.colgate.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eOral Hygiene;\u003c/b\u003e Maintaining excellent oral hygiene is paramount.\u0026nbsp;This includes thorough brushing and flossing to remove food particles that can become trapped in orthodontic appliances.\u003c/p\u003e \u003cp\u003eExpand (Proper oral hygiene is essential to minimize complications related to orthodontic miniscrews.\u0026nbsp;pmc.ncbi.nlm.nih.gov) (Careful brushing and flossing regularly can help remove any food particles or bacteria trapped in the orthodontic appliance or between the teeth.\u0026nbsp;theorthodonticplace.com.au)\u003c/p\u003e \u003cp\u003eRegular dental check-ups are also essential. Expand (These check-ups aren\u0026rsquo;t just routine. They\u0026rsquo;re an important part of making sure your orthodontic treatment is effective and your oral health is in good shape. southmiamiorthodontics.com)\u003c/p\u003e \u003cp\u003e\u003cb\u003eKey considerations;\u003c/b\u003e Orthodontic appliances create more places for food to become trapped, so diligent oral hygiene is essential.\u003c/p\u003e \u003cp\u003eExpand (Maintaining a diligent daily oral hygiene routine is critical when you have braces or aligners. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.smilesaversortho.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.smilesaversortho.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eKey considerations;\u003c/b\u003e It is very important to consult with your orthodontist, they can provide personalized dietary advice based on your specific treatment plan.\u003c/p\u003e \u003cp\u003eExpand (Patients receiving orthodontic treatment must have open lines of communication with their dentist and orthodontist on food preferences and any oral health issues. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ewww.batemanorthodontics.com\u003c/a\u003e\u003c/span\u003e\u003cspan address=\"http://www.batemanorthodontics.com\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eGemini credibility\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003e\u003cb\u003eCompatible but insufficient\u003c/b\u003e: did not include information for accumulation of food debris around braces, dental plaque, and related acidic environment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003eCompatible\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e\n\u003ch3\u003eCredibility of information resources\u003c/h3\u003e\n\u003cp\u003e \u003cstrong\u003eInformation credibility about tooth decay\u003c/strong\u003e \u003cp\u003eAAO web-source mentioned the need for swishing and rinsing with water, drinking water to prevent dental plaque to rinse away food debris, and acidic residues. The BOS leaflet contained detailed information about the effect of retained food, plaque, and sugary items on creating acidic environment which further causes tooth decays.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eChatGPT 4.0 contained recommendations regarding sticky and sugary foods mentioning risks of decalcification around the brackets, and cleaning difficulties. Oral care tips included minimizing snacks to neutralize acids, and to drink water frequently. Copilot emphasized the importance of drinking water emphasizing its benefits in increasing saliva flow and reducing acids that can cause tooth decay. Tooth decay was referred through avoiding sugary and acidic foods in Gemini, compatible with the guidelines. The responses of LLMs were compatible but insufficient, since neither of them gave a detailed explanation as in BOS, regarding sugary items effects on the bacterial plaque which further produce an acidic environment leading to tooth decay.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eInformation credibility about food\u003c/strong\u003e \u003cp\u003eBoth AAO and BOS gave detailed information on the types of foods to be preferred and avoided, especially for the initial days after the braces are inserted. Details included hard/crunchy food types, and reasons on why to pay attention to the requested foods. Both guidelines suggested consumption of soft foods requiring little or no chewing on first days. Only AAO web-source attracted attention on extreme temperatures to avoid tooth sensitivity, and provided essential tips during eating. AAO web-source also suggested sugar-free gums, to overcome soreness.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eConsistent information in ChatGPT-4.0 was evident for avoiding hard, sticky, sugary foods, excessive chewing, consuming calcium-rich foods, vitamins, minerals and healthy fats including information for foods that patients can eat and prepare more comfortably because of the braces. The response was compatible. More details were apparent regarding the need for soft foods especially for few days, essential nutrients referring to special fruits, vegetables, vitamins, proteins, whole grains, the need for avoiding hard, sticky foods in Copilot, which all provided a compatible response. In general, similar information was given in Gemini, declaring the need for a balanced diet rich in vitamins and minerals, fruits, vegetables, lean proteins and dairy products, as well as recommendations for consuming soft foods and soft cooked vegetables, and eating with small portions. The information for foods to avoid was detailed. Overall, the response was compatible.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eInformation credibility about beverages\u003c/strong\u003e \u003cp\u003eAll resources emphasized the need to drink plenty of water. AAO web-source recommended avoiding drinks with extreme temperatures to overcome tooth sensitivity especially during the initial stages of treatment. The BOS leaflet recommended consuming milk and plain water, avoiding acidic drinks, limiting fruit juices only at mealtimes, and using a straw while consuming such beverages.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eChatGPT 4.0 included examples of sugary drinks, like soda and juices, as drinks that should not be preferred. The effect of sugary drinks on the risk of cavities and decalcification were mentioned, and drinking water frequently was advised mentioning its\u0026rsquo; benefits on cleaning food debris and neutralize acids around the braces. The response was compatible. Copilot provided an explanatory response due to challenges of sugary and acidic beverages pointing out the risk of dental cavities, declaring a compatible response. Given as a separate heading, Gemini recommended to drink water to prevent dry mouth and contribute to oral hygiene, together with the warning for sugary drinks and acidic foods with a short explanation of tooth decay and enamel erosion risks. The response was compatible.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eInformation credibility about oral care\u003c/strong\u003e \u003cp\u003eAll resources mentioned the need for good oral hygiene. AAO web-source advised swishing water for rinsing away food debris and acidic residues after eating. Oral hygiene tools were given in detail, carrying travel toothbrush, or water flosser were mentioned. BOS leaflet suggested brushing for 2 minutes at least twice a day using a fluoride toothpaste.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eOral care tips are explained through recommendations as brushing and flossing after every meal, using interdental brushes and water flossers, drinking water frequently and minimizing snacking with ChatGPT 4.0. The responses were compatible. Rinsing away small food particles around the braces, practice of dental hygiene after consuming sugary and acidic foods or beverages were advised in Copilot. The response was in line, but the oral hygiene tools were not explained in detail. Gemini mentioned that orthodontic appliances create more places for food to become trapped, so diligent oral hygiene is essential, revealing the needs for brushing and flossing to remove food particles, and provided links for detailed information. So, the response was compatible.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eInformation credibility about further assistance\u003c/strong\u003e \u003cp\u003eAAO web-source emphasized that specific guidelines could be best provided by orthodontists, as their advices relied on years of experience and knowledge, further provided a referral opportunity to find the nearest orthodontist. BOS leaflet asked patients to refer to either an orthodontist, dentist or hygienist, in case of information needs. Besides, free links for patients to access further information were included.\u003c/p\u003e \u003c/p\u003e \u003cp\u003eChatGPT 4.0 did not contain any information for the need of consultation from orthodontists, which was incompatible with the guidelines. Copilot suggested patients to share their concerns with their orthodontists and nutritionists, and noted that they could receive personalized advice from them. The response was compatible with the guidelines. Gemini emphasized the importance of seeking personalized advice from orthodontists, which was found to be consistent with the guidelines.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eDISCERN scores\u003c/strong\u003e \u003cp\u003eThe DISCERN scores were calculated as mean, standard deviation, median, minimum and maximum, and presented in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. BOS leaflet had the highest mean DISCERN score (4.70\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27), followed by Gemini (4.54\u0026thinsp;\u0026plusmn;\u0026thinsp;1.03), AAO web-source (4.45\u0026thinsp;\u0026plusmn;\u0026thinsp;0.75), Copilot (3.87\u0026thinsp;\u0026plusmn;\u0026thinsp;1.64) and ChatGPT 4.0 (3.08\u0026thinsp;\u0026plusmn;\u0026thinsp;1.65). The results of the Friedman test indicated a statistically significant difference among the five groups (χ\u0026sup2; (4)\u0026thinsp;=\u0026thinsp;10.387, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.034). Following pairwise comparisons using the Wilcoxon signed-rank tests with Bonferroni correction indicating that none of the pairwise comparisons remained statistically significant after adjustment (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). The closest comparisons to significance were between ChatGPT 4.0 and BOS (adjusted p\u0026thinsp;=\u0026thinsp;0.398) and between ChatGPT 4.0 and Gemini (adjusted p\u0026thinsp;=\u0026thinsp;0.091). mean scores by resource by examiners are given in \u003cb\u003eFig.\u0026nbsp;1\u003c/b\u003e.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDescriptive statistics for DISCERN scores, and pairwise comparison among resources.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eResources\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eDISCERN Scores\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003eMean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003eMedian\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u003cb\u003e(Min-Max)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eP value\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAAO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.45 \u0026plusmn; 0.75\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.83\u003c/p\u003e \u003cp\u003e(3.00\u0026ndash;5.00)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003e0.034*\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBOS\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.70 \u0026plusmn; 0.28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.67\u003c/p\u003e \u003cp\u003e(4.33-5.00)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChatGPT 4.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.08 \u0026plusmn; 1.65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.50\u003c/p\u003e \u003cp\u003e(1.00\u0026ndash;5.00)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCopilot\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.87\u0026plusmn;1.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.83\u003c/p\u003e \u003cp\u003e(1.00\u0026ndash;5.00)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGemini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.54\u0026plusmn;1.03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5.00\u003c/p\u003e \u003cp\u003e(2.00\u0026ndash;5.00)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eSD, standard deviation, Min, minimum, Max, maximum. * Friedman test\u003c/p\u003e \u003cp\u003e \u003cb\u003eReadability scores\u003c/b\u003e: Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e demonstrated the results for FRES and FKGL levels for each source. AAO web-source readability was standard and/or plain with FRES score of 61.4, and the BOS leaflet was fairly easy to read with a 74.4 FRES score. The FRES levels of both guidelines revealed higher scores than the LLMs, showing that the responses were easier to read. The grades of FKGL revealed higher scores for Copilot and Gemini (Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eFlesch Reading Ease Scores and Flesch-Kincaid Grade Levels of each resource.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFlesch Reading Ease\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFlesch-Kincaid Grade Level\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBOS\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e74.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAAO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e61.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChatGPT 4.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e55.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCopilot\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e47.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGemini\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e42.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e9.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eAI chatbots connect with users through natural language processing and machine learning methodologies, aiming to improve access to healthcare services, deliver fast medical information, and reduce the burden on medical systems [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. In this regard, it\u0026rsquo;s crucial that clinicians examine criteria such as precision, reliability, quality, and readability of chatbot responses in the medical area [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. In this study, responses of different LLMs in terms of nutritional guidelines of orthodontic patients were compared with the AAO web-source and BOS leaflet. Present results showed that the responses of AI-based LLMs were understandable and compatible with the guidelines. In general, the null hypothesis was accepted, but several details especially regarding information for tooth decay were lacking in LLMs. The most detailed information about tooth decay was included in the BOS leaflet with a brief explanation of what tooth decay means. In other sources, there was not enough explanation about the mechanism of tooth decay risk caused by inadequate oral care due to orthodontic braces and the bacterial plaque. Eventually, the increased acidic environment and the effects of nutrition on these mechanisms were not fully mentioned. Avoiding hard, sticky, chewy and sugary foods were specified in all LLMs similar to the guidelines. Essential tips were provided in AAO web-source to avoid extreme temperatures to prevent tooth sensitivity especially at early days of braces, different than the other resources. Information regarding foods/beverages to avoid and prefer, and the balanced diet contents were compatible with the guidelines. Only the Copilot's response lacked information regarding oral hygiene tools. Concerning the necessity of referral to orthodontists, apart from ChatGPT 4.0, responses were sufficient in all resources.\u003c/p\u003e \u003cp\u003eOrthodontists' clinical experience, up-to-date knowledge and relationship with the patients enable them to convey more detailed and individualized information. Healthcare professionals are the most reliable source of information, but nowadays it is a fact that patients use the internet as a source of information [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. Patients seeking orthodontic treatment try to collect as much information as possible to facilitate their decision-making process [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]. Therefore, the use of internet as an information source is becoming increasingly widespread. It has even become the first source of information for some patients before consulting any specialist. A previous study reported that more than half of the individuals used social media and websites as a source of information for orthodontic treatment [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. Information-seeking through the internet depends also on patients' expectations from orthodontic treatment. Previously, differences were found between the expectations of patients and their parents in terms of expectations for treatment duration, demand for braces fitted at the initial visit, and dietary restrictions due to orthodontic treatment. The authors also reported that different ethnic groups exhibit different expectations regarding the initial orthodontic visit, and the impact of orthodontic treatment on nutrition [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. According to Firestone et al. [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] patients underestimated the required dietary changes as a result of pain associated with orthodontic treatment. Children expected significantly greater restrictions for food types that they could consume, than their parents, since they have learned the necessity of low-sugar and low-acid diet by consulting their friends who had previously received orthodontic treatment [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. This shows that patients seek various information before receiving treatment. AAO web-source provided information for the variety of healthy foods, how to consume them at the initial stages of treatment, with a detailed, empathetic and a conversational language. The importance of temperature of foods/drinks, possible relief with ice cream/sugar-free gums, and detailed useful tips for patients were also highlighted. The BOS leaflet put emphasis on avoiding sugary/hard foods and advised selecting healthy foods and beverages (advised using straw with fruit juices). All LLMs suggested consumption of calcium-rich foods for orthodontic patients. Similarly, Al Jawad et al. [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] recommended dairy products (milk, cheese, ice-cream), as they are soft and supportive in bone remodeling during tooth movement. Ozdemir et al. [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] noted a non-significant decrease in calcium and vitamin A intake in the third month of orthodontic treatment, compared to pretreatment values in adolescents. Additionally, the study found significantly decreased levels of vitamin C, vitamin E, and fiber intake, especially in the first weeks of orthodontic treatment, and declared that intake of these nutrients did not return to their initial levels even in the 12th week of orthodontic treatment. Only AAO web-source suggested sugar-free gums for reducing pain during fixed orthodontic treatment. A recent systematic review approved chewing gums to be effective for reducing orthodontic pain after 24 hours of initial wire placement in orthodontic patients [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e]. The need for good oral care was mentioned in all resources, with more details in the AAO web-source, such as the need for carrying travel toothbrush, or water flosser, and brushing and flossing especially after meals.\u003c/p\u003e \u003cp\u003eYeo et al. [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] noted that, unless explicitly stated, ChatGPT cannot distinguish specific guidelines relevant to a particular country. It can therefore be confusing if patients do not receive advice specific to their country, age, or existing problem. To avoid this, it is necessary to evaluate AI models according to limits and guidelines established by regulatory authorities or professional bodies [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. The need to consult an orthodontist to receive personalized advice and be able to ask further questions was suggested by all resources except for ChatGPT 4.0. In order to increase the satisfaction levels of patients, and meet their expectations, orthodontists and patients need to establish good communication [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]. This relationship starts with sharing information in initial orthodontic consultation. A study declared that good communication has significant effects on the success of orthodontic treatment [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]. As mentioned in a previous report, ChatGPT cannot ask questions to clarify the questions being asked [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]. Thus, the doctor-patient relationship remains the most important factor contributing to patient satisfaction [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]. The Copilot provided examples and answers to questions that patients may have about nutrition, such as \u0026ldquo;What happens if I don\u0026rsquo;t follow these guidelines?\u0026rdquo;, \u0026ldquo;What are the signs of potential damage to my braces?\u0026rdquo; and \u0026ldquo;What specific foods should I include in my diet?\u0026rdquo; The Gemini provided an expand option for the information given, by showing the link or the literature from which the information was obtained. Although these features provide superiority, they cannot replace the individual recommendations that can be given by orthodontists. Despite the fast and easily achievable response of LLMs, it would be useful to evaluate the consistency of their responses to different clinical scenario questions in further studies. Previously, most of the ChatGPT 4.0 responses about eye health were found to be appropriate, but it was emphasized that these platforms are not a source of factual information in their current form. Finally, it was stated that patients should limit their search for health-related information online [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. In Stevens et al.'s [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] study, 8% of the patients preferred to obtain online information, which was attributed to their concerns about the reliability of the information. Recommendations of the guidelines of various professional societies may vary. Additionally, the fact that information seeking behaviors may differ depending on different social and cultural expectations, reveals the importance of receiving face-to-face information, especially on medical issues.\u003c/p\u003e \u003cp\u003eSources of information, citations, and references are crucial in establishing the reliability of health information, and these factors were assessed in our study using the modified DISCERN tool. With respect to DISCERN scores, higher score was achieved in BOS leaflet (4.70\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27) followed by Gemini (4.54\u0026thinsp;\u0026plusmn;\u0026thinsp;1.03) and AAO web-source (4.45\u0026thinsp;\u0026plusmn;\u0026thinsp;0.75). Copilot (3.87\u0026thinsp;\u0026plusmn;\u0026thinsp;1.64) and ChatGPT 4.0 (3.08\u0026thinsp;\u0026plusmn;\u0026thinsp;1.65) had lower scores. However, pairwise comparisons did not yield statistically significant results. This may be attributed to the small sample size, which limits statistical power, and the conservative nature of the Bonferroni correction. Moreover, the limited score variability and possible similarity across systems may have reduced the ability to detect meaningful differences.\u003c/p\u003e \u003cp\u003eA study compared four LLMs about clear aligners by DISCERN tool and Copilot was found to be the most reliable, followed by Gemini, ChatGPT 4.0 and ChatGPT 3.5. Researchers reported that Copilot utilized 87 references in all inquiries. Gemini used only four references in four questions. The research also indicated that the majority of Copilot's citations and all of Gemini's citations were from websites [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. In contrast, in our study only Gemini utilized references in all presented information. However, the references were mostly from dental clinic websites and firms, instead of scientific sources/articles in this study. A significant limitation for chatbots such as ChatGPT 3.5 and ChatGPT 4.0 has been noted as their inability to provide references or citations for the information they produce [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. This is evident from ChatGPT 4.0\u0026rsquo;s low scores and Gemini\u0026rsquo;s high scores on the DISCERN question \u0026lsquo;specifying the information sources used in the creation of the content\u0026rsquo;. Another study assessing responses to clinical orthodontics inquiries indicated that, Copilot's answers achieved the greatest ratings, followed by ChatGPT 4.0, Gemini, and lastly ChatGPT 3.5. Nevertheless, according to researchers, all models occasionally generated responses that lacked comprehensiveness, scientific accuracy, clarity, and relevance [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Conversely Daraqel et al. [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e] stated that, ChatGPT 3.5 and Google Bard models have generally produced reasonably accurate and complete responses to posed general orthodontic inquiries asked by patients. The observed difference is believed to be associated with the content of the questions. Makrygiannakis et al. [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] posed detailed and technical inquiries regarding clinical orthodontics, whereas Daraqel et al. [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e] included questions commonly raised by patients concerning general orthodontics.\u003c/p\u003e \u003cp\u003eHealth literacy is the skill of being able to read and understand knowledge about health, which helps individuals make decisions and control their treatment processes [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Readability has been emphasized in the literature as a critical component of health literacy, facilitating comprehension of documents. Young people and adolescents who require information at their level, both in-person and online, represent the majority of orthodontic populations worldwide [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. To improve health literacy, the US Department of Health and Human Services has advised that patient-targeted materials should not be more complex than a sixth-grade reader [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. The FRES, introduced in 1948, is used to assess text readability, which indicates the estimated educational level required for an individual to understand a specific text with ease. The clarity of a document is represented by a numerical value ranging from 0 to 100, and scores near \u0026ldquo;100\u0026rdquo; show that the material is highly simple to read, while scores near \u0026ldquo;0\u0026rdquo; indicate the complexity to comprehend [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. The FKGL is a metric based on average sentence length and word complexity revealing the educational level required for an individual to understand a specific text [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. According to our findings, AAO web-source and BOS leaflet were simpler to read than the LLMs. Similarly, a research evaluating the quality of leaflets from two international orthodontic societies for orthodontic patients reported a median FRES of 70 [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. Although ChatGPT 4.0 had the highest FRES score among the LLMs in this study, it was found that all currently assessed AI chatbots were below 60 points declaring that they are difficult to read. A study evaluating the readability of information by different LLMs utilizing FRES, reported similar findings [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. In terms of orthodontics, another study revealed that general responses by ChatGPT were classified as very difficult to read, and treatment-related questions were classified as difficult to read [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. With respect to the written content's readability, it was suggested that the United States and the United Kingdom should ideally aim at eighth graders and fifth to sixth graders, respectively [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. In accordance, the mean FKGL for AAO web-source was 7.6 and 6 for BOS leaflet. Leaflets from two international orthodontic societies found similar FKGL scores with 7.2 [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. Copilot and Gemini demonstrated the highest FKGL scores. It implies that the responses of the Copilot and Gemini are less comprehensible than those of the other sources. Kılın\u0026ccedil; et al. [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] demonstrated that the mean FKGL for general orthodontic inquiries received from ChatGPT was 14.52\u0026thinsp;\u0026plusmn;\u0026thinsp;1.48, whereas the score for treatment-related questions was 11.90\u0026thinsp;\u0026plusmn;\u0026thinsp;2.08.\u003c/p\u003e \u003cp\u003eIt is believed that the general public's capacity to use chatbots will be limited by their readability issues. Future studies may aim to assess the correlation between complexity and readability of written texts. Websites created by experts may improve the performance of language models trained on diverse internet data by providing high-quality, clear, and comprehensible health information [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. The advantage of this study can be considered as examining the same topic at different times and with different AI-based language models. One of the most important shortcomings was that the information sources in ChatGPT and Copilot were unclear. It would be advisable to use more objective categories to evaluate the LLMs' responses to ensure consistency. Additionally, this study was conducted only in English, where other languages and the intelligibility of these models by individuals with different educational levels could be the subject of future studies. Overall, further studies are still needed to determine which topics patients seek information about most regarding orthodontic treatment, and to make comprehensive evaluations by including newly developed models.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eThe credibility of LLMs was found to be adequate in terms of basic information on dietary recommendations, but details on the relationship between nutrition and tooth decay mechanisms were lacking.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eBOS leaflet had the highest mean DISCERN score followed by Gemini and AAO web-source. Average reliability was found for Copilot and ChatGPT 4.0 responses when compared to the guidelines according to the DISCERN tool. The responses of LLMs had a difficult reading level according to the FRES and FKGL.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003ePatients' dietary habits and treatment requirements vary greatly, so despite the general and understandable responses of LLMs, it is ideal for orthodontists to provide detailed information about treatment content, dietary changes or oral care recommendations in a patient and treatment-specific manner.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eBOS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eThe British Orthodontic Society\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eAAO\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAssociation of Orthodontists\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eAI\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eartificial intelligence\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eChatGPT\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eChat Generative Pre-Trained Transformer\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eLLMs\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eLarge Language Models\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eLaMDA\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eLanguage Model for Dialogue Applications\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePaLM 2 LLM\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ePathways Language Model 2\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEthical approval was not relevant, as this study required no human/animal materials.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data used and/or analyzed during the current study are available from the corresponding author on reasonable request ([email protected]).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualization and Investigation, BBT, TS; Supervision, BBT, CT; Original draft, BBT, TS; Review and editing, BBT; Analyzing the data, BBT, CT, TS; Statistical Analysis, İK.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eTervonen MM, Pirttiniemi P, Lahti S. Development of a measure for orthodontists to evaluate patient compliance. Am J Orthod Dentofac Orthop. 2011;139(6):791\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarter LA, Geldenhuys M, Moynihan PJ, Slater DR, Exley CE, Rolland SL. The impact of orthodontic appliances on eating young people\u0026rsquo;s views and experiences. J Orthodont. 2015;42(2):114\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChoi SH, Cha JY, Lee KJ, Yu HS, Hwang CJ. Changes in psychological health, subjective food intake ability and oral health-related quality of life during orthodontic treatment. J Rehabil. 2017;44(11):860\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarshall TA. Chairside diet assessment of caries risk. J Am Dent Association. 2009;140(6):670\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSudjalim T, Woods M, Manton D. Prevention of white spot lesions in orthodontic practice: a contemporary review. Aust Dent J. 2006;51(4):284\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAljohani SR, Alsaggaf DH. Adherence to dietary advice and oral hygiene practices among orthodontic patients. Patient Prefer Adherence. 2020;1991\u0026ndash;2000.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbed Al Jawad F, Cunningham SJ, Croft N, Johal A. A qualitative study of the early effects of fixed orthodontic treatment on dietary intake and behaviour in adolescent patients. Eur J Orthod. 2012;34(4):432\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBritish Orthodontic Society. Patient information leaflet: teeth and brace-friendly food and drink. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.bos.org.uk/wp-content/uploads/2022/03/British-Orthodontic-Society-Food-and-DrinkMarch2019.pdf\u003c/span\u003e\u003cspan address=\"https://www.bos.org.uk/wp-content/uploads/2022/03/British-Orthodontic-Society-Food-and-DrinkMarch2019.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2019) Accessed 19 Feb 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAmerican Association of Orthodontists. What Can I Eat with Braces? A Comprehensive Guide. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://aaoinfo.org/blog/what-can-i-eat-with-braces/\u003c/span\u003e\u003cspan address=\"https://aaoinfo.org/blog/what-can-i-eat-with-braces/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023) Accessed 19 Feb 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrispino R, Mannocci A, Dilena IA, Sides J, Forchini F, Asif Alherawi WM, Frazier-Bowers SA, Grippaudo C. Orthodontic patients and the information found on the web: a cross-sectional study. BMC Oral Health. 2023;23(1):860.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEggmann F, Blatz MB. ChatGPT: Chances and Challenges for Dentistry. Compendium Continuing Educ Dentistry (15488578). 2023;44(4).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMakrygiannakis MA, Giannakopoulos K, Kaklamanos EG. Evidence-based potential of generative artificial intelligence large language models in orthodontics: A comparative study of ChatGPT, Google Bard, and Microsoft Bing. Eur J Orthod. 2024;cjae017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRahaman MS, Ahsan M, Anjum N, Rahman MM, Rahman MN. The AI race is on! Google's Bard and OpenAI's ChatGPT head to head: an opinion article. Mizanur and Rahman, Md Nafizur,. ; 2023. The AI Race is on.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDursun D, Bilici Ge\u0026ccedil;er R. Can artificial intelligence models serve as patient information consultants in orthodontics? BMC Med Inf Decis Mak. 2024;24(1):211.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Translational Med. 2023;13(3):e1216.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAcar AH. Can natural language processing serve as a consultant in oral surgery? J Stomatology Oral Maxillofacial Surg. 2024;125(3):101724.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKılın\u0026ccedil; DD, Mansız D. Examination of the reliability and readability of Chatbot Generative Pretrained Transformer\u0026rsquo;s (ChatGPT) responses to questions about orthodontics and the evolution of these responses in an updated version. Am J Orthod Dentofac Orthop. 2024;165(5):546\u0026ndash;55.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoodman RS, Patrinely JR, Stone CA, Zimmerman E, Donald RR, Chang SS, Berkowitz ST, Finn AP, Jahangir E, Scoville EA. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open. 2023;6(10):e2336483\u0026ndash;2336483.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarlas T, Altinova AE, Akturk M, Toruner FB. Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines. Int J Obes. 2024;48(2):271\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGupta S, Tarapore R, Haislup B, Fillar A. Microsoft copilot provides more accurate and reliable information about anterior cruciate ligament injury and repair than ChatGPT and Google Gemini; however, no resource was overall the best. Arthrosc Sports Med Rehabilitation. 2025;7(2):101043.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbu Arqub S, Al-Moghrabi D, Allareddy V, Upadhyay M, Vaid N, Yadav S. Content analysis of AI-generated (ChatGPT) responses concerning orthodontic clear aligners. Angle Orthod. 2024;94(3):263\u0026ndash;72.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCharnock D, Shepperd S. Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res. 2004;19(4):440\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFlesch R. A new readability yardstick. J Appl Psychol. 1948;32(3):221.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu J, Luo E, Song E, Xu X, Tan H, Zhao Y, Wang Y, Li Z. Patients\u0026rsquo; attitudes towards online dental information and a web-based virtual reality program for clinical dentistry: A pilot investigation in China. Int J Med Informatics. 2009;78(3):208\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWahab RMA, Hasan SK, Yamin NEM, Ibrahim Z. Awareness of fake braces usage among Y-generations. J Int Dent Med Res. 2019;12(2):663\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKamarozaman DMS, Kamarudin Y, Hamzah TNNT, Nor NAM. Self-perceived Orthodontic Need, Information Seeking Behaviour and Knowledge on Orthodontic Treatment Among Young Malaysian Adults. J Health Translational Med (JUMMEC). 2020;23(2):8\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSayers M, Newton J. Patients\u0026rsquo; expectations of orthodontic treatment: Part 2\u0026mdash;findings from a questionnaire survey. J Orthodont. 2007;34(1):25\u0026ndash;35.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFirestone AR, Scheurer PA, B\u0026uuml;rgin WB. Patients' anticipation of pain and pain-related side effects, and their perception of pain as a result of orthodontic treatment with fixed appliances. Eur J Orthod. 1999;21(4):387\u0026ndash;96.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOzdemir M, Ilhan A, Gorucu-Coskuner H, Taner T, Bilgic P. Assessment of food consumption changes in adolescents during orthodontic treatment. Am J Orthod Dentofac Orthop. 2021;159(5):604\u0026ndash;12.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMando M, Talaat S, Bourauel C. The efficacy of chewing gum in the reduction of orthodontic pain at its peak intensity: a systematic review and meta-analysis. Angle Orthod. 2023;93(5):580\u0026ndash;90.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYeo YH, Samaan JS, Ng WH, Ting P-S, Trivedi H, Vipani A, Ayoub W, Yang JD, Liran O, Spiegel B. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol. 2023;29(3):721.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChiang YC, Wu F, Ko S-H. Effective patient\u0026ndash;dentist communication with a Simulation System for Orthodontics. Healthc MDPI. 2023;11(10):1433.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKeles F, Bos A. Satisfaction with orthodontic treatment. Angle Orthod. 2013;83(3):507\u0026ndash;11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHoward A, Hope W, Gerada A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect Dis. 2023;23(4):405\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMomenaei B, Wakabayashi T, Shahlaee A, Durrani AF, Pandit SA, Wang K, Mansour HA, Abishek RM, Xu D, Sridhar J. Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases. Ophthalmol Retina. 2023;7(10):862\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStephens R, Ryan FS, Cunningham SJ. Information-seeking behavior of adolescent orthodontic patients. Am J Orthod Dentofac Orthop. 2013;143(3):303\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMohammad-Rahimi H, Ourang SA, Pourhoseingholi MA, Dianat O, Dummer PMH, Nosrat A. Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. Int Endod J. 2024;57(3):305\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDaraqel B, Wafaie K, Mohammed H, Cao L, Mheissen S, Liu Y, Zheng L. The performance of artificial intelligence models in generating responses to general orthodontic questions: ChatGPT vs Google Bard. Am J Orthod Dentofac Orthop. 2024;165(6):652\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOey CG, Livas C. The informative value and design of orthodontic practice websites in The Netherlands. Prog Orthodont. 2020;21:1\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMeade MJ, Dreyer CW. Orthodontic treatment consent forms: A readability analysis. J Orthodont. 2022;49(1):32\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeehra J, Cockerham L, Pandis N. A quality assessment of orthodontic patient information leaflets. Prog Orthodont. 2016;17:1\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWilliamson JML, Martin AG. Analysis of patient information leaflets provided by a district general hospital by the Flesch and Flesch\u0026ndash;Kincaid method. Int J Clin Pract. 2010;64(13):1824\u0026ndash;31.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKenny T, Wilson R, Purves I, Clark Sr J, Newton L, Newton D, Moseley D. A PIL for every ill? Patient information leaflets (PILs): a review of past, present and future use. Fam Pract. 1998;15(5):471\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-oral-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ohea","sideBox":"Learn more about [BMC Oral Health](http://bmcoralhealth.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ohea/default.aspx","title":"BMC Oral Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Orthodontics, nutrition, artificial intelligence, large language models (LLMs), ChatGPT, Copilot, Gemini, Leaflet","lastPublishedDoi":"10.21203/rs.3.rs-6792273/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6792273/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eTo evaluate the credibility of large language models (LLMs) compared to American Association of Orthodontists (AAO) and British Orthodontic Society (BOS) guides regarding nutritional guidelines for orthodontic patients.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eThe responses offered by ChatGPT 4.0, Copilot and Gemini were assessed for information credibility about tooth decay, food, beverages, oral care, and further assistance as; compatible, compatible but insufficient, partially incompatible and incompatible compared to the guidelines. Reliability was analyzed by the DISCERN tool. The readability of the sources was assessed using the Flesch Reading Ease Score and the Flesch\u0026ndash;Kincaid Grade Level. The Friedman test was conducted to compare DISCERN scores.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eResponses of LLMs were understandable and compatible with the guidelines, but detailed information on tooth decay, dental plaque and risks of acidic environments were inadequate. ChatGPT 4.0, Copilot and Gemini provided detailed lists of foods to avoid and include. Only AAO suggested being aware of extreme temperatures and usage of sugar-free gums. All sources mentioned the necessity of good oral hygiene, but oral hygiene tools were not mentioned in Copilot. All, except ChatGPT 4.0, recommended orthodontist consultation for personalized advice. BOS leaflet had the highest mean DISCERN score (4.70\u0026thinsp;\u0026plusmn;\u0026thinsp;0.27), followed by Gemini (4.54\u0026thinsp;\u0026plusmn;\u0026thinsp;1.03), AAO web-source (4.45\u0026thinsp;\u0026plusmn;\u0026thinsp;0.75), Copilot (3.87\u0026thinsp;\u0026plusmn;\u0026thinsp;1.64) and ChatGPT 4.0 (3.08\u0026thinsp;\u0026plusmn;\u0026thinsp;1.65), revealing no significant difference. BOS and AAO were more readable than the LLMs. ChatGPT 4.0 was more readable among the LLMs but was still found to be difficult for the readers.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eGuidelines have a superior narrative in terms of their detailed content and especially the justifications for the recommendations. AI-supported LLMs provided understandable, simple and accurate information, despite lack of some details on certain topics. The readability of the responses from LLMs was difficult. Overall, patients should be advised that pre-trained algorithms should be used with caution as a source of information and that they should receive individual information from their orthodontists.\u003c/p\u003e","manuscriptTitle":"Assessment of the credibility of generative artificial intelligence large language models on nutritional guidelines for orthodontic patients","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-30 08:06:17","doi":"10.21203/rs.3.rs-6792273/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-08-26T11:35:47+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"114221102340099872444788692066696615242","date":"2025-08-25T12:38:53+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-24T15:38:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"663801663411640283341233956587094264","date":"2025-08-23T14:52:43+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"40272661359823150848433864615795240406","date":"2025-08-23T08:26:30+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-06T09:56:21+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"240420378717119253869825710633639414935","date":"2025-07-29T06:21:55+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-06-23T13:52:54+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-06-04T08:11:06+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-06-02T11:38:24+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-02T11:34:59+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Oral Health","date":"2025-05-31T17:07:30+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-oral-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ohea","sideBox":"Learn more about [BMC Oral Health](http://bmcoralhealth.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ohea/default.aspx","title":"BMC Oral Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"dd9d7010-0cf0-410a-bf29-37f89f9ef2ec","owner":[],"postedDate":"June 30th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-12-01T16:02:06+00:00","versionOfRecord":{"articleIdentity":"rs-6792273","link":"https://doi.org/10.1186/s12903-025-07153-1","journal":{"identity":"bmc-oral-health","isVorOnly":false,"title":"BMC Oral Health"},"publishedOn":"2025-11-26 15:57:52","publishedOnDateReadable":"November 26th, 2025"},"versionCreatedAt":"2025-06-30 08:06:17","video":"","vorDoi":"10.1186/s12903-025-07153-1","vorDoiUrl":"https://doi.org/10.1186/s12903-025-07153-1","workflowStages":[]},"version":"v1","identity":"rs-6792273","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6792273","identity":"rs-6792273","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}