Evaluating ChatGPT’s Accuracy in Providing Screening Mammography Recommendations among Older Women: Artificial Intelligence and Cancer Communication

Evaluating ChatGPT’s Accuracy in Providing Screening Mammography Recommendations among Older Women: Artificial Intelligence and Cancer Communication | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Evaluating ChatGPT’s Accuracy in Providing Screening Mammography Recommendations among Older Women: Artificial Intelligence and Cancer Communication Dejana Braithwaite, Shama D. Karanth, Joel Divaker, Nancy Schoenborn, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3911155/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Mar, 2024 Read the published version in Journal of the American Geriatrics Society → Version 1 posted You are reading this latest preprint version Abstract Abstract Objective: The U.S. Preventive Services Task Force (USPSTF) recommends biennial screening mammography through age 74. Guidelines vary as to whether or not they recommended mammography screening to women aged 75 and older. This study aims to determine the ability of ChatGPT to provide appropriate recommendations for breast cancer screening in patients aged 75 years and older. Methods: 12 questions and 4 clinical vignettes addressing fundamental concepts about breast cancer screening and prevention in patients aged 75 years and older were created and asked to ChatGPT three consecutive times to generate 3 sets of responses. The responses were graded by a multi-disciplinary panel of experts in the intersection of breast cancer screening and aging . The responses were graded as ‘appropriate’, ‘inappropriate’, or ‘unreliable’ based on the reviewer’s clinical judgment, content of the response, and whether the content was consistent across the three responses . Appropriateness was determined through a majority consensus. Results: The responses generated by ChatGPT were appropriate for 11/17 questions (64%). Three questions were graded as inappropriate (18%) and 2 questions were graded as unreliable (12%). A consensus was not reached on one question (6%) and was graded as no consensus. Conclusions: While recognizing the limitations of ChatGPT, it has potential to provide accurate health care information and could be utilized by healthcare professionals to assist in providing recommendations for breast cancer screening in patients age 75 years and older. Physician oversight will be necessary, due to the possibility of ChatGPT to provide inappropriate and unreliable responses, and the importance of accuracy in medicine. Preventive Medicine Geriatrics & Gerontology Oncology Breast cancer screening artificial intelligence geriatrics Introduction ChatGPT 4 (Generative Pre-trained Transformer 4) 1 has captured the world’s attention due to its conversational ability; however, the extent to which Large Language Models can generate accurate health advice is not well established. Recent studies have demonstrated that ChatGPT can respond appropriately to basic questions regarding cancer screening and prevention. 2–5 Whether ChatGPT can accurately respond to complex questions regarding screening such as whether or not to screen women aged 75 and older for breast cancer, a clinical setting where guidelines do not recommend for or against screening due to insufficient evidence, remains unknown. 5 Therefore, we assessed the extent to which ChatGPT provided accurate advice regarding screening mammography among women aged 75+. Methods The study was performed in July 2023 using ChatGPT 4. Informed by prior ChatGPT studies of cancer screening, 2–5 we created 13 questions and four clinical vignettes that addressed key concepts regarding screening mammography and prevention among women aged 75+ (see Appendix ). We submitted each question to ChatGPT three times consecutively, recording each response. A multidisciplinary panel of six experts in the fields of general internal medicine, family medicine, geriatric medicine, population health, cancer control and radiology graded the appropriateness of ChatGPT responses. The response was graded "unreliable” if discrepancies or conflicting information were identified among the three answers provided by ChatGPT. Appropriateness was determined by majority consensus from the reviewers. Results ChatGPT responses were graded appropriate for 11/17 questions (64%) (Table 1 ). Responses to six questions were graded inappropriate (n = 3, 18%), unreliable (n = 2, 12%) or no consensus (n = 1, 6%). Mean word counts and Flesh-Kincaid readability grade level are reported in Table 1 . Specifically, the inappropriate responses contained incorrect advice on screening among women aged 75 + with mammographic dense breasts (Table 2 in Appendix) , decision tools to assist in deciding whether to undergo screening mammography after age 75; and the role of family history of breast cancer in deciding when to stop screening. Table 1 Evaluation of ChatGPT Recommendation for Fundamental Questions about Breast Cancer Screening in Patients aged 75 + by a multidisciplinary panel Question Reviewers Grade for Responses* Mean Word Count Flesch-Kincaid Readability Grade Level# At what age should I stop getting screened for breast cancer? ᵻ Appropriate 193 12.5 Should I continue breast cancer screening after age 75? Appropriate 210 15.7 If I stop breast cancer screening at age 75, will I die of breast cancer? Appropriate 259 13.6 How often do I need to get screened for breast cancer after age 75? Appropriate 270 13.9 Should I ever stop getting screened for breast cancer if my family member has breast cancer? Appropriate 283 13.9 What are the guideline recommendations for stopping breast cancer screening? No consensus 374 14.2 When should I stop getting screened for breast cancer if I have dense breasts? Inappropriate 290 13.9 I have lobular carcinoma in situ. When should I stop having mammograms? Appropriate 254 13.3 Does breast cancer screening after age 75 have any risks, downsides, or side effects? Unreliable 382 15.3 What are the potential harms of continuing breast cancer screening beyond age 74? Unreliable 384 15.1 Are there any tools to help me decide whether or not to have a mammogram after age 75? Inappropriate 277 15.4 What factors should patients and physicians consider when deciding when to stop breast cancer screening? Appropriate 410 14.1 Please provide me with a list of citations on breast cancer screening for women aged 75 and older. Appropriate 300 12.7 Clinical vignette : Emily, a 75-year old patient with severe dementia and a history of regular mammograms but no breast cancer. When should Emily stop getting screened for breast cancer? Appropriate 249 16.5 Clinical vignette : Jane, a 75-year-old patient who is in good overall health but has a history of breast cancer. When should Jane stop getting screened for breast cancer? Inappropriate 259 13.5 Clinical vignette : Linda, a 75-year-old woman with a previous abnormal mammogram and subsequent normal results. When should Linda stop getting screened for breast cancer? Appropriate 275 14.3 Clinical vignette : Mary, a 75-year-old patient who is in good overall health but has mammographically dense breasts. When should Mary stop getting screened for breast cancer? Appropriate 223 15.3 * Each question was submitted to ChatGPT 4 three times (total of three responses per question). ᵻ Example ChatGPT response: a) The age at which you should stop getting screened for breast cancer can vary depending on your overall health and personal risk factors. However, according to the guidelines from the American Cancer Society as of my last update in September 2021: · Women ages 40 to 44 should have the choice to start annual breast cancer screening with mammograms if they wish to do so. · Women age 45 to 54 should get mammograms every year. · Women 55 and older should switch to mammograms every 2 years, or they can continue yearly screening. Screening should continue as long as a woman is in good health and is expected to live at least 10 more years. It's important to note that these are general recommendations and individual risk factors such as family history, genetic predisposition (e.g. BRCA gene mutations), and personal health history could affect these guidelines. Always consult with your healthcare provider to make the most appropriate decision regarding cancer screenings. # The Flesch-Kincaid Readability Grade is a widely used readability formula which assesses the approximate reading grade level of a text Discussion ChatGPT-generated answers were generally correct or close to correct, demonstrating the chatbot’s potential in serving as a useful adjunct to patient education or decision-making. Specifically, this study found that 64% of the answers regarding screening mammography among women aged 75 + provided by ChatGPT were graded as ‘appropriate’ by the multidisciplinary expert panel. However, we observed a lower proportion of appropriate responses versus recent ChatGPT studies of cancer screening, which reported accuracy estimates ranging from 76–88%. 2–5 The lower accuracy rate of ChatGPT in this study, which may be due to the topic portending inherently more uncertainty than fundamental screening concepts that were examined in previous studies of breast cancer screening among younger women. 2 Key strengths of our study include the multidisciplinary panel of experts that evaluated ChatGPT responses and use of appropriateness ratings in response to questions and clinical vignettes based on published research regarding cancer screening advice. Notable limitations of this study include ChatGPT 4 being trained on data collected up to September 2021. Moreover, the readability of the chatbot's responses were lower than typically proposed by health literacy proponents, as evidenced by the Flesch-Kincaid readability grades, suggesting that the information may be less accessible to women with lower health literacy. Additional limitations of Large Language Models such as ChatGPT including output that may not be grounded in factual data, containing cancer misinformation. For example, our question “ When should I stop getting screened for breast cancer if I have dense breasts ?” generated a response that included guidelines from the American Cancer Society on supplemental screening for women with dense breasts. Crucially, these guidelines do not exist, and supplemental screening is not recommended. Given the American Cancer Society guidelines recommending screening with magnetic resonance imaging for younger women at high-risk (> 20–25%) lifetime risk, ChatGPT may have erroneously considered all older women with dense breasts to be at high enough risk to warrant this screening modality. Due to a dearth of evidence of the effectiveness of screening mammography among women aged 75+, it is clearly more challenging for ChatGPT to provide appropriate recommendations on topics such as screening mammography in this older population versus younger women. In conclusion, improving the accuracy of conversational artificial intelligence tools such as ChatGPT will be key to achieving their potential as patient education aids. Future research is needed to determine whether other Large Language Models provide accurate cancer screening advice consistently, and what the ideal infrastructure might be for future monitoring to ensure accuracy of cancer screening recommendations within the artificial intelligence ecosystem. Declarations Acknowledgments: This work was supported by the University of Florida Health Cancer Center (P30CA247796). Author Contributions Study Concept and Design: Dejana Braithwaite, Shama D. Karanth, Joel Divaker Acquistion of Data: Joel Divaker, Dejana Braithwaite Analysis of Data: Nancy Schoenborn, Kenneth Lin, Illana Richman, Bruno Hochhegger, Suzanne O’Neill, Mara Schonberg Preparation of Manuscript : Joel Divaker, Dejana Braithwaite, Shama D. Karanth, Conflict of Interest Statement: The authors declare no conflicts of interest. Sponsor’s Role: The sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data, and preparation or approval of the manuscript. References ChatGPT: optimizing language models for dialogue. Accessed December 11, 2022. https://openai.com/blog/chatgpt Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of Breast Cancer Prevention and Screening Recommendations Provided by ChatGPT. Radiology. 2023;307(4):e230424. Haver HL, Lin CT, Sirajuddin A, Yi PH, Jeudy J. Use of ChatGPT, GPT-4, and Bard to Improve Readability of ChatGPT's Answers to Common Questions on Lung Cancer and Lung Cancer Screening. AJR Am J Roentgenol. Jun 21 2023. Ghafoor S, Yahya A. “Assessing ChatGPT's Performance in Answering Cervical Cancer Screening Questions to ChatGPT-generated Clinical Vignettes: A Pilot Study”. Research Square; 2023. Emile SH, Horesh N, Freund M, et al. How appropriate are answers of online chat-based artificial intelligence (ChatGPT) to common questions on colon cancer? Surgery. 2023 Nov;174(5):1273-1275. Additional Declarations The authors declare no competing interests. Supplementary Files ChatGPTLetterAppendix.docx Cite Share Download PDF Status: Published Journal Publication published 14 Mar, 2024 Read the published version in Journal of the American Geriatrics Society → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3911155","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":270080889,"identity":"50d6b12f-9796-455e-99a7-791f23f2dea1","order_by":0,"name":"Dejana Braithwaite","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5UlEQVRIiWNgGAWjYDCCG0CcwHCAgYEZyPjAICcDpAyI0JIA0sLMwDiDwZiHOC0MIC1Aa5h5iNHCd7v34YeHP+4k9rfzH3xs22bAw8DevE0CnxbJO8eNJRISniXOOMzMbJwL0sJzrAyvFoMbaQxALYdzGw4zs0nntv3hYZDIMSOkhfkHSMv8w8zsvy1Btsi/IaiFDWzLBqAtzIwgLRI8+LVI3jnGZpGQdrh+42FmY8mecwY8bDxpxRb4tPDdbmO++cPmsLHc+YMPP/woM5DjZz+88QY+LZiAjTTlo2AUjIJRMAqwAQAnSkhmND6zgQAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0001-8376-5903","institution":"University of Florida","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Dejana","middleName":"","lastName":"Braithwaite","suffix":""},{"id":270080890,"identity":"ca7fa43b-cff8-471c-a391-531b64923732","order_by":1,"name":"Shama D. Karanth","email":"","orcid":"","institution":"University of Florida","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Shama","middleName":"D.","lastName":"Karanth","suffix":""},{"id":270080891,"identity":"2feb3fef-6a98-4c52-8d62-2a1251900fbf","order_by":2,"name":"Joel Divaker","email":"","orcid":"","institution":"University of Florida","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Joel","middleName":"","lastName":"Divaker","suffix":""},{"id":270080892,"identity":"a69def15-6eb2-41ee-95c3-c2fb5478a197","order_by":3,"name":"Nancy Schoenborn","email":"","orcid":"","institution":"Johns Hopkins Medical Center","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Nancy","middleName":"","lastName":"Schoenborn","suffix":""},{"id":270080893,"identity":"83e6e1aa-20f3-4c34-95e4-4416cd098def","order_by":4,"name":"Kenneth Lin","email":"","orcid":"","institution":"Penn Medicine Lancaster General Health","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Kenneth","middleName":"","lastName":"Lin","suffix":""},{"id":270080894,"identity":"a32c7b34-e027-4442-969a-6783a131e4ae","order_by":5,"name":"Ilana Richman","email":"","orcid":"","institution":"Yale University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ilana","middleName":"","lastName":"Richman","suffix":""},{"id":270080895,"identity":"d6adfdd0-3a0c-4ea3-971d-9a1a0749ec46","order_by":6,"name":"Bruno Hochhegger","email":"","orcid":"","institution":"University of Florida","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Bruno","middleName":"","lastName":"Hochhegger","suffix":""},{"id":270080896,"identity":"43310659-ffe8-466b-88e5-b65f1f956254","order_by":7,"name":"Suzanne O'Neill","email":"","orcid":"","institution":"Georgetown University School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Suzanne","middleName":"","lastName":"O'Neill","suffix":""},{"id":270080897,"identity":"00a5e5cd-f4b0-4978-a79a-b34a2ac74441","order_by":8,"name":"Mara Schonberg","email":"","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Mara","middleName":"","lastName":"Schonberg","suffix":""}],"badges":[],"createdAt":"2024-01-30 16:21:36","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false,"coiExplicitlySet":false},"doi":"10.21203/rs.3.rs-3911155/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3911155/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1111/jgs.18854","type":"published","date":"2024-03-14T04:28:01+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":59700396,"identity":"37ada8cc-54d6-4e03-951e-7c7160c989d8","added_by":"auto","created_at":"2024-07-05 04:28:05","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":358156,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3911155/v1/a4898e6e-0859-4177-b002-93a2ebe0d43e.pdf"},{"id":50447881,"identity":"0b93908a-942f-40a5-880f-fd4046f207a6","added_by":"auto","created_at":"2024-01-31 16:27:01","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":16071,"visible":true,"origin":"","legend":"","description":"","filename":"ChatGPTLetterAppendix.docx","url":"https://assets-eu.researchsquare.com/files/rs-3911155/v1/0047d1152d938352d7f308f1.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eEvaluating ChatGPT’s Accuracy in Providing Screening Mammography Recommendations among Older Women: Artificial Intelligence and Cancer Communication\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eChatGPT 4 (Generative Pre-trained Transformer 4)\u003csup\u003e1\u003c/sup\u003e has captured the world\u0026rsquo;s attention due to its conversational ability; however, the extent to which Large Language Models can generate accurate health advice is not well established. Recent studies have demonstrated that ChatGPT can respond appropriately to basic questions regarding cancer screening and prevention.\u003csup\u003e2\u0026ndash;5\u003c/sup\u003e Whether ChatGPT can accurately respond to complex questions regarding screening such as whether or not to screen women aged 75 and older for breast cancer, a clinical setting where guidelines do not recommend for or against screening due to insufficient evidence, remains unknown.\u003csup\u003e5\u003c/sup\u003e Therefore, we assessed the extent to which ChatGPT provided accurate advice regarding screening mammography among women aged 75+.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eThe study was performed in July 2023 using ChatGPT 4. Informed by prior ChatGPT studies of cancer screening,\u003csup\u003e2\u0026ndash;5\u003c/sup\u003e we created 13 questions and four clinical vignettes that addressed key concepts regarding screening mammography and prevention among women aged 75+ (see \u003cb\u003eAppendix\u003c/b\u003e). We submitted each question to ChatGPT three times consecutively, recording each response. A multidisciplinary panel of six experts in the fields of general internal medicine, family medicine, geriatric medicine, population health, cancer control and radiology graded the appropriateness of ChatGPT responses. The response was graded \"unreliable\u0026rdquo; if discrepancies or conflicting information were identified among the three answers provided by ChatGPT. Appropriateness was determined by majority consensus from the reviewers.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eChatGPT responses were graded appropriate for 11/17 questions (64%) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Responses to six questions were graded inappropriate (n\u0026thinsp;=\u0026thinsp;3, 18%), unreliable (n\u0026thinsp;=\u0026thinsp;2, 12%) or no consensus (n\u0026thinsp;=\u0026thinsp;1, 6%). Mean word counts and Flesh-Kincaid readability grade level are reported in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. Specifically, the inappropriate responses contained incorrect advice on screening among women aged 75\u0026thinsp;+\u0026thinsp;with mammographic dense breasts \u003cb\u003e(Table\u0026nbsp;2 in Appendix)\u003c/b\u003e, decision tools to assist in deciding whether to undergo screening mammography after age 75; and the role of family history of breast cancer in deciding when to stop screening.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eEvaluation of ChatGPT Recommendation for Fundamental Questions about Breast Cancer Screening in Patients aged 75\u0026thinsp;+\u0026thinsp;by a multidisciplinary panel\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eQuestion\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eReviewers Grade for Responses*\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMean Word Count\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eFlesch-Kincaid Readability Grade Level#\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAt what age should I stop getting screened for breast cancer? \u003csup\u003eᵻ\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e193\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e12.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eShould I continue breast cancer screening after age 75?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e210\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIf I stop breast cancer screening at age 75, will I die of breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e259\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHow often do I need to get screened for breast cancer after age 75?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e270\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eShould I ever stop getting screened for breast cancer if my family member has breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e283\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWhat are the guideline recommendations for stopping breast cancer screening?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNo consensus\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e374\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWhen should I stop getting screened for breast cancer if I have dense breasts?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInappropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e290\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eI have lobular carcinoma in situ. When should I stop having mammograms?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e254\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDoes breast cancer screening after age 75 have any risks, downsides, or side effects?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUnreliable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e382\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWhat are the potential harms of continuing breast cancer screening beyond age 74?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUnreliable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e384\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAre there any tools to help me decide whether or not to have a mammogram after age 75?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInappropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e277\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWhat factors should patients and physicians consider when deciding when to stop breast cancer screening?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e410\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePlease provide me with a list of citations on breast cancer screening for women aged 75 and older.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e300\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e12.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eClinical vignette\u003c/em\u003e: Emily, a 75-year old patient with severe dementia and a history of regular mammograms but no breast cancer. When should Emily stop getting screened for breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e249\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e16.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eClinical vignette\u003c/em\u003e: Jane, a 75-year-old patient who is in good overall health but has a history of breast cancer. When should Jane stop getting screened for breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInappropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e259\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eClinical vignette\u003c/em\u003e: Linda, a 75-year-old woman with a previous abnormal mammogram and subsequent normal results. When should Linda stop getting screened for breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e275\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eClinical vignette\u003c/em\u003e: Mary, a 75-year-old patient who is in good overall health but has mammographically dense breasts. When should Mary stop getting screened for breast cancer?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e223\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e* Each question was submitted to ChatGPT 4 three times (total of three responses per question).\u003c/p\u003e \u003cp\u003eᵻ Example ChatGPT response: a) The age at which you should stop getting screened for breast cancer can vary depending on your overall health and personal risk factors. However, according to the guidelines from the American Cancer Society as of my last update in September 2021:\u003c/p\u003e \u003cp\u003e\u0026middot; Women ages 40 to 44 should have the choice to start annual breast cancer screening with mammograms if they wish to do so.\u003c/p\u003e \u003cp\u003e\u0026middot; Women age 45 to 54 should get mammograms every year.\u003c/p\u003e \u003cp\u003e\u0026middot; Women 55 and older should switch to mammograms every 2 years, or they can continue yearly screening.\u003c/p\u003e \u003cp\u003eScreening should continue as long as a woman is in good health and is expected to live at least 10 more years.\u003c/p\u003e \u003cp\u003eIt's important to note that these are general recommendations and individual risk factors such as family history, genetic predisposition (e.g. BRCA gene mutations), and personal health history could affect these guidelines. Always consult with your healthcare provider to make the most appropriate decision regarding cancer screenings.\u003c/p\u003e \u003cp\u003e# The Flesch-Kincaid Readability Grade is a widely used readability formula which assesses the approximate reading grade level of a text\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eChatGPT-generated answers were generally correct or close to correct, demonstrating the chatbot\u0026rsquo;s potential in serving as a useful adjunct to patient education or decision-making. Specifically, this study found that 64% of the answers regarding screening mammography among women aged 75\u0026thinsp;+\u0026thinsp;provided by ChatGPT were graded as \u0026lsquo;appropriate\u0026rsquo; by the multidisciplinary expert panel. However, we observed a lower proportion of appropriate responses versus recent ChatGPT studies of cancer screening, which reported accuracy estimates ranging from 76\u0026ndash;88%.\u003csup\u003e2\u0026ndash;5\u003c/sup\u003e The lower accuracy rate of ChatGPT in this study, which may be due to the topic portending inherently more uncertainty than fundamental screening concepts that were examined in previous studies of breast cancer screening among younger women.\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eKey strengths of our study include the multidisciplinary panel of experts that evaluated ChatGPT responses and use of appropriateness ratings in response to questions and clinical vignettes based on published research regarding cancer screening advice. Notable limitations of this study include ChatGPT 4 being trained on data collected up to September 2021. Moreover, the readability of the chatbot's responses were lower than typically proposed by health literacy proponents, as evidenced by the Flesch-Kincaid readability grades, suggesting that the information may be less accessible to women with lower health literacy. Additional limitations of Large Language Models such as ChatGPT including output that may not be grounded in factual data, containing cancer misinformation. For example, our question \u0026ldquo;\u003cem\u003eWhen should I stop getting screened for breast cancer if I have dense breasts\u003c/em\u003e?\u0026rdquo; generated a response that included guidelines from the American Cancer Society on supplemental screening for women with dense breasts. Crucially, these guidelines do not exist, and supplemental screening is not recommended. Given the American Cancer Society guidelines recommending screening with magnetic resonance imaging for younger women at high-risk (\u0026gt;\u0026thinsp;20\u0026ndash;25%) lifetime risk, ChatGPT may have erroneously considered all older women with dense breasts to be at high enough risk to warrant this screening modality. Due to a dearth of evidence of the effectiveness of screening mammography among women aged 75+, it is clearly more challenging for ChatGPT to provide appropriate recommendations on topics such as screening mammography in this older population versus younger women.\u003c/p\u003e \u003cp\u003eIn conclusion, improving the accuracy of conversational artificial intelligence tools such as ChatGPT will be key to achieving their potential as patient education aids. Future research is needed to determine whether other Large Language Models provide accurate cancer screening advice consistently, and what the ideal infrastructure might be for future monitoring to ensure accuracy of cancer screening recommendations within the artificial intelligence ecosystem.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments:\u0026nbsp;\u003c/strong\u003eThis work was supported by the University of Florida Health Cancer Center (P30CA247796).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eStudy Concept and Design:\u003c/em\u003e Dejana Braithwaite, Shama D. Karanth, Joel Divaker\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAcquistion of Data:\u003c/em\u003e Joel Divaker, Dejana Braithwaite\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAnalysis of Data:\u0026nbsp;\u003c/em\u003eNancy Schoenborn, Kenneth Lin, Illana Richman, Bruno Hochhegger, Suzanne O’Neill, Mara Schonberg\u003c/p\u003e\n\u003cp\u003e\u003cem\u003ePreparation of Manuscript\u003c/em\u003e: Joel Divaker, Dejana Braithwaite, Shama D. Karanth,\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of Interest Statement:\u0026nbsp;\u003c/strong\u003eThe authors declare no conflicts of interest.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSponsor’s Role:\u003c/strong\u003e The sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data, and preparation or approval of the manuscript.\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eChatGPT: optimizing language models for dialogue. Accessed December 11, 2022. https://openai.com/blog/chatgpt\u003c/li\u003e\n \u003cli\u003eHaver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of Breast Cancer Prevention and Screening Recommendations Provided by ChatGPT. Radiology. 2023;307(4):e230424.\u003c/li\u003e\n \u003cli\u003eHaver HL, Lin CT, Sirajuddin A, Yi PH, Jeudy J. Use of ChatGPT, GPT-4, and Bard to Improve Readability of ChatGPT\u0026apos;s Answers to Common Questions on Lung Cancer and Lung Cancer Screening. AJR Am J Roentgenol. Jun 21 2023.\u003c/li\u003e\n \u003cli\u003eGhafoor S, Yahya A. \u0026ldquo;Assessing ChatGPT\u0026apos;s Performance in Answering Cervical Cancer Screening Questions to ChatGPT-generated Clinical Vignettes: A Pilot Study\u0026rdquo;. Research Square; 2023.\u003c/li\u003e\n \u003cli\u003eEmile SH, Horesh N, Freund M, et al. How appropriate are answers of online chat-based artificial intelligence (ChatGPT) to common questions on colon cancer? Surgery. 2023 Nov;174(5):1273-1275.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[{"identity":"7fb3ecd8-8355-495f-83af-2726e03630e0","identifier":"10.13039/100008275","name":"University of Florida Health","awardNumber":"P30CA247796","order_by":0}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"University of Florida Health Cancer Center","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Breast cancer, screening, artificial intelligence, geriatrics","lastPublishedDoi":"10.21203/rs.3.rs-3911155/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3911155/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eAbstract\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eObjective: \u003c/strong\u003eThe U.S. Preventive Services Task Force (USPSTF) recommends biennial screening mammography through age 74. Guidelines vary as to whether or not they recommended mammography screening to women aged 75 and older. This study aims to determine the ability of ChatGPT to provide appropriate recommendations for breast cancer screening in patients aged 75 years and older.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods: \u003c/strong\u003e12 questions and 4 clinical vignettes addressing fundamental concepts about breast cancer screening and prevention in patients aged 75 years and older\u003cstrong\u003e \u003c/strong\u003ewere created and asked to ChatGPT three consecutive times to generate 3 sets of responses. The responses were graded by a multi-disciplinary panel of experts in the intersection of breast cancer screening and aging\u003cstrong\u003e. \u003c/strong\u003eThe responses were graded as ‘appropriate’, ‘inappropriate’, or ‘unreliable’ based on the reviewer’s clinical judgment, content of the response, and whether the content was consistent across the three responses\u003cstrong\u003e. \u003c/strong\u003eAppropriateness was determined through a majority consensus.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults: \u003c/strong\u003eThe responses generated by ChatGPT were appropriate for 11/17 questions (64%). Three questions were graded as inappropriate (18%) and 2 questions were graded as unreliable (12%). A consensus was not reached on one question (6%) and was graded as no consensus.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions: \u003c/strong\u003eWhile recognizing the limitations of ChatGPT, it has potential to provide accurate health care information and could be utilized by healthcare professionals to assist in providing recommendations for breast cancer screening in patients age 75 years and older. Physician oversight will be necessary, due to the possibility of ChatGPT to provide inappropriate and unreliable responses, and the importance of accuracy in medicine.\u003c/p\u003e","manuscriptTitle":"Evaluating ChatGPT’s Accuracy in Providing Screening Mammography Recommendations among Older Women: Artificial Intelligence and Cancer Communication","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-31 16:26:57","doi":"10.21203/rs.3.rs-3911155/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8e9ecebf-9de2-4c8d-a336-9ea882cecf1b","owner":[],"postedDate":"January 31st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":28464347,"name":"Preventive Medicine"},{"id":28464348,"name":"Geriatrics \u0026 Gerontology"},{"id":28464349,"name":"Oncology"}],"tags":[],"updatedAt":"2024-07-05T04:28:01+00:00","versionOfRecord":{"articleIdentity":"rs-3911155","link":"https://doi.org/10.1111/jgs.18854","journal":{"identity":"journal-of-the-american-geriatrics-society","isVorOnly":true,"title":"Journal of the American Geriatrics Society"},"publishedOn":"2024-03-14 04:28:01","publishedOnDateReadable":"March 14th, 2024"},"versionCreatedAt":"2024-01-31 16:26:57","video":"","vorDoi":"10.1111/jgs.18854","vorDoiUrl":"https://doi.org/10.1111/jgs.18854","workflowStages":[]},"version":"v1","identity":"rs-3911155","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3911155","identity":"rs-3911155","version":["v1"]},"buildId":"cTy_lsJlmDsVRNrSptgXS","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}