Concordance Between the DeepSeek-V3 Language Model and Multidisciplinary Team Recommendations in Lung Cancer: A Retrospective Study

Concordance Between the DeepSeek-V3 Language Model and Multidisciplinary Team Recommendations in Lung Cancer: A Retrospective Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Concordance Between the DeepSeek-V3 Language Model and Multidisciplinary Team Recommendations in Lung Cancer: A Retrospective Study Yihan ZHao, Fangqi Yuan, Lingli Wang, Meifang Wang, Long Zhang, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9109873/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background The complexity of lung cancer multidisciplinary team (MDT) decision-making necessitates tools that can efficiently synthesize clinical data. Evaluating the concordance between large language model (LLM)-generated recommendations and MDT decisions is critical for clinical integration. Objective This study aimed to evaluate the overall and subgroup concordance between treatment recommendations generated by DeepSeek-V3 (the predominant clinical LLM in mid-2025) and the consensus decisions of institutional MDT, and to assess the clinical quality and utility of the model’s outputs via expert appraisal. Methods In this retrospective cohort study, 100 consecutive lung cancer patients were included. Identical anonymized clinical data were processed through DeepSeek-V3 (the predominant LLM version in clinical deployment as of June 2025) configured as a clinical decision support system, and reviewed by the institutional MDT. The primary outcome was the overall concordance of treatment recommendations measured by Cohen's Kappa. Secondary analyses included subgroup concordance by molecular markers and quality assessment via 5-point Likert scales by two independent oncologists. Results DeepSeek-V3 demonstrated substantial concordance with MDT recommendations (κ = 0.789, 95% CI: 0.723–0.855). Discordances primarily occurred between localized treatment modalities (12/16 discordant cases between definitive chemoradiotherapy and surgery ± adjuvant therapy, all of which were locally advanced NSCLC with high surgical risk factors). Subgroup Kappa values ranged from 0.55 to 0.83 across molecular phenotypes. Independent experts rated the model's outputs highly for guideline adherence (mean score 4.5 ± 0.6) and clinical utility (4.3 ± 0.7), with excellent inter-rater reliability (Spearman's ρ > 0.76, p < 0.001). Conclusion DeepSeek-V3 showed substantial concordance with MDT treatment recommendations in lung cancer, with outputs considered clinically relevant by domain experts. This supports its potential role as an assistive tool in MDT settings. Large Language Model DeepSeek-V3(2025) Multidisciplinary Team Lung Cancer Clinical Decision Support Diagnostic Concordance Full Text Additional Declarations No competing interests reported. Supplementary Files ZhaoSupplementaryMaterials.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9109873","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":618503570,"identity":"06e65bac-4f79-447d-b098-06a3d3994bcf","order_by":0,"name":"Yihan ZHao","email":"","orcid":"","institution":"Hubei University of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Yihan","middleName":"","lastName":"ZHao","suffix":""},{"id":618503571,"identity":"fb6261ab-628a-4aa9-9701-d958173d1b62","order_by":1,"name":"Fangqi Yuan","email":"","orcid":"","institution":"Hubei University of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Fangqi","middleName":"","lastName":"Yuan","suffix":""},{"id":618503572,"identity":"99dcb1d0-12d7-46f4-aec3-1b778ae9a9a5","order_by":2,"name":"Lingli Wang","email":"","orcid":"","institution":"Hubei University of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Lingli","middleName":"","lastName":"Wang","suffix":""},{"id":618503573,"identity":"8f711555-3a70-4e77-8a8a-b7e614ff74b2","order_by":3,"name":"Meifang Wang","email":"","orcid":"","institution":"Taihe Hospital","correspondingAuthor":false,"prefix":"","firstName":"Meifang","middleName":"","lastName":"Wang","suffix":""},{"id":618503574,"identity":"95c2ca06-21af-4f9b-a1db-fa7202b208e6","order_by":4,"name":"Long Zhang","email":"","orcid":"","institution":"Hubei University of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Long","middleName":"","lastName":"Zhang","suffix":""},{"id":618503575,"identity":"418cd62a-debf-40ed-8f20-f488edc41e8b","order_by":5,"name":"Tao Ren","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0UlEQVRIiWNgGAWjYDCCA8icDwwSRGlhbAAz2BgYGGeQrIWZhxh38R1vfv7g4557cubyzQ8f2/yxyONvYH746AYeLZJnjhk2znhWbGzZxmZsnNsmUSxxAMjIwaPF4EYOYzPPgYTEDccYzKRzGyQSGw7wsEkTo6V+wzH2b9IWfyQS5xOrJcHgGI+ZNAObROIGQlpAfpk540CC4YZjOcWGvW0SiRsPE/ALMMQefPhwIEHe4PDxjQ9+/KlLnHccGHT4tGABzKQpHwWjYBSMglGABQAAGHVPbc87pEQAAAAASUVORK5CYII=","orcid":"","institution":"Taihe Hospital","correspondingAuthor":true,"prefix":"","firstName":"Tao","middleName":"","lastName":"Ren","suffix":""},{"id":618503576,"identity":"b2cab6b3-b9e4-44da-9d23-a40b482503dc","order_by":6,"name":"Hansheng Wang","email":"","orcid":"","institution":"Taihe Hospital","correspondingAuthor":false,"prefix":"","firstName":"Hansheng","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2026-03-13 03:53:30","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9109873/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9109873/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106746398,"identity":"1641a153-cafe-4c20-b2f1-ea0051da7477","added_by":"auto","created_at":"2026-04-13 05:42:09","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":954801,"visible":true,"origin":"","legend":"","description":"","filename":"ZhaoLungCancerDeepSeekManuscript2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9109873/v1_covered_5119235a-4d4d-4b6d-abb8-ba1a786211fe.pdf"},{"id":106645357,"identity":"fc4f568e-c005-477d-a6fa-1978d701a59d","added_by":"auto","created_at":"2026-04-10 19:55:30","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":435101,"visible":true,"origin":"","legend":"","description":"","filename":"ZhaoSupplementaryMaterials.docx","url":"https://assets-eu.researchsquare.com/files/rs-9109873/v1/6df3b05f57fd4d4c58f72658.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Concordance Between the DeepSeek-V3 Language Model and Multidisciplinary Team Recommendations in Lung Cancer: A Retrospective Study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Large Language Model, DeepSeek-V3(2025), Multidisciplinary Team, Lung Cancer, Clinical Decision Support, Diagnostic Concordance","lastPublishedDoi":"10.21203/rs.3.rs-9109873/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9109873/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eThe complexity of lung cancer multidisciplinary team (MDT) decision-making necessitates tools that can efficiently synthesize clinical data. Evaluating the concordance between large language model (LLM)-generated recommendations and MDT decisions is critical for clinical integration.\u003c/p\u003e\u003ch2\u003eObjective\u003c/h2\u003e \u003cp\u003eThis study aimed to evaluate the overall and subgroup concordance between treatment recommendations generated by DeepSeek-V3 (the predominant clinical LLM in mid-2025) and the consensus decisions of institutional MDT, and to assess the clinical quality and utility of the model\u0026rsquo;s outputs via expert appraisal.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eIn this retrospective cohort study, 100 consecutive lung cancer patients were included. Identical anonymized clinical data were processed through DeepSeek-V3 (the predominant LLM version in clinical deployment as of June 2025) configured as a clinical decision support system, and reviewed by the institutional MDT. The primary outcome was the overall concordance of treatment recommendations measured by Cohen's Kappa. Secondary analyses included subgroup concordance by molecular markers and quality assessment via 5-point Likert scales by two independent oncologists.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eDeepSeek-V3 demonstrated substantial concordance with MDT recommendations (κ\u0026thinsp;=\u0026thinsp;0.789, 95% CI: 0.723\u0026ndash;0.855). Discordances primarily occurred between localized treatment modalities (12/16 discordant cases between definitive chemoradiotherapy and surgery\u0026thinsp;\u0026plusmn;\u0026thinsp;adjuvant therapy, all of which were locally advanced NSCLC with high surgical risk factors). Subgroup Kappa values ranged from 0.55 to 0.83 across molecular phenotypes. Independent experts rated the model's outputs highly for guideline adherence (mean score 4.5\u0026thinsp;\u0026plusmn;\u0026thinsp;0.6) and clinical utility (4.3\u0026thinsp;\u0026plusmn;\u0026thinsp;0.7), with excellent inter-rater reliability (Spearman's ρ\u0026thinsp;\u0026gt;\u0026thinsp;0.76, p\u0026thinsp;\u0026lt;\u0026thinsp;0.001).\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eDeepSeek-V3 showed substantial concordance with MDT treatment recommendations in lung cancer, with outputs considered clinically relevant by domain experts. This supports its potential role as an assistive tool in MDT settings.\u003c/p\u003e","manuscriptTitle":"Concordance Between the DeepSeek-V3 Language Model and Multidisciplinary Team Recommendations in Lung Cancer: A Retrospective Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-10 19:55:26","doi":"10.21203/rs.3.rs-9109873/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7c1e1fa4-2389-42e7-a176-9af4fbd40247","owner":[],"postedDate":"April 10th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-04-13T05:41:37+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-10 19:55:26","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9109873","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9109873","identity":"rs-9109873","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}