Improving clinical expertise in large language models using electronic medical records

Improving clinical expertise in large language models using electronic medical records | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Improving clinical expertise in large language models using electronic medical records Lifeng Zhu, Jingping Liu, Jiacheng Wang, Weiyan Zhang, Sihang Jiang, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5285540/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Electronic medical records (EMRs) are essential in clinical practice. Although current medical large language models (LLMs) excel in tasks like US Medical Licensing Examination, they struggle with real-world clinical applications due to insufficient large-scale EMR data in their training, hindering their clinical expertise. To address this limitation, we proposed EMR-LLM, an LLM for clinical practice using EMRs. Firstly, we continually pre-trained a general LLM on medical corpora to enhance its domain knowledge. Then, we designed three categories of instruction tasks using EMRs: structure understanding, numerical understanding, and downstream tasks. Finally, we introduced an ability-boosting instruction-tuning method, which mimics human learning, progressing from simple to complex tasks while introducing a data replay strategy to retain learned knowledge. Experimental results demonstrated that EMR-LLM outperformed strong competitors on six EMR tasks, nine medical benchmarks, and three open-domain benchmarks. Moreover, in discharge summary generation, EMR-LLM achieved performance levels close to those of expert clinicians. Health sciences/Health care/Public health Health sciences/Health care/Health services Full Text Additional Declarations There is NO Competing Interest. Supplementary Files 3SupplementaryInformation.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5285540","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":368958363,"identity":"00e34e72-06a6-4619-9230-570aa64be35e","order_by":0,"name":"Lifeng Zhu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAo0lEQVRIiWNgGAWjYDACdsaHDxh4DpCihZnZ2IBkLWYSDAykaJFvZmarrpC5w8Df3p3A8HMHEVoYgVpunuF5xiBx5uwGxt4zRLmL/9jNBp7DDAYSuRuYGduI0MLGzMxWSJoWHqAWRtK0SDAzM0sCtfCA/HKwlxgt8u3NjB8bew7L8bf3bnzwkxgtYMDYw8ADog8QqwEIfpCgdhSMglEwCkYeAAAS7S3nJMIslQAAAABJRU5ErkJggg==","orcid":"","institution":"Shanghai Jiaotong University School of Medicine","correspondingAuthor":true,"prefix":"","firstName":"Lifeng","middleName":"","lastName":"Zhu","suffix":""},{"id":368958364,"identity":"310bfeb5-27dc-48b9-8538-2d180c922139","order_by":1,"name":"Jingping Liu","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jingping","middleName":"","lastName":"Liu","suffix":""},{"id":368958365,"identity":"e7bcf90e-6230-4083-bb29-7ee59bd3c11a","order_by":2,"name":"Jiacheng Wang","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jiacheng","middleName":"","lastName":"Wang","suffix":""},{"id":368958366,"identity":"5bb77048-d719-457d-9d86-f44b2750094d","order_by":3,"name":"Weiyan Zhang","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Weiyan","middleName":"","lastName":"Zhang","suffix":""},{"id":368958367,"identity":"ee96b1f5-5186-446a-8e10-b44e91f6cd53","order_by":4,"name":"Sihang Jiang","email":"","orcid":"","institution":"Fudan University","correspondingAuthor":false,"prefix":"","firstName":"Sihang","middleName":"","lastName":"Jiang","suffix":""},{"id":368958368,"identity":"73118f20-eeec-4bf2-a159-54216c3726b1","order_by":5,"name":"Hai Yang","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Hai","middleName":"","lastName":"Yang","suffix":""},{"id":368958369,"identity":"ecd8086d-4de5-4da9-97e8-406497ccc3a4","order_by":6,"name":"Chao Wang","email":"","orcid":"","institution":"Fudan University","correspondingAuthor":false,"prefix":"","firstName":"Chao","middleName":"","lastName":"Wang","suffix":""},{"id":368958370,"identity":"453fd43e-8964-42a0-aec6-7aeb9c472081","order_by":7,"name":"Qi Ye","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Qi","middleName":"","lastName":"Ye","suffix":""},{"id":368958371,"identity":"3456377a-fc65-48ae-ba3b-a63dbf408cf3","order_by":8,"name":"Tong Ruan","email":"","orcid":"","institution":"East China University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Tong","middleName":"","lastName":"Ruan","suffix":""},{"id":368958372,"identity":"b60ccb29-b7a3-4d50-bf55-55919a9b0eb6","order_by":9,"name":"Xinkai Rui","email":"","orcid":"","institution":"Shanghai Joao Tong University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Xinkai","middleName":"","lastName":"Rui","suffix":""},{"id":368958373,"identity":"1a5b1612-68f7-4212-9ec2-acb2f916f34d","order_by":10,"name":"Huajun Chen","email":"","orcid":"https://orcid.org/0000-0001-5496-7442","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Huajun","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2024-10-18 00:55:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5285540/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5285540/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":67848652,"identity":"cc11b116-892d-4845-bb98-053e5ce7e733","added_by":"auto","created_at":"2024-10-30 10:09:17","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":738761,"visible":true,"origin":"","legend":"","description":"","filename":"2Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5285540/v1_covered_d076610b-ccf4-4aee-a0fd-c6f2227eae2f.pdf"},{"id":67809282,"identity":"e4b90d21-999b-45f5-81f1-4742595c290d","added_by":"auto","created_at":"2024-10-30 02:37:45","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":40164,"visible":true,"origin":"","legend":"","description":"","filename":"3SupplementaryInformation.docx","url":"https://assets-eu.researchsquare.com/files/rs-5285540/v1/79a6bf2a2baf2a8f53010c57.docx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Improving clinical expertise in large language models using electronic medical records","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5285540/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5285540/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eElectronic medical records (EMRs) are essential in clinical practice. Although current medical large language models (LLMs) excel in tasks like US Medical Licensing Examination, they struggle with real-world clinical applications due to insufficient large-scale EMR data in their training, hindering their clinical expertise. To address this limitation, we proposed EMR-LLM, an LLM for clinical practice using EMRs. Firstly, we continually pre-trained a general LLM on medical corpora to enhance its domain knowledge. Then, we designed three categories of instruction tasks using EMRs: structure understanding, numerical understanding, and downstream tasks. Finally, we introduced an ability-boosting instruction-tuning method, which mimics human learning, progressing from simple to complex tasks while introducing a data replay strategy to retain learned knowledge. Experimental results demonstrated that EMR-LLM outperformed strong competitors on six EMR tasks, nine medical benchmarks, and three open-domain benchmarks. Moreover, in discharge summary generation, EMR-LLM achieved performance levels close to those of expert clinicians.\u003c/p\u003e","manuscriptTitle":"Improving clinical expertise in large language models using electronic medical records","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-30 02:37:40","doi":"10.21203/rs.3.rs-5285540/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"811e98c7-6b15-4151-ad17-caa3192bfc39","owner":[],"postedDate":"October 30th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":39259087,"name":"Health sciences/Health care/Public health"},{"id":39259088,"name":"Health sciences/Health care/Health services"}],"tags":[],"updatedAt":"2024-10-30T10:01:07+00:00","versionOfRecord":[],"versionCreatedAt":"2024-10-30 02:37:40","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5285540","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5285540","identity":"rs-5285540","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}