Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field

Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field Abdulmalik Habaebi, Akeem Olowolayemo, Sharyar Wani This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7515191/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Purpose General-purpose large language models (LLMs) often lack the specialized accuracy required for the medical domain. This research aims to address this gap by developing and evaluating DocBot, a medical LLM, to demonstrate that domain-specific fine-tuning can significantly enhance performance even with constrained computational resources. Methods We fine-tuned the Meta LLaMa 3.1 8B model using Low-Rank Adaptation (LoRA), a parameter-efficient technique. The model was trained on a curated dataset of 2,000 patient-doctor dialogues sourced from ClinicalTrials, EMEA, and PubMed, using a single Tesla T4 GPU. Performance was evaluated against the base LLaMa model using BERTScore, BLEU, and ROUGE metrics, with responses from verified medical professionals serving as the reference. Results DocBot demonstrated significant improvements over the base LLaMa 3.1 8B model across all evaluation metrics. Specifically, DocBot achieved a higher BERTScore F1-score (83.56% vs. 81.47%), indicating enhanced semantic accuracy, fluency, and alignment with expert-generated text. The gains in precision and recall further confirm the model's superior ability to generate relevant and comprehensive medical information. Conclusion The successful development of DocBot showcases the feasibility and impact of creating domain-optimized LLMs efficiently. The results highlight the potential for specialized models to serve as reliable tools for augmenting clinical decision-making and delivering accessible medical support, particularly in resource-limited environments, paving the way for further innovation in specialized AI applications. Large Language Models (LLMs) Medical Chatbot Low-Rank Adaptation (LoRA) Fine-Tuning LLaMa 3 Parameter-Efficient Fine-Tuning (PEFT) Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7515191","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":509361320,"identity":"10e70db0-cb43-4c8d-9995-a587fb4bb57e","order_by":0,"name":"Abdulmalik Habaebi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyklEQVRIiWNgGAWjYDACCQY2IHmAhx/ESSggRYtkA0iLAQlaGAwOgHjEaNGd3fzswY+KOzLG51cnfnhgwCDPL3YAvxazO8fMDXvOPOMxu/F2swTQYYYzZycQ0HIjwUyCt+0wUMvZDSAtCQa3CWpJ/yb5999hHuMZZzf/IFJLjpk0b8NhHgP+3m3E2pJTbixz7BmPxA3ebRYJBhLE+CV928M3NXfs+fvPbr75o8JGnl+agBYEkACrlCBWOQjwHyBF9SgYBaNgFIwkAAAYhUi5qc7ggAAAAABJRU5ErkJggg==","orcid":"","institution":"International Islamic University Malaysia","correspondingAuthor":true,"prefix":"","firstName":"Abdulmalik","middleName":"","lastName":"Habaebi","suffix":""},{"id":509361321,"identity":"1f7d072b-c29e-4215-b43c-235fa3243018","order_by":1,"name":"Akeem Olowolayemo","email":"","orcid":"","institution":"International Islamic University Malaysia","correspondingAuthor":false,"prefix":"","firstName":"Akeem","middleName":"","lastName":"Olowolayemo","suffix":""},{"id":509361323,"identity":"49ea72b7-a0ad-46cf-bcc2-80ea5ad190fd","order_by":2,"name":"Sharyar Wani","email":"","orcid":"","institution":"International Islamic University Malaysia","correspondingAuthor":false,"prefix":"","firstName":"Sharyar","middleName":"","lastName":"Wani","suffix":""}],"badges":[],"createdAt":"2025-09-02 08:23:26","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7515191/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7515191/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90821844,"identity":"a7cb4608-b27a-4b74-866e-0a48aa9f5d70","added_by":"auto","created_at":"2025-09-08 14:24:04","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":430424,"visible":true,"origin":"","legend":"","description":"","filename":"DocBotManuscriptv0.1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7515191/v1_covered_920c2c43-f9ee-472a-87cd-4bf968ca1e4a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Large Language Models (LLMs), Medical Chatbot, Low-Rank Adaptation (LoRA), Fine-Tuning, LLaMa 3, Parameter-Efficient Fine-Tuning (PEFT)","lastPublishedDoi":"10.21203/rs.3.rs-7515191/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7515191/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003ePurpose\u003c/b\u003e\u003c/p\u003e\u003cp\u003eGeneral-purpose large language models (LLMs) often lack the specialized accuracy required for the medical domain. This research aims to address this gap by developing and evaluating DocBot, a medical LLM, to demonstrate that domain-specific fine-tuning can significantly enhance performance even with constrained computational resources.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe fine-tuned the Meta LLaMa 3.1 8B model using Low-Rank Adaptation (LoRA), a parameter-efficient technique. The model was trained on a curated dataset of 2,000 patient-doctor dialogues sourced from ClinicalTrials, EMEA, and PubMed, using a single Tesla T4 GPU. Performance was evaluated against the base LLaMa model using BERTScore, BLEU, and ROUGE metrics, with responses from verified medical professionals serving as the reference.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults\u003c/b\u003e\u003c/p\u003e\u003cp\u003eDocBot demonstrated significant improvements over the base LLaMa 3.1 8B model across all evaluation metrics. Specifically, DocBot achieved a higher BERTScore F1-score (83.56% vs. 81.47%), indicating enhanced semantic accuracy, fluency, and alignment with expert-generated text. The gains in precision and recall further confirm the model's superior ability to generate relevant and comprehensive medical information.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusion\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe successful development of DocBot showcases the feasibility and impact of creating domain-optimized LLMs efficiently. The results highlight the potential for specialized models to serve as reliable tools for augmenting clinical decision-making and delivering accessible medical support, particularly in resource-limited environments, paving the way for further innovation in specialized AI applications.\u003c/p\u003e","manuscriptTitle":"Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-04 09:16:23","doi":"10.21203/rs.3.rs-7515191/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d963a11e-2e8a-40e1-b945-3a44b52921ff","owner":[],"postedDate":"September 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-09-18T08:02:17+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-04 09:16:23","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7515191","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7515191","identity":"rs-7515191","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}