Adapting LLMs for Biomedical Natural Language Processing: A Comprehensive Benchmark Study on Fine-Tuning Methods

Adapting LLMs for Biomedical Natural Language Processing: A Comprehensive Benchmark Study on Fine-Tuning Methods | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Adapting LLMs for Biomedical Natural Language Processing: A Comprehensive Benchmark Study on Fine-Tuning Methods Jin Li, Junjie Zhu, Shen Zhao, Yiyan Deng, Yongming Miao, Jun Xu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7369550/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 26 You are reading this latest preprint version Abstract The application of large language models (LLMs) in biomedical natural language processing (NLP) shows promise, yet the complexity and specificity of biomedical texts often challenge general-purpose models. Fine-tuning, which customizes LLMs for domain-specific tasks, is essential to address these challenges and optimize performance. This study systematically evaluates various fine-tuning methods for adapting LLMs to biomedical NLP tasks, focusing on full fine-tuning (FFT) and parameter-efficient fine-tuning (PEFT) techniques such as LoRA, QLoRA, and P-tuning. Weassess these methods across 12 benchmark datasets from the Biomedical Literature Understanding with RelationBased Benchmarks (BLURB) corpus, using state-of-the-art LLMs of varying sizes, including LLaMA-3, FLAN-T5, Chat-GLM4, and UL2. Their performance is compared to domain-specific models like PubMedBERT and BioClinicalBERT. The results indicate that fine-tuning significantly enhances performance compared to zero-shot settings, with LoRA and QLoRA emerging as the most effective and computationally efficient approaches. FFT generally does not exhibit a distinct advantage over PEFT methods. Notably, fine-tuned LLMs outperformed domain-specific BERT-based models in most cases, underscoring the potential of LLMs for complex biomedical tasks when tailored appropriately. In contrast, few-shot experiments revealed that fine-tuning provides a more stable and effective optimization strategy than in-context learning methods for biomedical applications. This study offers a comprehensive analysis of fine-tuning techniques, shedding light on how LLMs can be effectively adapted for advanced biomedical NLP tasks, thereby contributing to more efficient and versatile models for biomedical research and clinical practice. The code for this study is publicly available on Github Large language models Natural language processing Parameter-Efficient Fine-tuning Comparative study Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 22 Nov, 2025 Reviews received at journal 20 Nov, 2025 Reviews received at journal 19 Nov, 2025 Reviews received at journal 17 Nov, 2025 Reviews received at journal 15 Nov, 2025 Reviewers agreed at journal 12 Nov, 2025 Reviewers agreed at journal 12 Nov, 2025 Reviewers agreed at journal 11 Nov, 2025 Reviewers agreed at journal 09 Nov, 2025 Reviewers agreed at journal 09 Nov, 2025 Reviewers agreed at journal 08 Nov, 2025 Reviews received at journal 07 Nov, 2025 Reviewers agreed at journal 07 Nov, 2025 Reviewers agreed at journal 06 Nov, 2025 Reviewers agreed at journal 06 Nov, 2025 Reviewers agreed at journal 05 Nov, 2025 Reviewers agreed at journal 05 Nov, 2025 Reviewers agreed at journal 04 Nov, 2025 Reviewers agreed at journal 04 Nov, 2025 Reviewers agreed at journal 04 Nov, 2025 Reviewers agreed at journal 03 Nov, 2025 Reviewers agreed at journal 03 Nov, 2025 Reviewers invited by journal 03 Nov, 2025 Editor assigned by journal 15 Aug, 2025 Submission checks completed at journal 15 Aug, 2025 First submitted to journal 13 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7369550","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":544158482,"identity":"607be8bf-9022-47ad-a26d-24327c4f667a","order_by":0,"name":"Jin Li","email":"","orcid":"","institution":"Nanjing University of Information Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jin","middleName":"","lastName":"Li","suffix":""},{"id":544158483,"identity":"9ad7fdfe-bb4c-4962-85fd-44125db285a7","order_by":1,"name":"Junjie Zhu","email":"","orcid":"","institution":"Nanjing University of Information Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Junjie","middleName":"","lastName":"Zhu","suffix":""},{"id":544158484,"identity":"fb1f0529-cb58-468a-a5e7-62d850279825","order_by":2,"name":"Shen Zhao","email":"","orcid":"","institution":"Fudan University Shanghai Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Shen","middleName":"","lastName":"Zhao","suffix":""},{"id":544158485,"identity":"a1fbc099-cd12-4af8-9ad1-b5ac56e9a9be","order_by":3,"name":"Yiyan Deng","email":"","orcid":"","institution":"Nanjing University of Information Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Yiyan","middleName":"","lastName":"Deng","suffix":""},{"id":544158486,"identity":"93068da7-9dda-4ae2-bf70-fb50a6bc4e4c","order_by":4,"name":"Yongming Miao","email":"","orcid":"","institution":"Nanjing University of Information Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Yongming","middleName":"","lastName":"Miao","suffix":""},{"id":544158487,"identity":"d74609fb-217b-4b95-bc9b-d8b3dc07eed3","order_by":5,"name":"Jun Xu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvUlEQVRIiWNgGAWjYBACxgYGxgcMDBJgjgSxWpgNQIp5iNYCBGxglcRrYZ6RY1Zd2GZhb8/AfPA2D4NdHmGH9Zwxuz2zTSKxh4Et2ZqHIbmYsJb2HrPbvG0SCTwMPGbSPAwHEhsIamnmMSsGarHnYeD/RqQWoC3MQC2MPQw8bERq6TlWLM1zDuiXw2zGlnMMkglrMZyRvPEzT1mdPXt788MbbyrsiNACV8EMIgwIqQcCeSLUjIJRMApGwUgHAMxDLpqhZPd0AAAAAElFTkSuQmCC","orcid":"","institution":"Nanjing University of Information Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Jun","middleName":"","lastName":"Xu","suffix":""}],"badges":[],"createdAt":"2025-08-14 04:08:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7369550/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7369550/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":95810895,"identity":"ec189156-57bc-4383-8c42-f0641b35b341","added_by":"auto","created_at":"2025-11-13 08:54:06","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":374604,"visible":true,"origin":"","legend":"","description":"","filename":"TheJournalofSupercomputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7369550/v1/aaaa9ec184f6236bef3dc9b4.pdf"},{"id":95810896,"identity":"d525fa0b-24de-4383-b972-f349d83da913","added_by":"auto","created_at":"2025-11-13 08:54:06","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":7590,"visible":true,"origin":"","legend":"","description":"","filename":"4ca31341bb624d2db76170d7e27931fa.json","url":"https://assets-eu.researchsquare.com/files/rs-7369550/v1/d567009fd9a4c217a013fc94.json"},{"id":95811621,"identity":"ba327ada-1d87-4404-be37-4c3c0d50ede1","added_by":"auto","created_at":"2025-11-13 08:55:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":438406,"visible":true,"origin":"","legend":"","description":"","filename":"TheJournalofSupercomputing.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7369550/v1_covered_76498836-44e6-4347-9d4d-ca6a1a22c3a4.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Adapting LLMs for Biomedical Natural Language Processing: A Comprehensive Benchmark Study on Fine-Tuning Methods","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Large language models, Natural language processing, Parameter-Efficient Fine-tuning, Comparative study","lastPublishedDoi":"10.21203/rs.3.rs-7369550/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7369550/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":" The application of large language models (LLMs) in biomedical natural language processing (NLP) shows promise, yet the complexity and specificity of biomedical texts often challenge general-purpose models. Fine-tuning, which customizes LLMs for domain-specific tasks, is essential to address these challenges and optimize performance. This study systematically evaluates various fine-tuning methods for adapting LLMs to biomedical NLP tasks, focusing on full fine-tuning (FFT) and parameter-efficient fine-tuning (PEFT) techniques such as LoRA, QLoRA, and P-tuning. Weassess these methods across 12 benchmark datasets from the Biomedical Literature Understanding with RelationBased Benchmarks (BLURB) corpus, using state-of-the-art LLMs of varying sizes, including LLaMA-3, FLAN-T5, Chat-GLM4, and UL2. Their performance is compared to domain-specific models like PubMedBERT and BioClinicalBERT. The results indicate that fine-tuning significantly enhances performance compared to zero-shot settings, with LoRA and QLoRA emerging as the most effective and computationally efficient approaches. FFT generally does not exhibit a distinct advantage over PEFT methods. Notably, fine-tuned LLMs outperformed domain-specific BERT-based models in most cases, underscoring the potential of LLMs for complex biomedical tasks when tailored appropriately. In contrast, few-shot experiments revealed that fine-tuning provides a more stable and effective optimization strategy than in-context learning methods for biomedical applications. This study offers a comprehensive analysis of fine-tuning techniques, shedding light on how LLMs can be effectively adapted for advanced biomedical NLP tasks, thereby contributing to more efficient and versatile models for biomedical research and clinical practice. The code for this study is publicly available on Github","manuscriptTitle":"Adapting LLMs for Biomedical Natural Language Processing: A Comprehensive Benchmark Study on Fine-Tuning Methods","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-13 08:46:38","doi":"10.21203/rs.3.rs-7369550/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-11-22T05:09:23+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-20T10:08:07+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-19T13:07:25+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-17T07:46:36+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-15T12:15:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"212168657898601253067053080029138251750","date":"2025-11-12T21:11:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"132887825395570597033141777840237820630","date":"2025-11-12T16:30:06+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"111237066359886725652375527750835530859","date":"2025-11-12T04:32:27+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"46175384135512413576981895360215643982","date":"2025-11-09T11:27:23+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"66111100663751889236843854526140080984","date":"2025-11-09T06:47:16+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"247361324336368884379056865990262294988","date":"2025-11-09T04:34:44+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-07T20:38:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"41015737857242009115772478249669178381","date":"2025-11-07T20:21:29+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"132234874870776269328264958144308122841","date":"2025-11-06T16:57:31+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"44916183634102054627487111954932113765","date":"2025-11-06T05:32:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"313233935587951152724040641119043060497","date":"2025-11-05T08:12:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"340025036407735968168217717332303622246","date":"2025-11-05T05:49:42+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"272673814409892642654269138774590492023","date":"2025-11-04T10:42:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"213417350888776140927924803182932706275","date":"2025-11-04T06:54:43+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"118073128282587472127092083523863941579","date":"2025-11-04T05:45:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"103072981045447307276041790858129099657","date":"2025-11-04T04:10:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"98400302373537117270645058586554427082","date":"2025-11-04T04:06:30+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-11-04T03:56:50+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-15T05:26:37+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-15T05:24:32+00:00","index":"","fulltext":""},{"type":"submitted","content":"The Journal of Supercomputing","date":"2025-08-14T03:58:45+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"the-journal-of-supercomputing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [The Journal of Supercomputing](https://www.springer.com/journal/11227)","snPcode":"11227","submissionUrl":"https://submission.nature.com/new-submission/11227/3","title":"The Journal of Supercomputing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"32560abd-4917-48ce-9952-9974afcc35e9","owner":[],"postedDate":"November 13th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-12-17T17:23:01+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-13 08:46:38","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7369550","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7369550","identity":"rs-7369550","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}