Merging LoRA Adapters for Multi-Task Code Analysis: An Empirical Study of Linear Combination and Task Interference

Merging LoRA Adapters for Multi-Task Code Analysis: An Empirical Study of Linear Combination and Task Interference | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Merging LoRA Adapters for Multi-Task Code Analysis: An Empirical Study of Linear Combination and Task Interference Sankalp Pathak, Sanjay Garg This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9189872/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Deploying multiple code analysis capabilities, including static code analysis (SCA) and vulnerability detection (VD), typically requires maintaining separate models or running independent inference passes. We investigate whether task-specific LoRA adapters, each fine-tuned independently on Meta-Llama-3.1-8B-Instruct, can be merged via weighted linear combination into a single adapter that preserves performance on both tasks. We evaluate 19 configurations: a \((4 \times 4)\) lambda grid ( \((\lsca, \lvd \in \{0.3, 0.5, 0.7, 1.0\})\) ) plus three baselines, on synthetic SCA data (3{,}463 samples, 11 categories) and PrimeVul vulnerability data (9{,}858 expert-verified C/C++ samples). Our SCA adapter achieves F1=0.994 and our VD adapter achieves F1=0.732 (MCC=0.466) as solo adapters. The best merged configuration retains 98% of solo VD performance (F1=0.717) while gaining SCA capability, and 91% of solo SCA performance (F1=0.907) while gaining VD capability. We find that interference is asymmetric : VD is more sensitive to SCA adapter weight than vice versa. Equal high lambdas ( \((\lsca = \lvd = 1.0)\) ) cause catastrophic degradation on both tasks. Three Pareto-optimal configurations span the trade-off space for practical deployment. Our results also document that VD dataset quality, not model capacity, is the primary bottleneck: switching from BigVul (F1=0.483) to PrimeVul (F1=0.732) on the same model produced the largest improvement. LoRA adapter merging code analysis vulnerability detection multi-task learning large language models Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 07 May, 2026 Reviews received at journal 08 Apr, 2026 Reviewers agreed at journal 08 Apr, 2026 Reviewers invited by journal 08 Apr, 2026 Editor assigned by journal 23 Mar, 2026 Submission checks completed at journal 23 Mar, 2026 First submitted to journal 22 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9189872","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":619992520,"identity":"79324b0d-5867-4eb4-bcef-1fd62e269331","order_by":0,"name":"Sankalp Pathak","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFklEQVRIiWNgGAWjYHACNgaGA0CKh4HxAIMBmxxI6MADvDqY4VqAVAWfMVhLAvFazsglNoAE8WkxZz9/7MGPM3Xy5jyHHxwubDNLnx92+CHQFjs53QbsWix7ktkNe24cNtzZ22ZweGZbWu7G22kGQC3JxmYHsGsxOJDMJsHz4QDjhvMMBod5247lbpydANJyIHEbLi3nH7NJ/vlQZ7/hPPsHoJb/6Yaz0z/g13IjmU2a5wZz4oazPQaHec6wJchL5+C3xXLGYzNpmTOHkzecOVNweEYFm+EG6ZyCAwkGuP1izp/4TPLNsTrbDWfSNz4uMGCTl5+dvvnDhwo7OZzeR+YwQwIEQ5yQFvkG3KpHwSgYBaNgZAIA40NrlC7lY8IAAAAASUVORK5CYII=","orcid":"","institution":"Jaypee University of Engineering and Technology","correspondingAuthor":true,"prefix":"","firstName":"Sankalp","middleName":"","lastName":"Pathak","suffix":""},{"id":619992521,"identity":"d03881df-ec16-4f30-91bd-80ed6d3b0898","order_by":1,"name":"Sanjay Garg","email":"","orcid":"","institution":"Jaypee University of Engineering and Technology","correspondingAuthor":false,"prefix":"","firstName":"Sanjay","middleName":"","lastName":"Garg","suffix":""}],"badges":[],"createdAt":"2026-03-22 08:09:53","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9189872/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9189872/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107065120,"identity":"0e52734a-1e0e-4c50-bc5b-b6ed200e66ec","added_by":"auto","created_at":"2026-04-16 10:57:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1234911,"visible":true,"origin":"","legend":"","description":"","filename":"MergingLoRAAdaptersforMultiTaskCodeAnalysisAnEmpiricalStudyofLinearCombinationandTaskInterference.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9189872/v1_covered_14af86eb-6cac-4820-bef7-02c08e14a404.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Merging LoRA Adapters for Multi-Task Code Analysis: An Empirical Study of Linear Combination and Task Interference","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"LoRA, adapter merging, code analysis, vulnerability detection, multi-task learning, large language models","lastPublishedDoi":"10.21203/rs.3.rs-9189872/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9189872/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDeploying multiple code analysis capabilities, including static code analysis (SCA) and vulnerability detection (VD), typically requires maintaining separate models or running independent inference passes. We investigate whether task-specific LoRA adapters, each fine-tuned independently on Meta-Llama-3.1-8B-Instruct, can be merged via weighted linear combination into a single adapter that preserves performance on both tasks. We evaluate 19 configurations: a \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((4 \\times 4)\\)\u003c/span\u003e\u003c/span\u003e lambda grid (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((\\lsca, \\lvd \\in \\{0.3, 0.5, 0.7, 1.0\\})\\)\u003c/span\u003e\u003c/span\u003e) plus three baselines, on synthetic SCA data (3{,}463 samples, 11 categories) and PrimeVul vulnerability data (9{,}858 expert-verified C/C++ samples). Our SCA adapter achieves F1=0.994 and our VD adapter achieves F1=0.732 (MCC=0.466) as solo adapters. The best merged configuration retains 98% of solo VD performance (F1=0.717) while gaining SCA capability, and 91% of solo SCA performance (F1=0.907) while gaining VD capability. We find that interference is \u003cem\u003easymmetric\u003c/em\u003e: VD is more sensitive to SCA adapter weight than vice versa. Equal high lambdas (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((\\lsca = \\lvd = 1.0)\\)\u003c/span\u003e\u003c/span\u003e) cause catastrophic degradation on both tasks. Three Pareto-optimal configurations span the trade-off space for practical deployment. Our results also document that VD dataset quality, not model capacity, is the primary bottleneck: switching from BigVul (F1=0.483) to PrimeVul (F1=0.732) on the same model produced the largest improvement.\u003c/p\u003e","manuscriptTitle":"Merging LoRA Adapters for Multi-Task Code Analysis: An Empirical Study of Linear Combination and Task Interference","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-16 10:55:32","doi":"10.21203/rs.3.rs-9189872/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"219572478653230657804166580304196250035","date":"2026-05-07T14:31:08+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-09T03:20:28+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"120650398728439665726468491769252435010","date":"2026-04-09T02:35:42+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-09T02:29:51+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-23T17:47:16+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-23T17:46:56+00:00","index":"","fulltext":""},{"type":"submitted","content":"Automated Software Engineering","date":"2026-03-22T08:05:04+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"automated-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ause","sideBox":"Learn more about [Automated Software Engineering](http://link.springer.com/journal/10515)","snPcode":"10515","submissionUrl":"https://submission.nature.com/new-submission/10515/3","title":"Automated Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"43b870fb-5532-4524-a0d2-2e6fdea23a2d","owner":[],"postedDate":"April 16th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewerAgreed","content":"219572478653230657804166580304196250035","date":"2026-05-07T14:31:08+00:00","index":13,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-16T10:55:32+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-16 10:55:32","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9189872","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9189872","identity":"rs-9189872","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}