FedDPGu: Adaptive Prompt-tuning with Built-in Unlearning for Federated Learning

FedDPGu: Adaptive Prompt-tuning with Built-in Unlearning for Federated Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article FedDPGu: Adaptive Prompt-tuning with Built-in Unlearning for Federated Learning Lishan Yang, Wei Emma Zhang, Ali Shakeri, Amin Beheshti, Weitong Chen, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7488594/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 12 You are reading this latest preprint version Abstract Pre-trained Language Models (PLMs) have demonstrated impressive performance in various NLP tasks. However, traditional fine-tuning methods for leveraging PLMs for downstream tasks entail significant computational overhead. Prompt-tuning has emerged as an efficient alternative that involves prepending a limited number of parameters to the input sequence and only updating them while the PLM's parameters are frozen. However, this technique's prompts remain fixed for all inputs, reducing the model's flexibility. The Federated Learning (FL) technique has gained attention in recent years to address the growing concerns around data privacy. However, challenges such as communication and computation limitations of clients still need to be addressed. To mitigate these challenges, this paper introduces the \textbf{Fed}erated D ynamic P rompt G enerator (FedDPG), which incorporates a dynamic prompt generator network to generate context-aware prompts based on the given input, ensuring flexibility and adaptability while prioritising data privacy in federated learning settings. Our experiments on three NLP benchmark datasets showcase that FedDPG outperforms the state-of-the-art parameter-efficient fine-tuning methods in terms of global model performance compared with five models on three datasets, with only one configuration having a marginal lower performance, and significantly reducing the calculation time and the number of parameters to be sent through the FL network. Finally, we propose FedDPGu, a re-label-based method designed to handle local client unlearning. By further integrating an efficient federated unlearning method, we extend it to fast-FedDPGu, which leverages model difference estimation to enable efficient global unlearning of a target client. Together, these methods ensure that FedDPG can effectively forget sensitive client information at both the local and global levels in federated settings. Our code is available at https://github.com/gotobcn8/FedDPG. Federated Learning Prompt Tuning Federated Unlearning Text Classification Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 02 Feb, 2026 Reviews received at journal 28 Dec, 2025 Reviewers agreed at journal 13 Dec, 2025 Reviews received at journal 09 Dec, 2025 Reviewers agreed at journal 07 Dec, 2025 Reviewers agreed at journal 06 Dec, 2025 Reviewers agreed at journal 06 Dec, 2025 Reviewers agreed at journal 06 Dec, 2025 Reviewers invited by journal 16 Oct, 2025 Editor assigned by journal 30 Aug, 2025 Submission checks completed at journal 30 Aug, 2025 First submitted to journal 29 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7488594","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":535269424,"identity":"2da6735e-6f04-4a41-a060-e05a2ef97800","order_by":0,"name":"Lishan Yang","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Lishan","middleName":"","lastName":"Yang","suffix":""},{"id":535269425,"identity":"b6754e97-fc9a-4cde-b017-f003561f3e8f","order_by":1,"name":"Wei Emma Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABMklEQVRIie2Rv0vDQBTHXzhIl1eynljrv3Ai1KGi/0qOQrMEqRRCBikHQrIUXAVB/4XmP2g5aJdQN4kgmCxOHeKWwV+XOuiQ0I6C94E77uB9+N57B6DR/EkMUe52eUrBhBYQYOpCtlIIUwpuoXxTKiZdK7BBYYtLATmMzqyb8MEvvGO0QhKl4He5aMSsUolnwrgGc0ifYu9xvOwjleaQQexwgW6l0km4IAjIReL2k2YgESR2qBFILqBGec4EeQPK75Ry/h584r7Eo8L4UIq1qkkxhOqU8UnizEkzmCJTKWqMSqHVKacxF7Mxs3mUuGS3tezhgeqF2nPnMKAvgyplJ1xkaeGP+G3iZK8r76TdvpdRnl90966s3qR6zADT9UcA/nqGrZZZV/9DI91co9FoNP+SL4ebYn4s6UJ8AAAAAElFTkSuQmCC","orcid":"","institution":"University of Adelaide","correspondingAuthor":true,"prefix":"","firstName":"Wei","middleName":"Emma","lastName":"Zhang","suffix":""},{"id":535269426,"identity":"ff690370-6e05-4ad4-a4ae-db0533c6d530","order_by":2,"name":"Ali Shakeri","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Ali","middleName":"","lastName":"Shakeri","suffix":""},{"id":535269427,"identity":"5ea82c4a-27f8-4fbd-a751-dd6f8c9ec92f","order_by":3,"name":"Amin Beheshti","email":"","orcid":"","institution":"Macquarie University","correspondingAuthor":false,"prefix":"","firstName":"Amin","middleName":"","lastName":"Beheshti","suffix":""},{"id":535269428,"identity":"b0570d94-0285-4b45-88dd-e76825e2ceaa","order_by":4,"name":"Weitong Chen","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Weitong","middleName":"","lastName":"Chen","suffix":""},{"id":535269429,"identity":"435443fb-1690-430c-94cc-351822dc600e","order_by":5,"name":"Jian Yang","email":"","orcid":"","institution":"Macquarie University","correspondingAuthor":false,"prefix":"","firstName":"Jian","middleName":"","lastName":"Yang","suffix":""}],"badges":[],"createdAt":"2025-08-29 12:23:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7488594/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7488594/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":94797501,"identity":"1446393c-750b-456a-810d-d854ffbe6ee0","added_by":"auto","created_at":"2025-10-30 20:33:17","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8314,"visible":true,"origin":"","legend":"","description":"","filename":"c61a37781af74e3e8d3a41d4c2d94a0b.json","url":"https://assets-eu.researchsquare.com/files/rs-7488594/v1/3e7ede899ef970140b98156d.json"},{"id":94826159,"identity":"c4ed4c67-0ad4-465b-9e4b-18962b3865da","added_by":"auto","created_at":"2025-10-31 06:51:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":760223,"visible":true,"origin":"","legend":"","description":"","filename":"IJDSAFedDPGsubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7488594/v1_covered_d984b9b8-57f8-4ab8-81bf-faceec036352.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"FedDPGu: Adaptive Prompt-tuning with Built-in Unlearning for Federated Learning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-data-science-and-analytics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jdsa","sideBox":"Learn more about [International Journal of Data Science and Analytics](http://link.springer.com/journal/41060)","snPcode":"41060","submissionUrl":"https://submission.nature.com/new-submission/41060/3","title":"International Journal of Data Science and Analytics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Federated Learning, Prompt Tuning, Federated Unlearning, Text Classification","lastPublishedDoi":"10.21203/rs.3.rs-7488594/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7488594/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003ePre-trained Language Models (PLMs) have demonstrated impressive performance in various NLP tasks. However, traditional fine-tuning methods for leveraging PLMs for downstream tasks entail significant computational overhead. Prompt-tuning has emerged as an efficient alternative that involves prepending a limited number of parameters to the input sequence and only updating them while the PLM's parameters are frozen. However, this technique's prompts remain fixed for all inputs, reducing the model's flexibility. The Federated Learning (FL) technique has gained attention in recent years to address the growing concerns around data privacy. However, challenges such as communication and computation limitations of clients still need to be addressed. To mitigate these challenges, this paper introduces the \\textbf{Fed}erated \u003cstrong\u003eD\u003c/strong\u003eynamic \u003cstrong\u003eP\u003c/strong\u003erompt \u003cstrong\u003eG\u003c/strong\u003eenerator (FedDPG), which incorporates a dynamic prompt generator network to generate context-aware prompts based on the given input, ensuring flexibility and adaptability while prioritising data privacy in federated learning settings. Our experiments on three NLP benchmark datasets showcase that FedDPG outperforms the state-of-the-art parameter-efficient fine-tuning methods in terms of global model performance compared with five models on three datasets, with only one configuration having a marginal lower performance, and significantly reducing the calculation time and the number of parameters to be sent through the FL network. Finally, we propose FedDPGu, a re-label-based method designed to handle local client unlearning. By further integrating an efficient federated unlearning method, we extend it to fast-FedDPGu, which leverages model difference estimation to enable efficient global unlearning of a target client. Together, these methods ensure that FedDPG can effectively forget sensitive client information at both the local and global levels in federated settings. Our code is available at https://github.com/gotobcn8/FedDPG.\u003c/p\u003e","manuscriptTitle":"FedDPGu: Adaptive Prompt-tuning with Built-in Unlearning for Federated Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-30 20:33:13","doi":"10.21203/rs.3.rs-7488594/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-02-03T01:11:17+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-29T03:47:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"334631806226681411095838193762727434903","date":"2025-12-13T18:27:41+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-09T12:54:34+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"120961996555703131605711104160159307169","date":"2025-12-07T12:48:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"145154708367495012909962489003897494285","date":"2025-12-07T04:20:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"218109511830904103858973026177942992157","date":"2025-12-06T09:41:00+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"280919186271521128578027015205317590682","date":"2025-12-06T08:21:45+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-10-17T03:33:12+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-30T14:52:43+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-30T14:04:32+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Data Science and Analytics","date":"2025-08-29T12:16:12+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-data-science-and-analytics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jdsa","sideBox":"Learn more about [International Journal of Data Science and Analytics](http://link.springer.com/journal/41060)","snPcode":"41060","submissionUrl":"https://submission.nature.com/new-submission/41060/3","title":"International Journal of Data Science and Analytics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"4981d5a0-1259-4903-9e63-bfb55d49164f","owner":[],"postedDate":"October 30th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-22T04:53:17+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-30 20:33:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7488594","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7488594","identity":"rs-7488594","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}