Restrictively Self-sampled and Compressed Local Differential Privacy in Federated Learning

Restrictively Self-sampled and Compressed Local Differential Privacy in Federated Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Restrictively Self-sampled and Compressed Local Differential Privacy in Federated Learning Jianzhe Zhao, Yuchen Li, Ronglin Zhang, Wuganjing Song, Rongrong Dong, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1891162/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract As a popular machine learning framework, federated learning (FL) enables clients to conduct cooperative training without sharing data, thus having higher security than conventional machine learning. However, by sharing parameters in the federated learning process, the attacker can still obtain private information from the sensitive data of participants by reverse parsing. Recently, local differential privacy (LDP) has worked well in preserving privacy for federated learning. However, it faces the inherent problem of balancing privacy, model performance, and algorithm efficiency. In this paper, we propose a novel local differential privacy method in federated learning (SLDP-FL), which achieves the privacy amplification effect by the client self-sampling and provides compressed and private parameters in each iteration by a compressed LDP mechanism. Thereby, it improves the model performance as well as efficiency observably.Moreover, we theoretically analyze the relationship between the model accuracy and client self-sampling probability. A restrictive client self-sampling technology is proposed, which eliminates the randomness of self-sampling probability settings in existing studies and improves the utilization of the federated system. Comprehensive experiments on MNIST and Fashion-MNIST datasets show that the SLDP-FL optimizes the existing federated learning framework through compression mechanism and self-sampling technique with restrictive probability since it is superior to the current algorithms' accuracy and convergence and communication efficiency. Federated learning Compressed LDP Client self-sampling Local differential privacy Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1891162","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":124284756,"identity":"f336d008-5f30-472f-85b9-41e255b8ba20","order_by":0,"name":"Jianzhe Zhao","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jianzhe","middleName":"","lastName":"Zhao","suffix":""},{"id":124284757,"identity":"2056ea79-d3f7-461a-ae6f-542ac7a60f16","order_by":1,"name":"Yuchen Li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAklEQVRIie3PMUsDMRTA8YRAbgl2faVYv0IkcHQQ/SoXDm5yEARxKJgjEMdb72MogjheCcTloGsGB0vBycFuTra9QbBDjtLJIf/pEfjxXhCKxf5hA4LL5fctHQ/+PPJeMrzXWqD2SAwV2ZPw9tWMsBnLh2ZfgrzU4oqm+Gk+d4BfzqRK9DOg6VtQ4FqWy5oVJPU5BdwWUjF3A8h9BAkBqTkDR1NPtsRYqeAyBaxskFCQBhhfM1HZjqylOvnsJ4zNzIhlFDjqDjPNdgvrJ5CU+rRuKAefi4k0uTCsuJ5kLkwubLJ4X/3Qu6qaLfzKnB9XiX30X9Mw2S3rfvc7xGKxWOzwNtthUco91iiWAAAAAElFTkSuQmCC","orcid":"","institution":"Northeastern University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Yuchen","middleName":"","lastName":"Li","suffix":""},{"id":124284758,"identity":"8666e298-707c-4c47-b5c1-4b5333cf0b52","order_by":2,"name":"Ronglin Zhang","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Ronglin","middleName":"","lastName":"Zhang","suffix":""},{"id":124284760,"identity":"07bd75b6-d791-48af-9e1a-d804d1da28a1","order_by":3,"name":"Wuganjing Song","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Wuganjing","middleName":"","lastName":"Song","suffix":""},{"id":124284761,"identity":"51263fe1-afb0-4460-970f-1460d6663954","order_by":4,"name":"Rongrong Dong","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Rongrong","middleName":"","lastName":"Dong","suffix":""},{"id":124284763,"identity":"f822b196-2b2f-44ca-967c-88d92309bc37","order_by":5,"name":"Stan Matwin","email":"","orcid":"","institution":"Dalhousie University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Stan","middleName":"","lastName":"Matwin","suffix":""}],"badges":[],"createdAt":"2022-07-24 16:29:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1891162/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1891162/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":24597552,"identity":"2afb0c27-2397-4b0d-8fb7-1ee96316b90c","added_by":"auto","created_at":"2022-08-01 14:37:20","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4520502,"visible":true,"origin":"","legend":"","description":"","filename":"RestrictivelySelfsampledandCompressedLocalDifferentialPrivacyinFederatedLearning.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1891162/v1_covered.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Restrictively Self-sampled and Compressed Local Differential Privacy in Federated Learning","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1891162/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Federated learning, Compressed LDP, Client self-sampling, Local differential privacy","lastPublishedDoi":"10.21203/rs.3.rs-1891162/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1891162/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"As a popular machine learning framework, federated learning (FL) enables clients to conduct cooperative training without sharing data, thus having higher security than conventional machine learning. However, by sharing parameters in the federated learning process, the attacker can still obtain private information from the sensitive data of participants by reverse parsing. Recently, local differential privacy (LDP) has worked well in preserving privacy for federated learning. However, it faces the inherent problem of balancing privacy, model performance, and algorithm efficiency. In this paper, we propose a novel local differential privacy method in federated learning (SLDP-FL), which achieves the privacy amplification effect by the client self-sampling and provides compressed and private parameters in each iteration by a compressed LDP mechanism. Thereby, it improves the model performance as well as efficiency observably.Moreover, we theoretically analyze the relationship between the model accuracy and client self-sampling probability. A restrictive client self-sampling technology is proposed, which eliminates the randomness of self-sampling probability settings in existing studies and improves the utilization of the federated system. Comprehensive experiments on MNIST and Fashion-MNIST datasets show that the SLDP-FL optimizes the existing federated learning framework through compression mechanism and self-sampling technique with restrictive probability since it is superior to the current algorithms' accuracy and convergence and communication efficiency.","manuscriptTitle":"Restrictively Self-sampled and Compressed Local Differential Privacy in Federated Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-08-01 14:37:13","doi":"10.21203/rs.3.rs-1891162/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6db1e016-4f29-4398-93fe-b441937d0f33","owner":[],"postedDate":"August 1st, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2022-08-19T06:44:18+00:00","versionOfRecord":[],"versionCreatedAt":"2022-08-01 14:37:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1891162","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1891162","identity":"rs-1891162","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}