Self-Paced Online Multi-Task Learning via Alternating Direction Method of Multipliers

Self-Paced Online Multi-Task Learning via Alternating Direction Method of Multipliers | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Self-Paced Online Multi-Task Learning via Alternating Direction Method of Multipliers xuewei li, bing ma, haoran zheng, qiang zhao, qian xing, yonglan qi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6188766/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Online multi-task learning (OMTL) enhances streaming data processing by harnessing relationships across tasks, typically formulated as an optimization problem with a shared loss function. However, conventional gradient-based methods often grapple with gradient vanishing and conditioning issues. Moreover, their centralized nature hampers online parallel optimization, crucial for big data. Drawing from the cognitive principle of learning from simple to complex, this study introduces a Self-Paced Online Multi-Task Learning (SPOMTL) framework using the Alternating Direction Method of Multipliers (ADMM). Task relationships are dynamically modeled to adapt to online changes, while the self-paced mechanism prioritizes easier tasks and instances, gradually introducing harder ones. In a distributed architecture with a central server, this SPOMTL-ADMM outperforms SGD methods in accuracy and efficiency. To mitigate server bottlenecks with large data, we also developed a decentralized version, enabling nodes to operate via local neighbor interactions. Experiments on synthetic and real-world datasets highlight the efficiency of our approach. Physical sciences/Mathematics and computing/Information technology Physical sciences/Mathematics and computing/Computational science Physical sciences/Mathematics and computing/Computer science Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6188766","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":432803241,"identity":"f6398ec6-daca-462e-8a6f-4dac6f52d94b","order_by":0,"name":"xuewei li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA3ElEQVRIiWNgGAWjYBACPmYIzcMvwcBwAMhgbCCkhQ2mRXIGUMsBorTAGAY3GMDWEKGFncdM4mObjYzx7eaHhz8w2MhuOMD87AF+h7GlSc5sS+Mxu3PMAOiwNOMNB9jMDfBrYT4mzbvtMI/ZjQSQlsOJGw7wsEng18LYJv13238e4xnpH4Ba/hOjBWgL47YDPAYSOSBbDhCjhS3ZsvdfMo/EjZyCA2cMko1nHmYzw6uFn/+M4Y0fZ+zs+Wekb/5QUWEn23e8+RleLWgAFFTMJKgfBaNgFIyCUYAdAADB8EZBCf1EqgAAAABJRU5ErkJggg==","orcid":"","institution":"Henan Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"xuewei","middleName":"","lastName":"li","suffix":""},{"id":432803242,"identity":"39c65f75-8a13-434e-8d06-5a63cfe45d14","order_by":1,"name":"bing ma","email":"","orcid":"","institution":"Shanxi Institute of Mechanical and Electrical Engineering","correspondingAuthor":false,"prefix":"","firstName":"bing","middleName":"","lastName":"ma","suffix":""},{"id":432803243,"identity":"a23cebe7-84f9-4662-a262-7511bbd51fb9","order_by":2,"name":"haoran zheng","email":"","orcid":"","institution":"University of Sheffield","correspondingAuthor":false,"prefix":"","firstName":"haoran","middleName":"","lastName":"zheng","suffix":""},{"id":432803246,"identity":"c4909ebf-f243-4eea-a3bc-78634d4b7e8e","order_by":3,"name":"qiang zhao","email":"","orcid":"","institution":"Henan Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"qiang","middleName":"","lastName":"zhao","suffix":""},{"id":432803248,"identity":"58fa528a-ea5a-410d-b43c-c5ad6eb7185e","order_by":4,"name":"qian xing","email":"","orcid":"","institution":"Henan Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"qian","middleName":"","lastName":"xing","suffix":""},{"id":432803250,"identity":"e2fc3967-4bec-479f-816f-632c4a1861ef","order_by":5,"name":"yonglan qi","email":"","orcid":"","institution":"Henan Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"yonglan","middleName":"","lastName":"qi","suffix":""}],"badges":[],"createdAt":"2025-03-09 13:08:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6188766/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6188766/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":81724701,"identity":"5c4b4907-c6e3-4e50-9569-654c16435abf","added_by":"auto","created_at":"2025-04-30 17:01:37","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":532108,"visible":true,"origin":"","legend":"","description":"","filename":"main.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6188766/v1_covered_37798e51-43a0-4edd-8412-d6010c20d2f6.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Self-Paced Online Multi-Task Learning via Alternating Direction Method of Multipliers","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6188766/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6188766/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Online multi-task learning (OMTL) enhances streaming data processing by harnessing relationships across tasks, typically formulated as an optimization problem with a shared loss function. However, conventional gradient-based methods often grapple with gradient vanishing and conditioning issues. Moreover, their centralized nature hampers online parallel optimization, crucial for big data. Drawing from the cognitive principle of learning from simple to complex, this study introduces a Self-Paced Online Multi-Task Learning (SPOMTL) framework using the Alternating Direction Method of Multipliers (ADMM). Task relationships are dynamically modeled to adapt to online changes, while the self-paced mechanism prioritizes easier tasks and instances, gradually introducing harder ones. In a distributed architecture with a central server, this SPOMTL-ADMM outperforms SGD methods in accuracy and efficiency. To mitigate server bottlenecks with large data, we also developed a decentralized version, enabling nodes to operate via local neighbor interactions. Experiments on synthetic and real-world datasets highlight the efficiency of our approach.","manuscriptTitle":"Self-Paced Online Multi-Task Learning via Alternating Direction Method of Multipliers","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-27 04:28:40","doi":"10.21203/rs.3.rs-6188766/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a88cb0d7-be23-48cc-9386-72a54104f4b1","owner":[],"postedDate":"March 27th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":46088725,"name":"Physical sciences/Mathematics and computing/Information technology"},{"id":46088726,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":46088727,"name":"Physical sciences/Mathematics and computing/Computer science"}],"tags":[],"updatedAt":"2025-04-30T16:53:30+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-27 04:28:40","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6188766","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6188766","identity":"rs-6188766","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}