Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces Rodrigo Maulen-Soto, Jalal Fadili, Hedy Attouch This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4076301/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract To solve non-smooth convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz's continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex and strongly convex case. AMS subject classifications. 37N40, 46N10, 49M99, 65B99, 65K05, 65K10, 90B50, 90C25, 60H10, 49J52. Stochastic optimization inertial gradient system Convex optimization Non-smooth optimization Stochastic Differential Equation Stochastic Differential Inclusion Tikhonov regularization Error bound inequality Łojasiewicz inequality KL inequality Convergence rate Asymptotic behavior Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4076301","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":279106129,"identity":"4c172328-4d4b-4d25-b633-916ffdc127e6","order_by":0,"name":"Rodrigo Maulen-Soto","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFUlEQVRIiWNgGAWjYHACA4YECIPxAAODDYhuAHPZGwhrYQBqSUNo4TmARwsMANUcRojj0mLe3rztw4NfNgy67WcMDnzccV7OnP9w44cfFXcYeKSx65E5c6x4RmJfGoPZmRyDgzPP3Da2nJHYLNlz5hkDD18CVi0SEjnGDIk9hxnMDuQYHOZtu5244QZjGwNv22EGex7sDkNoOf/G4PDftnP1G84fbGP8++8wAw8+LQk/gFpuAG1hbDuQYHAgsY2ZtwGPFp5jxQyJDWk8ZjeeFRzsbUs23HAjsVla5thhHpxa2Js3M/74YyNndj5544OfbXbyBuePP/z4puawHC4tYAD0L6Y0Pg1A8Ae/9CgYBaNgFIxwAADMtWNq8HnWCAAAAABJRU5ErkJggg==","orcid":"","institution":"École Nationale Supérieure d'Ingénieurs de Caen","correspondingAuthor":true,"prefix":"","firstName":"Rodrigo","middleName":"","lastName":"Maulen-Soto","suffix":""},{"id":279106130,"identity":"699e7cb6-1547-4a3c-ba7e-e6c29ce82cd5","order_by":1,"name":"Jalal Fadili","email":"","orcid":"","institution":"École Nationale Supérieure d'Ingénieurs de Caen","correspondingAuthor":false,"prefix":"","firstName":"Jalal","middleName":"","lastName":"Fadili","suffix":""},{"id":279106131,"identity":"92aef494-d59e-4d50-9ee8-0a960ceb0ffd","order_by":2,"name":"Hedy Attouch","email":"","orcid":"","institution":"University of Montpellier","correspondingAuthor":false,"prefix":"","firstName":"Hedy","middleName":"","lastName":"Attouch","suffix":""}],"badges":[],"createdAt":"2024-03-11 16:01:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4076301/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4076301/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":57457039,"identity":"4bb9e4ee-353d-4e44-9570-503b207d6240","added_by":"auto","created_at":"2024-05-31 01:08:59","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":451269,"visible":true,"origin":"","legend":"","description":"","filename":"SDITikhonov20.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4076301/v1_covered_c5b82523-29e5-4cbd-8a96-9015b098e4d2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Stochastic optimization, inertial gradient system, Convex optimization, Non-smooth optimization, Stochastic Differential Equation, Stochastic Differential Inclusion, Tikhonov regularization, Error bound inequality, Łojasiewicz inequality, KL inequality, Convergence rate, Asymptotic behavior","lastPublishedDoi":"10.21203/rs.3.rs-4076301/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4076301/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTo solve non-smooth convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz's continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex and strongly convex case.\u003c/p\u003e\n\u003cp\u003eAMS subject classifications. 37N40, 46N10, 49M99, 65B99, 65K05, 65K10, 90B50, 90C25, 60H10, 49J52.\u003c/p\u003e","manuscriptTitle":"Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-14 18:17:07","doi":"10.21203/rs.3.rs-4076301/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"aa4e07d2-f2d1-4044-bc90-e301e979461d","owner":[],"postedDate":"March 14th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-05-31T01:08:41+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-14 18:17:07","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4076301","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4076301","identity":"rs-4076301","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}