Leveraging Single Tasks for Better Generalization of Multitask Gaussian Process on Multivariate Time Series

Leveraging Single Tasks for Better Generalization of Multitask Gaussian Process on Multivariate Time Series | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Leveraging Single Tasks for Better Generalization of Multitask Gaussian Process on Multivariate Time Series Zhongkui Sun, Kai Chen This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4839107/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract By leveraging the knowledge of separate single tasks, we propose a simple and principled algorithm for multitask Gaussian process (GP), known as stochastic hyperparameter averaging (SHA), to obtain better generalization. Specifically, we focus on multivariate time series learning to improve the generalization of extrapolation and interpolation. The knowledge of a single task is extracted by a GP separately trained on one task-specific dimension of a multivariate time series. The single task GP (STGP) has the same kernel with the latent functions in multitask GP. By averaging hyperparameters of separate STGPs to initialize the latent functions of multitask GP,SHA identifies solutions that are significantly better than those found by popular training methods, but with only a few training steps of STGPs. SHA is kernel agnostic, remarkably straightforward to implement, and enhances generalization performance. Our SHA attains a significant boost in test accuracy across various diverse multivariate time series tasks, including interpolation, extrapolation, robustness with varying model complexities, and insensitivity to different hyperparameter initializations. multitask Gaussian process linear model of coregionalization stochastic hyperparameter averaging single task multivariate time series Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4839107","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":336371779,"identity":"8c8c4162-932f-4fd9-9853-27151d5ca282","order_by":0,"name":"Zhongkui Sun","email":"","orcid":"","institution":"Central South University","correspondingAuthor":false,"prefix":"","firstName":"Zhongkui","middleName":"","lastName":"Sun","suffix":""},{"id":336371780,"identity":"9a09b77c-b493-4806-bed0-862739636504","order_by":1,"name":"Kai Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIie3NsQqDMBCA4YigS4rridRnEAJS8GXM5CJS6OLgkFLQsauPUegLCAFdUro6dHFxLrhLLU4dGtKtQ34IR+A+DiGd7g8DZDBzmb6z/k11QlymTtY1GjSqxK34cdrnj5h03Qgojyizb42UeJievFqMWSjSEJBIKMNZLCU+oqW5Kfkh7LEFRskpAxzIiTMsZOb0WtsjGLMC8eB9hXF6ARSCwRSIWw8nD7ecgEjJLm4TUuJUTuCeDBMuuO9U3dA/i2h7toWcfBYvz/phX6fT6XRfegGjNEEBnertWAAAAABJRU5ErkJggg==","orcid":"","institution":"Central South University","correspondingAuthor":true,"prefix":"","firstName":"Kai","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2024-08-01 04:02:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4839107/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4839107/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61904820,"identity":"26024dab-db68-4405-aa0c-508f342ffc65","added_by":"auto","created_at":"2024-08-07 01:19:03","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":985179,"visible":true,"origin":"","legend":"","description":"","filename":"submission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4839107/v1_covered_075ac92a-361f-489f-aace-11eedd9c663a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Leveraging Single Tasks for Better Generalization of Multitask Gaussian Process on Multivariate Time Series","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"multitask Gaussian process, linear model of coregionalization, stochastic hyperparameter averaging, single task, multivariate time series","lastPublishedDoi":"10.21203/rs.3.rs-4839107/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4839107/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"By leveraging the knowledge of separate single tasks, we propose a simple and principled algorithm for multitask Gaussian process (GP), known as stochastic hyperparameter averaging (SHA), to obtain better generalization. Specifically, we focus on multivariate time series learning to improve the generalization of extrapolation and interpolation. The knowledge of a single task is extracted by a GP separately trained on one task-specific dimension of a multivariate time series. The single task GP (STGP) has the same kernel with the latent functions in multitask GP. By averaging hyperparameters of separate STGPs to initialize the latent functions of multitask GP,SHA identifies solutions that are significantly better than those found by popular training methods, but with only a few training steps of STGPs. SHA is kernel agnostic, remarkably straightforward to implement, and enhances generalization performance. Our SHA attains a significant boost in test accuracy across various diverse multivariate time series tasks, including interpolation, extrapolation, robustness with varying model complexities, and insensitivity to different hyperparameter initializations. ","manuscriptTitle":"Leveraging Single Tasks for Better Generalization of Multitask Gaussian Process on Multivariate Time Series","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-07 01:10:57","doi":"10.21203/rs.3.rs-4839107/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"bdaf2704-7fc4-403e-a0e5-699b90bd1db3","owner":[],"postedDate":"August 7th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-08-07T01:10:57+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-07 01:10:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4839107","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4839107","identity":"rs-4839107","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}