An adaptive reference vector strategy with Q-learning for many-objective workflow scheduling problem in the cloud computing environment

An adaptive reference vector strategy with Q-learning for many-objective workflow scheduling problem in the cloud computing environment | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An adaptive reference vector strategy with Q-learning for many-objective workflow scheduling problem in the cloud computing environment Tingting Dong, Wenyu Fan, Peiwen Wang, Fei Xue, Yuezheng Chen This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4467172/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Cloud computing environment is widely used in various fields, and the scientific workflow scheduling problem in this environment is a many-objective optimization problem and has attracted much attention. Aiming at meeting the different demands of multiple users, this paper proposes an adaptive many-objective algorithm (AD\_CLIA) based on cascade clustering and reference point incremental learning algorithm (CLIA). First, this paper constructs a workflow scheduling model with four objectives: completion time (makespan), cost load, and average resource utilization (AU). Then, for improving the convergence and diversity of CLIA, a reinforcement learning method for adaptively selecting effective reference vectors is proposed. And at the same time, a double-faced mirror strategy is constructed to deal with the problem of uneven distribution of the optimal solution set. It has shown advantages in both low-dimensional DTLZ test problems and high-dimensional WFG and MaF test problems. Finally, the proposed algorithm is applied to four famous real workflow problems and the results are satisfactory. Cloud computing Many-objective workflow scheduling Adaptive reference vector parameter selection Reinforcement learning Double-fanced mirror strategy Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4467172","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":309335943,"identity":"5cc26607-c9f0-40a5-9c6b-fd0689c52a99","order_by":0,"name":"Tingting Dong","email":"","orcid":"","institution":"Beijing Wuzi University","correspondingAuthor":false,"prefix":"","firstName":"Tingting","middleName":"","lastName":"Dong","suffix":""},{"id":309335944,"identity":"b5076e4a-9b0e-4455-8682-2c30bcb1fa93","order_by":1,"name":"Wenyu Fan","email":"","orcid":"","institution":"Beijing Wuzi University","correspondingAuthor":false,"prefix":"","firstName":"Wenyu","middleName":"","lastName":"Fan","suffix":""},{"id":309335945,"identity":"d6a99bcb-ebef-44f0-b171-17aff9c8d85a","order_by":2,"name":"Peiwen Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYNACAwkefvYGEMOCWC0FNjKSPQfAeonV8iHNxuBGAohFhBaD471HN90wOMzDcPP51Q0/CiQY+Nu7E/BrOXMu7XYOUAvj7Jyymz1Ah0mcObsBrxazGzlmYC3M0jlpN3iAWgwkcglouf8GooVN8kzazT9EabnBA9KSxsMjwX7sNlG22J8BO8yGR4Inh+22DDCCCPpFsv0MUMsfCXv748ef3Xzzx0aOv70XvxYkwGMAJolVDgLsD0hRPQpGwSgYBSMIAACVTEgzAuTxhwAAAABJRU5ErkJggg==","orcid":"","institution":"Beijing Wuzi University","correspondingAuthor":true,"prefix":"","firstName":"Peiwen","middleName":"","lastName":"Wang","suffix":""},{"id":309335946,"identity":"db0de853-e07a-47d7-8ae4-00a49adb88e3","order_by":3,"name":"Fei Xue","email":"","orcid":"","institution":"Beijing Wuzi University","correspondingAuthor":false,"prefix":"","firstName":"Fei","middleName":"","lastName":"Xue","suffix":""},{"id":309335947,"identity":"b5137aae-1a3d-4b57-8d5a-51b883a37ddb","order_by":4,"name":"Yuezheng Chen","email":"","orcid":"","institution":"Beijing Wuzi University","correspondingAuthor":false,"prefix":"","firstName":"Yuezheng","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2024-05-23 13:22:56","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4467172/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4467172/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":78022336,"identity":"445bcb36-1bf9-4014-9ce9-39dd35ac8340","added_by":"auto","created_at":"2025-03-08 04:02:00","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6803237,"visible":true,"origin":"","legend":"","description":"","filename":"ADCLIA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4467172/v1_covered_d1e207d3-0f43-49ab-b830-7bbfb570fa4f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"An adaptive reference vector strategy with Q-learning for many-objective workflow scheduling problem in the cloud computing environment","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Cloud computing, Many-objective workflow scheduling, Adaptive reference vector parameter selection, Reinforcement learning, Double-fanced mirror strategy","lastPublishedDoi":"10.21203/rs.3.rs-4467172/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4467172/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCloud computing environment is widely used in various fields, and the scientific workflow scheduling problem in this environment is a many-objective optimization problem and has attracted much attention. Aiming at meeting the different demands of multiple users, this paper proposes an adaptive many-objective algorithm (AD\\_CLIA) based on cascade clustering and reference point incremental learning algorithm (CLIA). First, this paper constructs a workflow scheduling model with four objectives: completion time (makespan), cost load, and average resource utilization (AU). Then, for improving the convergence and diversity of CLIA, a reinforcement learning method for adaptively selecting effective reference vectors is proposed. And at the same time, a double-faced mirror strategy is constructed to deal with the problem of uneven distribution of the optimal solution set. It has shown advantages in both low-dimensional DTLZ test problems and high-dimensional WFG and MaF test problems. Finally, the proposed algorithm is applied to four famous real workflow problems and the results are satisfactory.\u003c/p\u003e","manuscriptTitle":"An adaptive reference vector strategy with Q-learning for many-objective workflow scheduling problem in the cloud computing environment","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-05 18:03:18","doi":"10.21203/rs.3.rs-4467172/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c796cedf-084a-4173-aad7-c3be0d1ea157","owner":[],"postedDate":"June 5th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-03-08T03:53:29+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-05 18:03:18","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4467172","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4467172","identity":"rs-4467172","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}