Efficient Multi-Agent Reinforcement Learning HVAC Power Consumption Optimization

Efficient Multi-Agent Reinforcement Learning HVAC Power Consumption Optimization | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Efficient Multi-Agent Reinforcement Learning HVAC Power Consumption Optimization Chenyang Miao, Yunduan Cui, Huiyun Li, Xinyu Wu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4644056/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Intelligent and energy-efficient heating, ventilation, and air conditioning (HVAC) system plays an important role in reducing energy consumption and protecting our environment. In this work, we focus on exploring suitable power optimization strategies using reinforcement learning (RL) without relying on human prior knowledge. A novel RL approach, multi-stabilization network(MADDPG-MSN) is proposed to tackle the sample-efficiency issue of current RL-based approaches for HVAC systems. Employing the multi-stabilization network trick, MADDPG-MSN efficiently learns to balance temperature control and power consumption with a limited number of interactions. Evaluated by the simulated data center scenario, it reduced 28% powerusage without compromising temperature control capability compared with the traditional model-predictive controller. In the real-world air conditioner testing, it demonstrated superior control performances than the built-in controller with 35% less power consumption and 21% smaller standard deviation of the indoor temperature after 72 hours’ learning. These results demonstrate the superior effectiveness and practicality of MADDPG-MSN in HVAC power consumption optimization, expanding the potential of RL as an emerging direction to more energy-saving systems. Physical sciences/Engineering/Electrical and electronic engineering Physical sciences/Mathematics and computing/Computer science Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4644056","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":329102691,"identity":"1e0e5967-6985-465e-a0a1-0af8ea4057cb","order_by":0,"name":"Chenyang Miao","email":"","orcid":"","institution":"University of Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Chenyang","middleName":"","lastName":"Miao","suffix":""},{"id":329102692,"identity":"61ef71ad-5ea5-4a3b-a5f1-083e4209cc20","order_by":1,"name":"Yunduan Cui","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0klEQVRIiWNgGAWjYFACxgYgwQzCB4CEhAwpWtgSQFp4iLUKpIXHAMQirIV/RnKbNO8O68T+2T2fX92oseBhYD98dAM+LRI3EoFazqQnzrhzdpt1zjGgw3jS0m7g02IgAdLSdjix4UbuNuMcNqAWCR4z4rTMv5HzzDjnHylaNtzIYX6c20aEFokzD5st57alG2+8kWbGnNsnwcNGyC/87ekPb7xts5addyP58eecb3Vy/OyHj+HVAgQsElAGG5jBRkA5CDB/QGeMglEwCkbBKEABAJmjR3Q830gAAAAAAElFTkSuQmCC","orcid":"","institution":"Shenzhen Institutes of Advanced Technology","correspondingAuthor":true,"prefix":"","firstName":"Yunduan","middleName":"","lastName":"Cui","suffix":""},{"id":329102694,"identity":"9bc30bcc-c71c-49b3-bacc-34e52a4dd8cc","order_by":2,"name":"Huiyun Li","email":"","orcid":"","institution":"Shenzhen Institutes of Advanced Technology","correspondingAuthor":false,"prefix":"","firstName":"Huiyun","middleName":"","lastName":"Li","suffix":""},{"id":329102695,"identity":"58e73fc0-e04e-4993-b3fb-c07423693f81","order_by":3,"name":"Xinyu Wu","email":"","orcid":"","institution":"Shenzhen Institutes of Advanced Technology","correspondingAuthor":false,"prefix":"","firstName":"Xinyu","middleName":"","lastName":"Wu","suffix":""}],"badges":[],"createdAt":"2024-06-26 16:29:09","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4644056/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4644056/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61481705,"identity":"201a2986-89ba-4869-9680-1a7d9fd4db95","added_by":"auto","created_at":"2024-07-31 08:57:44","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4226816,"visible":true,"origin":"","legend":"","description":"","filename":"IntelligentHVACcontrol.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4644056/v1_covered_483f7c27-372e-4006-a2ef-e27f4f942081.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Efficient Multi-Agent Reinforcement Learning HVAC Power Consumption Optimization","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4644056/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4644056/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Intelligent and energy-efficient heating, ventilation, and air conditioning (HVAC) system plays an important role in reducing energy consumption and protecting our environment. In this work, we focus on exploring suitable power optimization strategies using reinforcement learning (RL) without relying on human prior knowledge. A novel RL approach, multi-stabilization network(MADDPG-MSN) is proposed to tackle the sample-efficiency issue of current RL-based approaches for HVAC systems. Employing the multi-stabilization network trick, MADDPG-MSN efficiently learns to balance temperature control and power consumption with a limited number of interactions. Evaluated by the simulated data center scenario, it reduced 28% powerusage without compromising temperature control capability compared with the traditional model-predictive controller. In the real-world air conditioner testing, it demonstrated superior control performances than the built-in controller with 35% less power consumption and 21% smaller standard deviation of the indoor temperature after 72 hours’ learning. These results demonstrate the superior effectiveness and practicality of MADDPG-MSN in HVAC power consumption optimization, expanding the potential of RL as an emerging direction to more energy-saving systems.","manuscriptTitle":"Efficient Multi-Agent Reinforcement Learning HVAC Power Consumption Optimization","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-22 03:44:15","doi":"10.21203/rs.3.rs-4644056/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e45a3ea3-eb80-4b97-ad38-e3f4e16e1729","owner":[],"postedDate":"July 22nd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":34839857,"name":"Physical sciences/Engineering/Electrical and electronic engineering"},{"id":34839858,"name":"Physical sciences/Mathematics and computing/Computer science"}],"tags":[],"updatedAt":"2024-07-31T08:49:34+00:00","versionOfRecord":[],"versionCreatedAt":"2024-07-22 03:44:15","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4644056","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4644056","identity":"rs-4644056","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}