Soft Fuzzy Reinforcement Neural Network PD Controller

Soft Fuzzy Reinforcement Neural Network PD Controller | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Soft Fuzzy Reinforcement Neural Network PD Controller Qiang Han, Farid Boussaid, Mohammed Bennamoun This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5367881/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Fuzzy PD controllers are widely used in industry due to their excellent control performance and robustness. However, their performance heavily relies on manually designed fuzzy logic. Fuzzy neural networks (FNNs) combine neural networks and fuzzy logic, allowing them to utilize expert knowledge and possess self-learning capabilities. FNNs can be used to automatically tune PD parameters, resulting in a more robust fuzzy neural network PD (FNNPD) controller. Most current research on FNNPD controllers focuses on training using system mean square error, which can be inefficient and unstable, especially in complex systems. By integrating a critic in an actor-critic framework, learning efficiency improves as the critic provides direct feedback by evaluating actions, leading to faster and more stable training. This paper proposes an algorithm that applies the Soft Actor-Critic (SAC) framework to FNNPD controller, referred to as soft fuzzy reinforcement neural network PD (SFPD) controller. The SFPD algorithm can pre-set the parameters of FNNs using expert knowledge, ensuring good control performance from the beginning of training. By using reinforcement learning, the network parameters can be automatically adjusted. The use of a stochastic algorithm with entropy during the exploration process greatly accelerates exploration. Reported results demonstrate that the proposed method achieves rapid learning speed and superior control performance, and exhibits robustness to noise. The utilization of expert knowledge enables the proposed controller to exhibit good control performance from the start of training. Physical sciences/Engineering/Electrical and electronic engineering Physical sciences/Engineering/Mechanical engineering Deep reinforcement learning Soft Actor-Critic Fuzzy neural networks PD control Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5367881","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":389153616,"identity":"6fad308f-7506-46d2-91d0-474866c5d710","order_by":0,"name":"Qiang Han","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABLUlEQVRIiWNgGAWjYDACZjB5gIGBvQEqIgGTOkBICw9MBUEtcFmJBCK18B1nfvbwC8MdeXPJtw8f/qiwy+ef3cAmzVPDIMd3I4HxMw+mFsnDbObGMgzPDHfOTjc25jmTbDnjzgGglmMMxpI3EpilsWgxOMxgJi3BcJhxw+00NmnGNmYDA4kENukcNobEDTcSGLBrYf8G0mK/4eYx9p8//9VDtfxjqAdqYf6NVQuPmeQHhsNAM9nYGHgbDkO05LYxJBjcADKw+oWnTBqoMXnDmTSgy48dN5C4kdhs/bdPwnDmmYdtlnOwhNj549skf1Qctt1w/Bjjxx811Qb8M5IP3pzxzUae73jy4RtvsITyAWBs8higCDE2MEBiB8zAqoXxB1aZUTAKRsEoGAVQAACsVGVLWj2/fAAAAABJRU5ErkJggg==","orcid":"","institution":"University of Western Australia","correspondingAuthor":true,"prefix":"","firstName":"Qiang","middleName":"","lastName":"Han","suffix":""},{"id":389153617,"identity":"4c2c6e43-9a55-4871-920f-793bb0f4aa4e","order_by":1,"name":"Farid Boussaid","email":"","orcid":"","institution":"University of Western Australia","correspondingAuthor":false,"prefix":"","firstName":"Farid","middleName":"","lastName":"Boussaid","suffix":""},{"id":389153620,"identity":"af7bfbc9-edc5-4638-804e-cb0cf3137618","order_by":2,"name":"Mohammed Bennamoun","email":"","orcid":"","institution":"University of Western Australia","correspondingAuthor":false,"prefix":"","firstName":"Mohammed","middleName":"","lastName":"Bennamoun","suffix":""}],"badges":[],"createdAt":"2024-10-31 15:38:26","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5367881/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5367881/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":71756321,"identity":"16455bb2-e47b-41f1-98ce-7084f45d7420","added_by":"auto","created_at":"2024-12-18 10:09:27","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":579916,"visible":true,"origin":"","legend":"","description":"","filename":"SoftFuzzyReinforcementNeuralNetworkPDController.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5367881/v1_covered_c0a809fe-76ae-4577-974e-573d7b585681.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Soft Fuzzy Reinforcement Neural Network PD Controller","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Deep reinforcement learning, Soft Actor-Critic, Fuzzy neural networks, PD control","lastPublishedDoi":"10.21203/rs.3.rs-5367881/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5367881/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eFuzzy PD controllers are widely used in industry due to their excellent control performance and robustness. However, their performance heavily relies on manually designed fuzzy logic. Fuzzy neural networks (FNNs) combine neural networks and fuzzy logic, allowing them to utilize expert knowledge and possess self-learning capabilities. FNNs can be used to automatically tune PD parameters, resulting in a more robust fuzzy neural network PD (FNNPD) controller. Most current research on FNNPD controllers focuses on training using system mean square error, which can be inefficient and unstable, especially in complex systems. By integrating a critic in an actor-critic framework, learning efficiency improves as the critic provides direct feedback by evaluating actions, leading to faster and more stable training. This paper proposes an algorithm that applies the Soft Actor-Critic (SAC) framework to FNNPD controller, referred to as soft fuzzy reinforcement neural network PD (SFPD) controller. The SFPD algorithm can pre-set the parameters of FNNs using expert knowledge, ensuring good control performance from the beginning of training. By using reinforcement learning, the network parameters can be automatically adjusted. The use of a stochastic algorithm with entropy during the exploration process greatly accelerates exploration. Reported results demonstrate that the proposed method achieves rapid learning speed and superior control performance, and exhibits robustness to noise. The utilization of expert knowledge enables the proposed controller to exhibit good control performance from the start of training.\u003c/p\u003e","manuscriptTitle":"Soft Fuzzy Reinforcement Neural Network PD Controller","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-17 03:44:37","doi":"10.21203/rs.3.rs-5367881/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"26dfbe87-7699-4e29-a070-c9e6dba7bdd7","owner":[],"postedDate":"December 17th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":41475752,"name":"Physical sciences/Engineering/Electrical and electronic engineering"},{"id":41475753,"name":"Physical sciences/Engineering/Mechanical engineering"}],"tags":[],"updatedAt":"2024-12-18T10:08:53+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-17 03:44:37","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5367881","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5367881","identity":"rs-5367881","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}