QRL-AFOFA: Q-Learning Enhanced Self-Adaptive Fractional Order Firefly Algorithm for Large-Scale and Dynamic Multiobjective Optimization Problems

QRL-AFOFA: Q-Learning Enhanced Self-Adaptive Fractional Order Firefly Algorithm for Large-Scale and Dynamic Multiobjective Optimization Problems | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article QRL-AFOFA: Q-Learning Enhanced Self-Adaptive Fractional Order Firefly Algorithm for Large-Scale and Dynamic Multiobjective Optimization Problems Yashar Mousavi, Parastoo Akbari, Rashin Mousavi, Ibrahim Beklan Kucukdemiral, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7228356/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 19 Feb, 2026 Read the published version in Artificial Intelligence Review → Version 1 posted 22 You are reading this latest preprint version Abstract This paper introduces QRL-AFOFA, a Q-learning-enhanced adaptive fractional-order firefly algorithm developed to address the challenges of large-scale and dynamic multiobjective optimization problems. While fractional-order metaheuristics provide memory-driven search dynamics and reinforcement learning (RL) offers adaptive policy control, existing hybrid methods often face critical limitations such as parameter sensitivity, premature convergence, and poor diversity preservation. To overcome these challenges, QRL-AFOFA integrates five synergistic innovations: real-time adaptive tuning of fractional-order parameters, entropy-regularized Q-value updates, stagnation-aware restart strategies, reflection-based boundary handling, and dual-phase learning rate scheduling. Extensive experiments on the 2021 IEEE Congress on Evolutionary Computation (CEC2021) benchmark functions demonstrate that QRL-AFOFA consistently outperforms other state-of-the-art algorithms across diverse problem categories. Accordingly, the proposed QRL-AFOFA demonstrated superior performance in 97.5% of test cases and outperformed the state-of-the-art algorithms in 34-40 out of 40 benchmark problems, with particularly impressive gains in dynamic and large-scale scenarios. Statistical validation using the Wilcoxon signed-rank and Friedman tests confirms the significance of the improvements. Notably, QRL-AFOFA achieves exceptional performance in high-dimensional (up to 10,000 variables) and dynamic optimization settings. Its self-adaptive design eliminates manual parameter tuning, making it a robust, scalable, and intelligent optimization framework for complex real-world applications. Reinforcement learning Q-learning Large-scale optimization Dynamic multiobjective optimization Fractional calculus Adaptive parameter control Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 19 Feb, 2026 Read the published version in Artificial Intelligence Review → Version 1 posted Editorial decision: Revision requested 07 Sep, 2025 Reviews received at journal 31 Aug, 2025 Reviews received at journal 31 Aug, 2025 Reviews received at journal 29 Aug, 2025 Reviews received at journal 21 Aug, 2025 Reviews received at journal 20 Aug, 2025 Reviewers agreed at journal 18 Aug, 2025 Reviewers agreed at journal 17 Aug, 2025 Reviewers agreed at journal 17 Aug, 2025 Reviewers agreed at journal 16 Aug, 2025 Reviews received at journal 16 Aug, 2025 Reviewers agreed at journal 16 Aug, 2025 Reviewers agreed at journal 12 Aug, 2025 Reviewers agreed at journal 12 Aug, 2025 Reviewers agreed at journal 12 Aug, 2025 Reviewers agreed at journal 10 Aug, 2025 Reviewers agreed at journal 10 Aug, 2025 Reviewers agreed at journal 09 Aug, 2025 Reviewers invited by journal 09 Aug, 2025 Editor assigned by journal 08 Aug, 2025 Submission checks completed at journal 02 Aug, 2025 First submitted to journal 27 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7228356","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":501426591,"identity":"549f061c-b928-4137-a4c0-0c4ba8c5a1ed","order_by":0,"name":"Yashar Mousavi","email":"","orcid":"","institution":"Glasgow Caledonian University","correspondingAuthor":false,"prefix":"","firstName":"Yashar","middleName":"","lastName":"Mousavi","suffix":""},{"id":501426592,"identity":"61f84e80-04bc-4d42-85a3-406e8d5ed8bd","order_by":1,"name":"Parastoo Akbari","email":"","orcid":"","institution":"Iowa State University","correspondingAuthor":false,"prefix":"","firstName":"Parastoo","middleName":"","lastName":"Akbari","suffix":""},{"id":501426593,"identity":"6dcb2ea2-79ff-4e04-8a42-75a8ea9c9c17","order_by":2,"name":"Rashin Mousavi","email":"","orcid":"","institution":"Paradise Research Center","correspondingAuthor":false,"prefix":"","firstName":"Rashin","middleName":"","lastName":"Mousavi","suffix":""},{"id":501426597,"identity":"58d3b520-eebd-4a11-8eb6-89d48c829c3d","order_by":3,"name":"Ibrahim Beklan Kucukdemiral","email":"","orcid":"","institution":"Glasgow Caledonian University","correspondingAuthor":false,"prefix":"","firstName":"Ibrahim","middleName":"Beklan","lastName":"Kucukdemiral","suffix":""},{"id":501426598,"identity":"7da380d4-3c76-4260-9506-ce57c4d22989","order_by":4,"name":"Afef Fekih","email":"","orcid":"","institution":"University of Louisiana at Lafayette","correspondingAuthor":false,"prefix":"","firstName":"Afef","middleName":"","lastName":"Fekih","suffix":""},{"id":501426601,"identity":"99ca9f94-7cf1-4618-9ace-84676df4a1d2","order_by":5,"name":"Umit Cali","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtElEQVRIiWNgGAWjYFACxgYQKcPG3kC8lkaQWh42ngOMRGsCq+RhkEggUovBtcPtDxhq7Hj4JN8ef1zYxpDHT1DL7USgw44l87BJ5yU2z2xjKJYkZBdYC2MDM1BLjmEzbxtD4oYDxGmp52GTPEOalsM8bBI8RGqRBGqZkXDsODCQcwxn85yTSJxJyC98t9MffPhQUy0n337G4DNPmU1iPwEdEJCAYEoQpWEUjIJRMApGAQEAAO5fO5Qg8HZ9AAAAAElFTkSuQmCC","orcid":"","institution":"Norwegian University of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Umit","middleName":"","lastName":"Cali","suffix":""}],"badges":[],"createdAt":"2025-07-27 21:38:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7228356/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7228356/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10462-026-11511-y","type":"published","date":"2026-02-19T15:58:01+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":103252297,"identity":"1d66c699-55dc-43be-8939-e1d5d5d176c2","added_by":"auto","created_at":"2026-02-23 16:14:11","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1028354,"visible":true,"origin":"","legend":"","description":"","filename":"LatexManuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7228356/v1_covered_c06a8764-19e7-4144-a7ca-39841cf5f864.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"QRL-AFOFA: Q-Learning Enhanced Self-Adaptive Fractional Order Firefly Algorithm for Large-Scale and Dynamic Multiobjective Optimization Problems","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"artificial-intelligence-review","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"aire","sideBox":"Learn more about [Artificial Intelligence Review](http://link.springer.com/journal/10462)","snPcode":"10462","submissionUrl":"https://submission.nature.com/new-submission/10462/3","title":"Artificial Intelligence Review","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Reinforcement learning, Q-learning, Large-scale optimization, Dynamic multiobjective optimization, Fractional calculus, Adaptive parameter control","lastPublishedDoi":"10.21203/rs.3.rs-7228356/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7228356/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis paper introduces QRL-AFOFA, a Q-learning-enhanced adaptive fractional-order firefly algorithm developed to address the challenges of large-scale and dynamic multiobjective optimization problems. While fractional-order metaheuristics provide memory-driven search dynamics and reinforcement learning (RL) offers adaptive policy control, existing hybrid methods often face critical limitations such as parameter sensitivity, premature convergence, and poor diversity preservation. To overcome these challenges, QRL-AFOFA integrates five synergistic innovations: real-time adaptive tuning of fractional-order parameters, entropy-regularized Q-value updates, stagnation-aware restart strategies, reflection-based boundary handling, and dual-phase learning rate scheduling. Extensive experiments on the 2021 IEEE Congress on Evolutionary Computation (CEC2021) benchmark functions demonstrate that QRL-AFOFA consistently outperforms other state-of-the-art algorithms across diverse problem categories. Accordingly, the proposed QRL-AFOFA demonstrated superior performance in 97.5% of test cases and outperformed the state-of-the-art algorithms in 34-40 out of 40 benchmark problems, with particularly impressive gains in dynamic and large-scale scenarios. Statistical validation using the Wilcoxon signed-rank and Friedman tests confirms the significance of the improvements. Notably, QRL-AFOFA achieves exceptional performance in high-dimensional (up to 10,000 variables) and dynamic optimization settings. Its self-adaptive design eliminates manual parameter tuning, making it a robust, scalable, and intelligent optimization framework for complex real-world applications.\u003c/p\u003e","manuscriptTitle":"QRL-AFOFA: Q-Learning Enhanced Self-Adaptive Fractional Order Firefly Algorithm for Large-Scale and Dynamic Multiobjective Optimization Problems","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-18 04:41:35","doi":"10.21203/rs.3.rs-7228356/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-09-08T01:54:00+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-01T01:24:15+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-31T11:15:31+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-29T13:38:26+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-21T19:57:32+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-21T01:31:26+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"87647848951546589246681330727781717196","date":"2025-08-18T14:40:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"116942894308755136901572510494848229379","date":"2025-08-17T07:39:04+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"32547306868827324238682932974275341726","date":"2025-08-17T06:26:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"22063695142195911315536437414045514879","date":"2025-08-17T00:09:28+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-16T22:41:49+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"96857165219644559700402828215784818424","date":"2025-08-16T22:18:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"269533663034221192183485137663451178874","date":"2025-08-13T00:59:59+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"167024069800182680618122224368031559023","date":"2025-08-12T13:14:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"133760221835184552481716329200088764169","date":"2025-08-12T06:26:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"179054964557022331312769685610217722964","date":"2025-08-10T14:06:23+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"38173192506990710025237075296348148952","date":"2025-08-10T12:23:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"164376220101741975805011613572652915674","date":"2025-08-09T19:34:03+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-08-09T19:27:38+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-09T02:47:25+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-02T13:27:25+00:00","index":"","fulltext":""},{"type":"submitted","content":"Artificial Intelligence Review","date":"2025-07-27T21:24:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"artificial-intelligence-review","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"aire","sideBox":"Learn more about [Artificial Intelligence Review](http://link.springer.com/journal/10462)","snPcode":"10462","submissionUrl":"https://submission.nature.com/new-submission/10462/3","title":"Artificial Intelligence Review","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"dd6c12a5-606c-4e54-baa6-340c83b44aa2","owner":[],"postedDate":"August 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-02-23T16:09:53+00:00","versionOfRecord":{"articleIdentity":"rs-7228356","link":"https://doi.org/10.1007/s10462-026-11511-y","journal":{"identity":"artificial-intelligence-review","isVorOnly":false,"title":"Artificial Intelligence Review"},"publishedOn":"2026-02-19 15:58:01","publishedOnDateReadable":"February 19th, 2026"},"versionCreatedAt":"2025-08-18 04:41:35","video":"","vorDoi":"10.1007/s10462-026-11511-y","vorDoiUrl":"https://doi.org/10.1007/s10462-026-11511-y","workflowStages":[]},"version":"v1","identity":"rs-7228356","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7228356","identity":"rs-7228356","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}