A multimodal deep reinforcement learning framework for multi-period inventory decision-making under demand uncertainty

A multimodal deep reinforcement learning framework for multi-period inventory decision-making under demand uncertainty | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A multimodal deep reinforcement learning framework for multi-period inventory decision-making under demand uncertainty Yu-Xin Tian, Chuan Zhang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6843287/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 25 Sep, 2025 Read the published version in Fuzzy Optimization and Decision Making → Version 1 posted You are reading this latest preprint version Abstract We investigate the problem of multi-period inventory decision-making driven by multisource multimodal data and propose a deep reinforcement learning method--WET-TD3--that integrates multimodal environmental perception with policy optimization to generate end-to-end replenishment quantities for each period. First, based on demand-related structured features and unstructured customer review texts from multiple sources, we design a set of multimodal feature-aware agent neural networks incorporating word embeddings and Transformer modules, thereby constructing a state space adaptable to dynamic market environments. Second, we enhance the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to support a multimodal Actor-Critic architecture tailored for high-dimensional heterogeneous inputs. Additionally, we introduce delayed policy updates, experience replay, and exploration noise mechanisms to improve training stability. Finally, experiments based on real-world data show that the WET-TD3 method significantly outperforms benchmark approaches in multi-period inventory management, achieving an average cost reduction of over 53.69%. The method dynamically adjusts replenishment strategies in response to changes in the relative magnitude of unit holding and underage costs, maintaining stable performance under varying cost structures. These findings highlight that the deep integration of unstructured textual reviews and structured features from multiple sources is fundamental to achieving high-accuracy replenishment, while the reinforcement learning framework effectively supports long-term optimization goals in uncertain and dynamic demand environments. Inventory optimization Deep reinforcement learning Multimodal data fusion Decision-making under uncertainty Transformer Full Text Additional Declarations No competing interests reported. Supplementary Files Supplementalmaterials2025.pdf Cite Share Download PDF Status: Published Journal Publication published 25 Sep, 2025 Read the published version in Fuzzy Optimization and Decision Making → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6843287","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":485704483,"identity":"75b353a4-7903-4475-8a43-b5abcdc8fc18","order_by":0,"name":"Yu-Xin Tian","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Yu-Xin","middleName":"","lastName":"Tian","suffix":""},{"id":485704484,"identity":"e6ee7dd4-d37e-4a8e-9d58-c8dd1e0ea2ae","order_by":1,"name":"Chuan Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYFACHhBhw8DAzNwAFUkgSksaUAsjaVoOAzGxWvj7zx78XPDrfDR/O2MDc0HFYQZ+9hwDhp87cGuROHAuWXpm3+3cGYeBWmacOcwg2fPGgLH3DG4tBow9BtK8PbdzG0BaeNsOMxjcyDFgZmzDo4WZx/g3b8+53PlgLf8OM9gT1MLGYybN8+NA7gawlgagLRIEtEic4TGz5m1Izt0I1HKY51g6j8SZZwUHe/Fo4e8/Y3yb549d7rzzhw8+5qmxluNvT9744CceLWAAc8YBBmg0HSCgAQj+EFYyCkbBKBgFIxgAAK7oTmRVnWLQAAAAAElFTkSuQmCC","orcid":"","institution":"Northeastern University","correspondingAuthor":true,"prefix":"","firstName":"Chuan","middleName":"","lastName":"Zhang","suffix":""}],"badges":[],"createdAt":"2025-06-07 14:08:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6843287/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6843287/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10700-025-09462-0","type":"published","date":"2025-09-25T15:57:24+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":92431110,"identity":"e01bef66-b56c-4712-9384-ad0595e88c8b","added_by":"auto","created_at":"2025-09-29 16:08:28","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1683642,"visible":true,"origin":"","legend":"","description":"","filename":"ManuscriptFODM2025.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6843287/v1_covered_e12695a2-dc8e-486e-9f19-c966fb9020e3.pdf"},{"id":86915744,"identity":"eee33b04-d873-44ae-b6dc-489777a24bf6","added_by":"auto","created_at":"2025-07-17 06:26:28","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":538313,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementalmaterials2025.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6843287/v1/66a7a35cb0e1dd761df1496d.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A multimodal deep reinforcement learning framework for multi-period inventory decision-making under demand uncertainty","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Inventory optimization, Deep reinforcement learning, Multimodal data fusion, Decision-making under uncertainty, Transformer","lastPublishedDoi":"10.21203/rs.3.rs-6843287/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6843287/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"We investigate the problem of multi-period inventory decision-making driven by multisource multimodal data and propose a deep reinforcement learning method--WET-TD3--that integrates multimodal environmental perception with policy optimization to generate end-to-end replenishment quantities for each period. First, based on demand-related structured features and unstructured customer review texts from multiple sources, we design a set of multimodal feature-aware agent neural networks incorporating word embeddings and Transformer modules, thereby constructing a state space adaptable to dynamic market environments. Second, we enhance the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to support a multimodal Actor-Critic architecture tailored for high-dimensional heterogeneous inputs. Additionally, we introduce delayed policy updates, experience replay, and exploration noise mechanisms to improve training stability. Finally, experiments based on real-world data show that the WET-TD3 method significantly outperforms benchmark approaches in multi-period inventory management, achieving an average cost reduction of over 53.69%. The method dynamically adjusts replenishment strategies in response to changes in the relative magnitude of unit holding and underage costs, maintaining stable performance under varying cost structures. These findings highlight that the deep integration of unstructured textual reviews and structured features from multiple sources is fundamental to achieving high-accuracy replenishment, while the reinforcement learning framework effectively supports long-term optimization goals in uncertain and dynamic demand environments.","manuscriptTitle":"A multimodal deep reinforcement learning framework for multi-period inventory decision-making under demand uncertainty","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-17 06:26:23","doi":"10.21203/rs.3.rs-6843287/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ce5fa455-7cf4-43d6-a9ec-fbb6c70169c1","owner":[],"postedDate":"July 17th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-09-29T16:07:30+00:00","versionOfRecord":{"articleIdentity":"rs-6843287","link":"https://doi.org/10.1007/s10700-025-09462-0","journal":{"identity":"fuzzy-optimization-and-decision-making","isVorOnly":false,"title":"Fuzzy Optimization and Decision Making"},"publishedOn":"2025-09-25 15:57:24","publishedOnDateReadable":"September 25th, 2025"},"versionCreatedAt":"2025-07-17 06:26:23","video":"","vorDoi":"10.1007/s10700-025-09462-0","vorDoiUrl":"https://doi.org/10.1007/s10700-025-09462-0","workflowStages":[]},"version":"v1","identity":"rs-6843287","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6843287","identity":"rs-6843287","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}