Knowledge Graph Decision Transformer:Conditional sequence decision-making model for Offline RL integrated with knowledge graph

Knowledge Graph Decision Transformer:Conditional sequence decision-making model for Offline RL integrated with knowledge graph | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Knowledge Graph Decision Transformer:Conditional sequence decision-making model for Offline RL integrated with knowledge graph Shan He, Zhaoyang Lu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4446588/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Decision Transformer is one of the representative algorithms in the field of offline reinforcement learning for conditional sequence modeling. It can learn from past decisions and predict future action strategies, performing well in most tasks. However, recent research indicates its limitations in concatenating different trajectory segments, meaning it struggles to utilize suboptimal trajectories effectively. This paper combines Decision Transformer with knowledge graph link prediction, leveraging the strengths of both methods to enhance decision-making in complex environments. While Decision Transformer provides a robust framework for handling sequential data, knowledge graph link prediction offers structured representations of entity relationships and predicts potential links between entities, allowing the model to infer unobserved state transitions. By integrating knowledge graph predictions into the Decision Transformer model, we can enhance decision accuracy by leveraging additional contextual and relational information. This synergy enables more effective utilization of available data and better adaptation to diverse scenarios. Through experiments and evaluations, we have demonstrated the effectiveness of this integration approach in improving the performance of offline reinforcement learning models across various tasks and domains. Decision Transformer Sequential decision-making Link prediction Knowledge Graph Offline reinforcement learning Trajectory optimization Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4446588","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":308248858,"identity":"d18a577f-59c5-4b67-b923-69a1d3af6c28","order_by":0,"name":"Shan He","email":"","orcid":"","institution":"Southwest Petroleum University","correspondingAuthor":false,"prefix":"","firstName":"Shan","middleName":"","lastName":"He","suffix":""},{"id":308248861,"identity":"1a5a6ee6-0dd5-4890-b504-39b9299f4a95","order_by":1,"name":"Zhaoyang Lu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABJklEQVRIie3Rv0rEMBzA8ZRAu0Sy5jh6vsKvBHoORV+l5aCTRd0ccxy0k7pW8CEKgnNKQZe6CzoUhJscDhxOuIImN6hDijc65EumJB/IH4Rstn8YxbjuVsCIu18LWP1accQAGRX5LChPowllWJyVCP4m0LbhmKxSPio98U52IegpVqeCJqme5/PbqO8TQdvwlaDIryRedgbhlLEymrzUC57lkAh2POUEpbyS7hQMBLNYxqCJTHKeCU327sYE6RniMgNxWSKU2ZJic9ArQh81+RwkhDSOkKCvnwiOXEVQpokcJMzLsSNAPTKpRXCRc56zk3VwAzN+3bihiRw1dL3pe/WVXtHBRz/xr+h92r2dH/qXD4uliRhutx36qfBO+7+VzWaz2X76ArwAaazhdHiKAAAAAElFTkSuQmCC","orcid":"","institution":"Southwest Petroleum University","correspondingAuthor":true,"prefix":"","firstName":"Zhaoyang","middleName":"","lastName":"Lu","suffix":""}],"badges":[],"createdAt":"2024-05-20 04:36:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4446588/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4446588/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":58074947,"identity":"40a9a848-b745-4fcc-a68c-fed4f0f91008","added_by":"auto","created_at":"2024-06-10 21:01:42","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1996946,"visible":true,"origin":"","legend":"","description":"","filename":"snarticle.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4446588/v1_covered_eabd3a20-896c-40e8-98ce-657334996f4d.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Knowledge Graph Decision Transformer:Conditional sequence decision-making model for Offline RL integrated with knowledge graph","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Decision Transformer, Sequential decision-making, Link prediction, Knowledge Graph, Offline reinforcement learning, Trajectory optimization","lastPublishedDoi":"10.21203/rs.3.rs-4446588/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4446588/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Decision Transformer is one of the representative algorithms in the field of offline reinforcement learning for conditional sequence modeling. It can learn from past decisions and predict future action strategies, performing well in most tasks. However, recent research indicates its limitations in concatenating different trajectory segments, meaning it struggles to utilize suboptimal trajectories effectively. This paper combines Decision Transformer with knowledge graph link prediction, leveraging the strengths of both methods to enhance decision-making in complex environments. While Decision Transformer provides a robust framework for handling sequential data, knowledge graph link prediction offers structured representations of entity relationships and predicts potential links between entities, allowing the model to infer unobserved state transitions. By integrating knowledge graph predictions into the Decision Transformer model, we can enhance decision accuracy by leveraging additional contextual and relational information. This synergy enables more effective utilization of available data and better adaptation to diverse scenarios. Through experiments and evaluations, we have demonstrated the effectiveness of this integration approach in improving the performance of offline reinforcement learning models across various tasks and domains.","manuscriptTitle":"Knowledge Graph Decision Transformer:Conditional sequence decision-making model for Offline RL integrated with knowledge graph","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-03 06:28:21","doi":"10.21203/rs.3.rs-4446588/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"132e3ba5-c7ee-4679-91a7-34222c7da458","owner":[],"postedDate":"June 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-06-13T11:18:52+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-03 06:28:21","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4446588","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4446588","identity":"rs-4446588","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}