DPP-CL: Orthogonal Subspace Continual Learning for Dialogue Policy Planning

DPP-CL: Orthogonal Subspace Continual Learning for Dialogue Policy Planning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article DPP-CL: Orthogonal Subspace Continual Learning for Dialogue Policy Planning Yue Han, Rong Jiang, Yinxuan Huang, Aiping Li, Weihong Han This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7024974/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 18 Oct, 2025 Read the published version in World Wide Web → Version 1 posted 10 You are reading this latest preprint version Abstract Continual learning (CL) is critical for large language models (LLMs) to tackle multiple tasks in various downstream applications. However, LLMs struggle to maintain performance in multiple-task training due to catastrophic forgetting, which undermines the learned parameters for old tasks during training for new tasks. Although existing studies typically employ three methods to address this issue, including regularization-based, rehearsal-based, and parameter isolation-based methods, these methods simultaneously lead to new issues, such as privacy concerns, difficulties in handling long sequences. To address existing problems and enhance performance, we propose DPP-CL, a Dialogue Policy Planner (DPP) utilized by CL to facilitate forgetting and improve performance. Specifically, we used gradient descent in an orthogonal subspace to learn new tasks for DPP in multitask learning. Regarding the parameter isolation-based approach that struggles to handle long sequences, we introduce the concatenation of hyperbolic spherical embeddings and Euclidean embeddings to form a novel representation, thereby enhancing the capability of LLMs to understand long-sequence texts. In particular, the DPP training process proceeds sequentially through supervised learning, knowledge distillation, and reinforcement learning. We performed comparison experiments using the standard CL benchmarks and incorporated retrieval-augmented generation into the dialogue architecture for multitask inference in cybersecurity. Continual Learning Dialogue Policy Planning Knowledge Inference Knowledge Graph Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 18 Oct, 2025 Read the published version in World Wide Web → Version 1 posted Editorial decision: Revision requested 01 Sep, 2025 Reviews received at journal 26 Aug, 2025 Reviewers agreed at journal 11 Aug, 2025 Reviewers agreed at journal 10 Aug, 2025 Reviews received at journal 04 Aug, 2025 Reviewers agreed at journal 16 Jul, 2025 Reviewers invited by journal 12 Jul, 2025 Editor assigned by journal 04 Jul, 2025 Submission checks completed at journal 04 Jul, 2025 First submitted to journal 02 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7024974","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":486572151,"identity":"dcaf21c1-205c-4ea7-b71b-53a02e715042","order_by":0,"name":"Yue Han","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Yue","middleName":"","lastName":"Han","suffix":""},{"id":486572152,"identity":"5fffcf6d-79cb-49a2-a4f3-993939471993","order_by":1,"name":"Rong Jiang","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Rong","middleName":"","lastName":"Jiang","suffix":""},{"id":486572153,"identity":"b0f60aaa-90cc-417e-9605-996a03e33d11","order_by":2,"name":"Yinxuan Huang","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Yinxuan","middleName":"","lastName":"Huang","suffix":""},{"id":486572154,"identity":"0557a57b-082b-4a7a-af66-edff4a99334b","order_by":3,"name":"Aiping Li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA20lEQVRIiWNgGAWjYBACPmYGhgMMBgwM/ECOBBAzNhDSwgbTIglUKnGAKC0whsEBorWwMz88XFBwx27z7R7D2x8YbGQ3HGB+9gC/w9gMDs8weJa87c4ZY4sDDGnGGw6wmRsQ8IvBYR6Dw8lmN3LMgA47nLjhAA+bBH4t7B/AWoxngLX8J0YLD9gWOwMJsJYDRGkpAPrlcILEjbRiizMGycYzD7OZ4dXCz3988+eCP4ft+Wckb7xRUWEn23e8+RleLSAAjE2GxAYw0wDKJUaLPRHqRsEoGAWjYKQCAC8aR5I3lptLAAAAAElFTkSuQmCC","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":true,"prefix":"","firstName":"Aiping","middleName":"","lastName":"Li","suffix":""},{"id":486572155,"identity":"547b0a78-9b3c-45ba-a7bb-2158ed6f5a10","order_by":4,"name":"Weihong Han","email":"","orcid":"","institution":"Pengcheng Laboratory","correspondingAuthor":false,"prefix":"","firstName":"Weihong","middleName":"","lastName":"Han","suffix":""}],"badges":[],"createdAt":"2025-07-02 04:23:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7024974/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7024974/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11280-025-01380-w","type":"published","date":"2025-10-18T15:57:29+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":93956896,"identity":"35ecc5df-c522-4aa7-8143-389c0b0376a7","added_by":"auto","created_at":"2025-10-20 16:12:40","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":585837,"visible":true,"origin":"","legend":"","description":"","filename":"WWWJDPPCL4.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7024974/v1_covered_5615c71d-5ac3-40af-9b19-43329f486243.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"DPP-CL: Orthogonal Subspace Continual Learning for Dialogue Policy Planning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"world-wide-web","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"wwwj","sideBox":"Learn more about [World Wide Web](http://link.springer.com/journal/11280)","snPcode":"11280","submissionUrl":"https://submission.nature.com/new-submission/11280/3","title":"World Wide Web","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Continual Learning, Dialogue Policy Planning, Knowledge Inference, Knowledge Graph","lastPublishedDoi":"10.21203/rs.3.rs-7024974/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7024974/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Continual learning (CL) is critical for large language models (LLMs) to tackle multiple tasks in various downstream applications. However, LLMs struggle to maintain performance in multiple-task training due to catastrophic forgetting, which undermines the learned parameters for old tasks during training for new tasks. Although existing studies typically employ three methods to address this issue, including regularization-based, rehearsal-based, and parameter isolation-based methods, these methods simultaneously lead to new issues, such as privacy concerns, difficulties in handling long sequences. To address existing problems and enhance performance, we propose DPP-CL, a Dialogue Policy Planner (DPP) utilized by CL to facilitate forgetting and improve performance. Specifically, we used gradient descent in an orthogonal subspace to learn new tasks for DPP in multitask learning. Regarding the parameter isolation-based approach that struggles to handle long sequences, we introduce the concatenation of hyperbolic spherical embeddings and Euclidean embeddings to form a novel representation, thereby enhancing the capability of LLMs to understand long-sequence texts. In particular, the DPP training process proceeds sequentially through supervised learning, knowledge distillation, and reinforcement learning. We performed comparison experiments using the standard CL benchmarks and incorporated retrieval-augmented generation into the dialogue architecture for multitask inference in cybersecurity.","manuscriptTitle":"DPP-CL: Orthogonal Subspace Continual Learning for Dialogue Policy Planning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-18 07:39:23","doi":"10.21203/rs.3.rs-7024974/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-09-01T07:49:17+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-27T03:33:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"280071375803995394888220755642238310575","date":"2025-08-11T04:40:58+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"159541181065269482805169917823984010105","date":"2025-08-10T09:02:13+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-04T22:46:45+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"93095156263433149832167312113502594766","date":"2025-07-17T00:20:36+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-13T02:30:50+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-04T06:50:10+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-04T06:46:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"World Wide Web","date":"2025-07-02T04:11:07+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"world-wide-web","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"wwwj","sideBox":"Learn more about [World Wide Web](http://link.springer.com/journal/11280)","snPcode":"11280","submissionUrl":"https://submission.nature.com/new-submission/11280/3","title":"World Wide Web","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"66e124f1-baa2-404f-bd04-ff4ddfe18861","owner":[],"postedDate":"July 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-10-20T16:10:45+00:00","versionOfRecord":{"articleIdentity":"rs-7024974","link":"https://doi.org/10.1007/s11280-025-01380-w","journal":{"identity":"world-wide-web","isVorOnly":false,"title":"World Wide Web"},"publishedOn":"2025-10-18 15:57:29","publishedOnDateReadable":"October 18th, 2025"},"versionCreatedAt":"2025-07-18 07:39:23","video":"","vorDoi":"10.1007/s11280-025-01380-w","vorDoiUrl":"https://doi.org/10.1007/s11280-025-01380-w","workflowStages":[]},"version":"v1","identity":"rs-7024974","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7024974","identity":"rs-7024974","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}