A Novel Capsule Network with Attention Routing for Text Classification

A Novel Capsule Network with Attention Routing for Text Classification | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Novel Capsule Network with Attention Routing for Text Classification Weisheng Zhang, Shengfa Miao, Qian Yu, Jian Wang, Huibo Li, Ruoshu Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4021532/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Convolutional Neural Networks(CNNs) and Recurrent Neural Networks (RNNs) often neglect the relationship between local and global semantics in text. In contrast, capsule networks encode word position information and multi-level semantic information using vector capsules and capture the relationship between local and global semantics through dynamic routing. However, capsule networks commonly neglect contextual information during capsule generation. Moreover, complex dynamic routing in capsule networks results in significant computational cost during training and evaluation. Therefore, we introduce AARCapsNet, a novel capsule network with attention routing for text classification. AARCapsNet incorporates two well-designed routings: self-attention routing and fast attention routing. Self-attention routing encodes contextual information into semantic capsules while suppressing noisy capsules. Fast attention routing adaptively learns the connection relationship between semantic capsules and class capsules, which offers a cost-effective alternative to intricate dynamic routing. Experiments on five benchmark datasets demonstrate that our proposed method achieves competitive performance. Text classification Capsule network Routing algorithm Attention mechanism. Full Text Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4021532","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":340792312,"identity":"01ba2b3c-3e48-4a2e-b5cd-45861c842466","order_by":0,"name":"Weisheng Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAApElEQVRIiWNgGAWjYDACZgaGAwwMNjz8/A2kaUmTkZxxgDS7DtsYNCQQqdbgOPvDAx9zzvMYMBxg/PAxhxgth3kMDs7cdpvHnLmBWXLmNiK0mB3mYTjMC9Ri2XCAjZmXOC3sD4BazvEYHEggWgvQbbzbDpCgxR7il2QeyRkHm4nzi2T/8ccfPm6zs+fnbz4IZBChBQkwNpCmfhSMglEwCkYBbgAAU543cxkuUQsAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0009-9368-247X","institution":"Yunnan University","correspondingAuthor":true,"prefix":"","firstName":"Weisheng","middleName":"","lastName":"Zhang","suffix":""},{"id":340792313,"identity":"aa138c99-e7be-4814-a5ad-4ecb11c30762","order_by":1,"name":"Shengfa Miao","email":"","orcid":"https://orcid.org/0000-0003-1210-1135","institution":"Yunnan University","correspondingAuthor":false,"prefix":"","firstName":"Shengfa","middleName":"","lastName":"Miao","suffix":""},{"id":340792314,"identity":"9f5a194f-5a2f-4783-9ad4-657394a6c416","order_by":2,"name":"Qian Yu","email":"","orcid":"","institution":"Yunnan University","correspondingAuthor":false,"prefix":"","firstName":"Qian","middleName":"","lastName":"Yu","suffix":""},{"id":340792315,"identity":"866fbcd8-7c4b-47e4-b8c7-5cb045b966b9","order_by":3,"name":"Jian Wang","email":"","orcid":"","institution":"Fengtu Technology(Shenzhen) Co., Ltd.","correspondingAuthor":false,"prefix":"","firstName":"Jian","middleName":"","lastName":"Wang","suffix":""},{"id":340792316,"identity":"9d9d2c9f-dec5-4d56-846b-d292aaa3fa75","order_by":4,"name":"Huibo Li","email":"","orcid":"","institution":"National Engineering Research Center for Risk Perception and Prevention","correspondingAuthor":false,"prefix":"","firstName":"Huibo","middleName":"","lastName":"Li","suffix":""},{"id":340792317,"identity":"beb38f02-85f0-4ee1-8e52-f1ebc9907095","order_by":5,"name":"Ruoshu Wang","email":"","orcid":"","institution":"Yunnan University","correspondingAuthor":false,"prefix":"","firstName":"Ruoshu","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2024-03-06 15:18:52","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4021532/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4021532/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":66508795,"identity":"4a331a60-8e01-4455-b373-8765284ba330","added_by":"auto","created_at":"2024-10-13 20:08:24","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":643091,"visible":true,"origin":"","legend":"","description":"","filename":"SCAARCapsuleNet.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4021532/v1_covered_6f9518a3-9694-4121-b0e4-9b212eacca5f.pdf"}],"financialInterests":"","formattedTitle":"A Novel Capsule Network with Attention Routing for Text Classification","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Text classification, Capsule network, Routing algorithm, Attention mechanism.","lastPublishedDoi":"10.21203/rs.3.rs-4021532/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4021532/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eConvolutional Neural Networks(CNNs) and Recurrent Neural Networks (RNNs) often neglect the relationship between local and global semantics in text. In contrast, capsule networks encode word position information and multi-level semantic information using vector capsules and capture the relationship between local and global semantics through dynamic routing. However, capsule networks commonly neglect contextual information during capsule generation. Moreover, complex dynamic routing in capsule networks results in significant computational cost during training and evaluation. Therefore, we introduce AARCapsNet, a novel capsule network with attention routing for text classification. AARCapsNet incorporates two well-designed routings: self-attention routing and fast attention routing. Self-attention routing encodes contextual information into semantic capsules while suppressing noisy capsules. Fast attention routing adaptively learns the connection relationship between semantic capsules and class capsules, which offers a cost-effective alternative to intricate dynamic routing. Experiments on five benchmark datasets demonstrate that our proposed method achieves competitive performance.\u003c/p\u003e","manuscriptTitle":"A Novel Capsule Network with Attention Routing for Text Classification","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-09-12 10:50:47","doi":"10.21203/rs.3.rs-4021532/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"545ae36e-5429-4b8e-a399-7a3b2b5ebd0e","owner":[],"postedDate":"September 12th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-10-13T20:00:17+00:00","versionOfRecord":[],"versionCreatedAt":"2024-09-12 10:50:47","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4021532","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4021532","identity":"rs-4021532","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}