A Levitated Control Attention for Named Entity Recognition

A Levitated Control Attention for Named Entity Recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Levitated Control Attention for Named Entity Recognition Rong Huang, YanPing Chen, Ruizhang Huang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3814586/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Controlled attention is a mechanism developed in cognitive neuroscience. It has been successfully applied to support named entity recognition, where the start and end boundaries of a possible named entity are marked by two specific tokens to indicate its position in a sentence. Then, it is fed into a deep network for classification. The entity boundary markers enable a deep neural network to be aware of entity boundaries and to build the contextual dependency of a sentence relevant to a possible named entity. The problem is that every possible named entity should be evaluated independently. This leads to very high computational complexity and cannot construct the semantic dependency between different named entities. In this paper, a levitated control attention mechanism is presented for named entity recognition. The levitated control attention is composed of a separable matrix, which enables the weighting of discriminative contextual features relevant to each specific entity. All possible named entities are together fed into a deep network for one-pass classification, which can establish the semantic dependency between contextual features and possible named entities. The levitated control attention is evaluated on four public datasets. Experiments show that it not only considerably reduces the computational complexity but also improves performance. controlled attention nested named entity neural network cognitive neuroscience Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3814586","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":264180257,"identity":"8e4b1a4e-eaaf-447c-b9e0-2c9d05532bbb","order_by":0,"name":"Rong Huang","email":"","orcid":"","institution":"Guizhou University","correspondingAuthor":false,"prefix":"","firstName":"Rong","middleName":"","lastName":"Huang","suffix":""},{"id":264180258,"identity":"38250c46-6add-4fdb-ae19-c4fe5a4856c6","order_by":1,"name":"YanPing Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAs0lEQVRIiWNgGAWjYLACxgYbENl4gBQtaWCSJC2HwTRxWgxuNx978HPHebu17YeBttTYRBPWcudYumHvmdvJ284kArUcS8ttIKjlRo6ZNGPb7WSzA0AtQBcSoyX/G1DLuWSz8w+J1pLDBtRywM7sBrG2SN45ZibZ25acYHYDaEsCMX7hu938TOJnm5292fn0hw8+1NgQ1qJwA0InglUmEFIOAvIzILQ9MYpHwSgYBaNghAIAYJFLZ6GpNb4AAAAASUVORK5CYII=","orcid":"","institution":"Guizhou University","correspondingAuthor":true,"prefix":"","firstName":"YanPing","middleName":"","lastName":"Chen","suffix":""},{"id":264180259,"identity":"4419f065-2acc-49d3-bd68-d2d7074e8715","order_by":2,"name":"Ruizhang Huang","email":"","orcid":"","institution":"Guizhou University","correspondingAuthor":false,"prefix":"","firstName":"Ruizhang","middleName":"","lastName":"Huang","suffix":""}],"badges":[],"createdAt":"2023-12-28 03:14:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3814586/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3814586/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":50797067,"identity":"dae91bbc-5ff2-490f-87c7-18a03045978a","added_by":"auto","created_at":"2024-02-07 12:26:01","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1395872,"visible":true,"origin":"","legend":"","description":"","filename":"LCA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3814586/v1_covered_c992b719-844a-474a-8f42-7da1b093e803.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Levitated Control Attention for Named Entity Recognition","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"controlled attention, nested named entity, neural network, cognitive neuroscience","lastPublishedDoi":"10.21203/rs.3.rs-3814586/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3814586/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Controlled attention is a mechanism developed in cognitive neuroscience. It has been successfully applied to support named entity recognition, where the start and end boundaries of a possible named entity are marked by two specific tokens to indicate its position in a sentence. Then, it is fed into a deep network for classification. The entity boundary markers enable a deep neural network to be aware of entity boundaries and to build the contextual dependency of a sentence relevant to a possible named entity. The problem is that every possible named entity should be evaluated independently. This leads to very high computational complexity and cannot construct the semantic dependency between different named entities. In this paper, a levitated control attention mechanism is presented for named entity recognition. The levitated control attention is composed of a separable matrix, which enables the weighting of discriminative contextual features relevant to each specific entity. All possible named entities are together fed into a deep network for one-pass classification, which can establish the semantic dependency between contextual features and possible named entities. The levitated control attention is evaluated on four public datasets. Experiments show that it not only considerably reduces the computational complexity but also improves performance.","manuscriptTitle":"A Levitated Control Attention for Named Entity Recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-01 11:50:08","doi":"10.21203/rs.3.rs-3814586/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ce56142e-9824-4d75-b036-1f847f3b7ef9","owner":[],"postedDate":"January 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-02-21T23:19:58+00:00","versionOfRecord":[],"versionCreatedAt":"2024-01-01 11:50:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3814586","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3814586","identity":"rs-3814586","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}