MCANet: A lightweight action recognition network with multidimensional convolution and attention

MCANet: A lightweight action recognition network with multidimensional convolution and attention | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article MCANet: A lightweight action recognition network with multidimensional convolution and attention Qiuhong Tian, Weilun Miao, Lizao Zhang, Ziyu Yang, Yu Yang, Yanying Zhao, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4596829/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 15 Nov, 2024 Read the published version in International Journal of Machine Learning and Cybernetics → Version 1 posted 7 You are reading this latest preprint version Abstract The majority of lightweight networks currently employed in action recognition tasks are based on convolutional neural networks. The spatial inductive biases of convolutional neural networks, which enables them to complete action recognition tasks with fewer parameters and faster reasoning speeds. Although convolutional neural networks are highly effective at extracting local spatiotemporal features, they lack the global spatiotemporal modelling capabilities of transformer-based networks. However, the transformer-based network has a considerable number of model parameters and a relatively slow inference time, which makes it incompatible with the lightweight requirements. In order to meet the requirements of both lightweight design and high recognition accuracy, this paper proposes MCANet, an action recognition network that is suitable for deployment on lightweight devices. MCANet integrates the strengths of convolutional neural networks and transformer-based networks. Furthermore, the network maintains a low number of model parameters and a fast reasoning speed, while also exhibiting local and global spatiotemporal modelling capabilities, thus achieving high recognition accuracy. The efficacy of the proposed methodology was validated on a diverse array of datasets, including Kinetics400, UCF101, and HMDB51, demonstrate the effectiveness of the proposed method. Action recognition Convolutional neural network Self-attention Lightweight network Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 15 Nov, 2024 Read the published version in International Journal of Machine Learning and Cybernetics → Version 1 posted Reviews received at journal 01 Jul, 2024 Reviewers agreed at journal 22 Jun, 2024 Reviewers agreed at journal 21 Jun, 2024 Reviewers invited by journal 21 Jun, 2024 Editor assigned by journal 21 Jun, 2024 Submission checks completed at journal 18 Jun, 2024 First submitted to journal 17 Jun, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4596829","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":321371943,"identity":"863b0085-a769-4312-a62f-9a60a7da1011","order_by":0,"name":"Qiuhong Tian","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA70lEQVRIiWNgGAWjYDACZhBhwJAAJBkfQMUM8OrgQdLCbABVTUALlAZpYZMgSos9O/OzB28KDufxS7dfq/zx509iA3vzNgmGmjt4HMZmbjjH4HCx5JwzZTck2wwSG3iOlUkwHHuGzy9m0jwGhxM33MhJu2HYANQikWMmwdhwGI8W9m9wLQUJf4Ba5N8Q0sIDsyX9GMMBNpAtPAS0HOYpk5xjkJ44c0YOs2Rjm7FxG09asUXCMdxa2PuPb5N488c6sV8i/eHHH3/kZPvZD2+88aEGtxaIVRASEh1sICIBvwaYFvYHhNSNglEwCkbBCAUAdGNQOwR+wQQAAAAASUVORK5CYII=","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":true,"prefix":"","firstName":"Qiuhong","middleName":"","lastName":"Tian","suffix":""},{"id":321371944,"identity":"d5cd26c5-5716-4b7f-be78-41aa74b5233b","order_by":1,"name":"Weilun Miao","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Weilun","middleName":"","lastName":"Miao","suffix":""},{"id":321371945,"identity":"1c23202d-b5f2-4b40-97ac-7e2677a471ba","order_by":2,"name":"Lizao Zhang","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Lizao","middleName":"","lastName":"Zhang","suffix":""},{"id":321371946,"identity":"99516333-8a99-48fc-b64a-a4a069bed317","order_by":3,"name":"Ziyu Yang","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Ziyu","middleName":"","lastName":"Yang","suffix":""},{"id":321371947,"identity":"07b60ef1-80f6-4362-acc7-5d38adc22cd4","order_by":4,"name":"Yu Yang","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Yu","middleName":"","lastName":"Yang","suffix":""},{"id":321371948,"identity":"be7c6627-74b9-4703-9906-d97f5cdde801","order_by":5,"name":"Yanying Zhao","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Yanying","middleName":"","lastName":"Zhao","suffix":""},{"id":321371949,"identity":"71015a4a-af24-4ec5-8573-c623a091c543","order_by":6,"name":"Lan Yao","email":"","orcid":"","institution":"Zhejiang Sci-Tech University","correspondingAuthor":false,"prefix":"","firstName":"Lan","middleName":"","lastName":"Yao","suffix":""}],"badges":[],"createdAt":"2024-06-18 02:53:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4596829/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4596829/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s13042-024-02454-3","type":"published","date":"2024-11-15T15:57:41+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":69284846,"identity":"d2eb76e1-bff6-4f1b-87e6-ca8575b97e13","added_by":"auto","created_at":"2024-11-18 19:23:07","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2468944,"visible":true,"origin":"","legend":"","description":"","filename":"xlw3.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4596829/v1_covered_40f41ef6-8b3d-4fc0-9e2b-2df79e3e832c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"MCANet: A lightweight action recognition network with multidimensional convolution and attention","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-machine-learning-and-cybernetics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jmlc","sideBox":"Learn more about [International Journal of Machine Learning and Cybernetics](http://actavetscand.biomedcentral.com/)","snPcode":"13042","submissionUrl":"https://submission.nature.com/new-submission/13042/3","title":"International Journal of Machine Learning and Cybernetics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Action recognition, Convolutional neural network, Self-attention, Lightweight network","lastPublishedDoi":"10.21203/rs.3.rs-4596829/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4596829/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The majority of lightweight networks currently employed in action recognition tasks are based on convolutional neural networks. The spatial inductive biases of convolutional neural networks, which enables them to complete action recognition tasks with fewer parameters and faster reasoning speeds. Although convolutional neural networks are highly effective at extracting local spatiotemporal features, they lack the global spatiotemporal modelling capabilities of transformer-based networks. However, the transformer-based network has a considerable number of model parameters and a relatively slow inference time, which makes it incompatible with the lightweight requirements. In order to meet the requirements of both lightweight design and high recognition accuracy, this paper proposes MCANet, an action recognition network that is suitable for deployment on lightweight devices. MCANet integrates the strengths of convolutional neural networks and transformer-based networks. Furthermore, the network maintains a low number of model parameters and a fast reasoning speed, while also exhibiting local and global spatiotemporal modelling capabilities, thus achieving high recognition accuracy. The efficacy of the proposed methodology was validated on a diverse array of datasets, including Kinetics400, UCF101, and HMDB51, demonstrate the effectiveness of the proposed method.","manuscriptTitle":"MCANet: A lightweight action recognition network with multidimensional convolution and attention","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-03 15:53:55","doi":"10.21203/rs.3.rs-4596829/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2024-07-01T18:48:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"196845667362116825248462711678829470479","date":"2024-06-22T05:58:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"178363564510535583974995830068978067184","date":"2024-06-21T13:40:21+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-06-21T13:18:58+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-06-21T13:09:14+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-06-18T06:01:50+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Machine Learning and Cybernetics","date":"2024-06-18T02:51:05+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-machine-learning-and-cybernetics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jmlc","sideBox":"Learn more about [International Journal of Machine Learning and Cybernetics](http://actavetscand.biomedcentral.com/)","snPcode":"13042","submissionUrl":"https://submission.nature.com/new-submission/13042/3","title":"International Journal of Machine Learning and Cybernetics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"d1a0214d-c140-44a5-bfce-a1c298a2214f","owner":[],"postedDate":"July 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-11-18T19:15:10+00:00","versionOfRecord":{"articleIdentity":"rs-4596829","link":"https://doi.org/10.1007/s13042-024-02454-3","journal":{"identity":"international-journal-of-machine-learning-and-cybernetics","isVorOnly":false,"title":"International Journal of Machine Learning and Cybernetics"},"publishedOn":"2024-11-15 15:57:41","publishedOnDateReadable":"November 15th, 2024"},"versionCreatedAt":"2024-07-03 15:53:55","video":"","vorDoi":"10.1007/s13042-024-02454-3","vorDoiUrl":"https://doi.org/10.1007/s13042-024-02454-3","workflowStages":[]},"version":"v1","identity":"rs-4596829","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4596829","identity":"rs-4596829","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}