Leveraging key-points Motion History Maps for Human Activity Recognition

Leveraging key-points Motion History Maps for Human Activity Recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Leveraging key-points Motion History Maps for Human Activity Recognition Luca Minutillo, Rosa Altilio, Francesco Chirico, Goffredo Foglia This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5678495/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract There has been a great interest in investigating machine learning techniques for rec-ognizing human activities from video sequences or images. The task is particularlychallenging due to potential problems that could appear related to clutter, partial oc-clusion, changes in scale, viewpoint, lighting, and appearance. Additional, many fieldsof applications could require an activity recognition tool including video surveillancesystems, human-computer interaction, e-health applications, and robotics. In this pa-per, we present a novel two-phase approach for extracting and learning knowledgefrom a subset of possible human actions, working in a multi person scenario. Ourapproach first employs a deep neural network to perform the pose estimation and theskeleton key-points extraction of a subject. Successively, the key points are used tofirstly track each subject in the scene and secondly build the motion history images.These ones represent the inputs of a convolutional neural network able to recognizeand correctly classify seven different possible human actions. Despite every sub-blockof our machine learning pipeline being an off-the-shelf solution by itself, the wholesystem represents a pretty impactful innovation. In fact, it enables fast and efficientreal-time multi-target activity recognition, which is something seldomly investigatedin literature, but of great importance for real world applications. Experimental evalu-ations demonstrate that the solution is competitive with existing approaches, reachingan accuracy near to 90% on a real world-dataset. Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5678495","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":393215583,"identity":"27acf0e4-46ca-4433-b9b1-3bf92377cddc","order_by":0,"name":"Luca Minutillo","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzElEQVRIiWNgGAWjYBACPgglAcSMjQceMNiARQ7g08IG18LG2HAggSGNgQ0odACfHjZkFlDLYbAWvNawSSQ//MCYY5FncL8ZaEvFeTk2+QbGwx/wakkzlmDcJlFscAzksDO3jQk7TCKHAaQlcQNIS2Lb7cQ2IrQw/0DScq6eGC1syLYcSCDsMJ5nZhaJQC0zjyWC/JJs2MYGZJzBo4WfPfnxjY/b6hL7Dh9/+OBDhZ08P/Phwx8q8GgBgwRULmMDIQ2jYBSMglEwCggAAGPJT5aDXzJCAAAAAElFTkSuQmCC","orcid":"","institution":"Elettronica S.P.A.","correspondingAuthor":true,"prefix":"","firstName":"Luca","middleName":"","lastName":"Minutillo","suffix":""},{"id":393215584,"identity":"076363ab-2564-4da5-b167-4c737df94010","order_by":1,"name":"Rosa Altilio","email":"","orcid":"","institution":"Elettronica S.P.A.","correspondingAuthor":false,"prefix":"","firstName":"Rosa","middleName":"","lastName":"Altilio","suffix":""},{"id":393215585,"identity":"0c9bc628-114a-4843-99be-a88ffbfbc95b","order_by":2,"name":"Francesco Chirico","email":"","orcid":"","institution":"Elettronica S.P.A.","correspondingAuthor":false,"prefix":"","firstName":"Francesco","middleName":"","lastName":"Chirico","suffix":""},{"id":393215589,"identity":"e49adc7f-0425-470c-b6e5-4825d668550c","order_by":3,"name":"Goffredo Foglia","email":"","orcid":"","institution":"Elettronica S.P.A.","correspondingAuthor":false,"prefix":"","firstName":"Goffredo","middleName":"","lastName":"Foglia","suffix":""}],"badges":[],"createdAt":"2024-12-19 16:38:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5678495/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5678495/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":72228254,"identity":"3a9a0760-b15f-42e6-9b7d-a82c449d0ca4","added_by":"auto","created_at":"2024-12-24 02:48:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":370557,"visible":true,"origin":"","legend":"","description":"","filename":"LeveragingkeypointsMotionHistoryMapsforHumanActivityRecognition.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5678495/v1_covered_bd7e2b5d-0b5b-43a0-9958-d13813455ab3.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Leveraging key-points Motion History Maps for Human Activity Recognition","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5678495/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5678495/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"There has been a great interest in investigating machine learning techniques for rec-ognizing human activities from video sequences or images. The task is particularlychallenging due to potential problems that could appear related to clutter, partial oc-clusion, changes in scale, viewpoint, lighting, and appearance. Additional, many fieldsof applications could require an activity recognition tool including video surveillancesystems, human-computer interaction, e-health applications, and robotics. In this pa-per, we present a novel two-phase approach for extracting and learning knowledgefrom a subset of possible human actions, working in a multi person scenario. Ourapproach first employs a deep neural network to perform the pose estimation and theskeleton key-points extraction of a subject. Successively, the key points are used tofirstly track each subject in the scene and secondly build the motion history images.These ones represent the inputs of a convolutional neural network able to recognizeand correctly classify seven different possible human actions. Despite every sub-blockof our machine learning pipeline being an off-the-shelf solution by itself, the wholesystem represents a pretty impactful innovation. In fact, it enables fast and efficientreal-time multi-target activity recognition, which is something seldomly investigatedin literature, but of great importance for real world applications. Experimental evalu-ations demonstrate that the solution is competitive with existing approaches, reachingan accuracy near to 90% on a real world-dataset.","manuscriptTitle":"Leveraging key-points Motion History Maps for Human Activity Recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-24 02:40:34","doi":"10.21203/rs.3.rs-5678495/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"591a3c76-85a8-43e0-87b9-81cf3fdf4bfd","owner":[],"postedDate":"December 24th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-12-24T02:40:34+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-24 02:40:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5678495","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5678495","identity":"rs-5678495","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}