DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer Fangjie Deng, Chao Yang, Hui Guo, Yansong Wang, Lipeng Xu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4546491/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Fatigue driving stands as a primary contributor to severe traffic accidents. Despite notable advancements in fatigue monitoring techniques, current methods frequently overlook the utilization of shifts in drivers' body postures induced by fatigue. These postural changes can offer valuable indicators of drivers' levels of alertness and degree of fatigue. Therefore, a real-time system with the ability to process body information is needed. This study presents a novel framework that integrates facial fatigue characteristics with alterations in drivers' upper body postures to comprehensively monitor fatigue across multiple dimensions. The neural network architecture at the core of the proposed approach utilizes a video vision transformer framework, with the self-attention mechanism optimized to accommodate the magnitude of fatigue-induced posture changes. This optimization ensures high accuracy while simultaneously reducing computational expenses. This gives the approach the potential for embedded deployment. According to requirements, the standard DMD dataset captured in real-world driving scenarios has been selected for validating the proposed method. In terms of accuracy, compared to detection methods that solely rely on facial and upper body poses, the multi-feature fatigue detection approach has achieved an improvement of 3.4% and 11.9%, reaching 96.24%. Additionally, it has been observed that the adoption of the Dynamic Distributed Attention-based backbone has led to a significant increase in inference speed of over 30%. Fatigue driving facial fatigue body posture video vision transformer fatigue monitoring system Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4546491","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":314920195,"identity":"6b89ecce-4f4c-4108-9d9a-3cbe95408e51","order_by":0,"name":"Fangjie Deng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5klEQVRIiWNgGAWjYBACNvmDjY//GEjI8csfPvggoaKGsBY+CebDBjwFFsaSM9iSDR6cOUZYi5wEW5oEz4eKxA03eNQkH7YwE+Ew6R4DCQkDCWOG2z1sFYkNbAz87d0J+LXInDEAAgk5xjlnj91I3CHDIHHm7Ab8WhhyDBISgLYwM+Sl3Ug8w8ZgIJFLWMuBAwYSiW0MOWYFiW3MRGiRSEtsbABq6ZHIMWMgTgvP4cMgZcYSPMeSJRLOHOMh6Bf59sb23wx/6uTsjzcf/PijokaOv70XvxYMwEOa8lEwCkbBKBgFWAEAa8tHrDaFgbgAAAAASUVORK5CYII=","orcid":"","institution":"Shanghai University of Engineering Sciences","correspondingAuthor":true,"prefix":"","firstName":"Fangjie","middleName":"","lastName":"Deng","suffix":""},{"id":314920196,"identity":"0de36a51-8f61-4245-9100-e4ae0518aca5","order_by":1,"name":"Chao Yang","email":"","orcid":"","institution":"Shanghai University of Engineering Sciences","correspondingAuthor":false,"prefix":"","firstName":"Chao","middleName":"","lastName":"Yang","suffix":""},{"id":314920197,"identity":"c09dbb79-bc32-4e63-bcb6-9dc84397a591","order_by":2,"name":"Hui Guo","email":"","orcid":"","institution":"Shanghai University of Engineering Sciences","correspondingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Guo","suffix":""},{"id":314920198,"identity":"127e8160-ab22-454a-96d4-c742e8447eb2","order_by":3,"name":"Yansong Wang","email":"","orcid":"","institution":"Shanghai University of Engineering Sciences","correspondingAuthor":false,"prefix":"","firstName":"Yansong","middleName":"","lastName":"Wang","suffix":""},{"id":314920199,"identity":"8dabe162-b403-4f12-88f0-84f4a5439feb","order_by":4,"name":"Lipeng Xu","email":"","orcid":"","institution":"Shanghai University of Engineering Sciences","correspondingAuthor":false,"prefix":"","firstName":"Lipeng","middleName":"","lastName":"Xu","suffix":""}],"badges":[],"createdAt":"2024-06-07 13:44:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4546491/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4546491/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":69384639,"identity":"c2ddf8a7-1ac3-4658-9b2b-e8263481f0de","added_by":"auto","created_at":"2024-11-19 20:31:42","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":752185,"visible":true,"origin":"","legend":"","description":"","filename":"DAViViTFatigueDetectionFrameworkUsingJointandFacialKeypointFeatureswithDynamicDistributedAttentionVideoVisionTransformer10page.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4546491/v1_covered_37ba78f0-3237-4c76-a97b-58a4ee35ba40.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Fatigue driving, facial fatigue, body posture, video vision transformer, fatigue monitoring system","lastPublishedDoi":"10.21203/rs.3.rs-4546491/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4546491/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Fatigue driving stands as a primary contributor to severe traffic accidents. Despite notable advancements in fatigue monitoring techniques, current methods frequently overlook the utilization of shifts in drivers' body postures induced by fatigue. These postural changes can offer valuable indicators of drivers' levels of alertness and degree of fatigue. Therefore, a real-time system with the ability to process body information is needed. This study presents a novel framework that integrates facial fatigue characteristics with alterations in drivers' upper body postures to comprehensively monitor fatigue across multiple dimensions. The neural network architecture at the core of the proposed approach utilizes a video vision transformer framework, with the self-attention mechanism optimized to accommodate the magnitude of fatigue-induced posture changes. This optimization ensures high accuracy while simultaneously reducing computational expenses. This gives the approach the potential for embedded deployment. According to requirements, the standard DMD dataset captured in real-world driving scenarios has been selected for validating the proposed method. In terms of accuracy, compared to detection methods that solely rely on facial and upper body poses, the multi-feature fatigue detection approach has achieved an improvement of 3.4% and 11.9%, reaching 96.24%. Additionally, it has been observed that the adoption of the Dynamic Distributed Attention-based backbone has led to a significant increase in inference speed of over 30%.","manuscriptTitle":"DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-21 10:48:13","doi":"10.21203/rs.3.rs-4546491/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2d046611-f4a0-492f-b4cc-5464fb6f39a0","owner":[],"postedDate":"June 21st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-11-19T20:23:32+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-21 10:48:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4546491","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4546491","identity":"rs-4546491","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}