{"paper_id":"2e92a6d9-af38-461e-a31f-c8a6322a300b","body_text":"Research on multimodal fatigue driving detection method based on bidirectional temporal modeling and cross-attention mechanism | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Research on multimodal fatigue driving detection method based on bidirectional temporal modeling and cross-attention mechanism FengTong Wang, ZuoZheng Lian, YunLong Li, Jin Wang, Zheng Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7339213/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Traffic safety issues caused by driver fatigue are becoming increasingly prominent, necessitating an efficient, robust, and deployable fatigue detection assistance system. Traditional geometric feature-based fatigue detection methods typically rely on manually set fixed thresholds and judge fatigue status using single or fused facial geometric features. These methods lack adaptability to individual differences and struggle to ensure stability and accuracy in complex scenarios. While purely image feature-driven methods can capture rich visual details, they often overlook the temporal evolution of keypoints, limiting their ability to model the dynamics of fatigue status. To overcome these challenges, this paper proposes a multimodal feature fusion-based fatigue detection model, MM-DMBICA (MediaPipe MobileNetV3-Dual Modal BiGRU CrossAttention). This model employs a dual-branch architecture: the geometry branch utilizes MediaPipe to extract facial keypoint coordinate sequences and models their temporal dynamics using a bidirectional GRU. The image branch employs MobileNetV3 as a frame-level feature extractor, combined with a BiGRU to capture temporal dependencies between video frames. Furthermore, a bidirectional CrossAttention mechanism, called CrossAttention, is introduced, leveraging a learnable query vector to enhance the interaction between the geometric and image modalities, enabling each modality to focus on the other's important temporal information. Finally, a gated fusion mechanism adaptively integrates the bimodal attention outputs, dynamically balancing the contributions of different features and improving classification robustness. Experiments demonstrate that this model effectively integrates spatial visual details with temporal behavioral patterns, significantly enhancing the ability to discriminate fatigue states in complex environments and providing a highly accurate solution for real-time fatigue monitoring in assisted driving systems. Physical sciences/Engineering Physical sciences/Mathematics and computing Fatigue detection MediaPipe MobileNetV3 CrossAttention BiGRU Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-7339213\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":true,\"archivedVersions\":[],\"articleType\":\"Article\",\"associatedPublications\":[],\"authors\":[{\"id\":524193096,\"identity\":\"331b1c3c-f863-42bd-88ba-dc1ac2aadea2\",\"order_by\":0,\"name\":\"FengTong Wang\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYBAC+/b+hw8+/rGpt29vIFKLAc8ZZsOZDWkJBjwHiNUikcMmzNtwOMFAIoFILeYMuccYeHcczjOXfLzxBkONTTRBLZYN59IeSJ5JL7acnVZswXAsLbeBoJ6DDeYGBmzWjA23c8wkGBsOE6HlMIOZRAIbM2PDzTNEajE4xmMmcbDNOXHDDR4itUj2sCUbNpxJM5bsAfolgRi/8Ms/Pvj4T4WNHD/74Y03PtTYEOEXZEcSHTVIWkjVMQpGwSgYBSMDAABWSUOEJhA4zQAAAABJRU5ErkJggg==\",\"orcid\":\"\",\"institution\":\"Qiqihar University\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"FengTong\",\"middleName\":\"\",\"lastName\":\"Wang\",\"suffix\":\"\"},{\"id\":524193097,\"identity\":\"1369626a-90c4-4cec-a702-d51e5d89716b\",\"order_by\":1,\"name\":\"ZuoZheng Lian\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Qiqihar University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"ZuoZheng\",\"middleName\":\"\",\"lastName\":\"Lian\",\"suffix\":\"\"},{\"id\":524193098,\"identity\":\"3d82edde-09c4-496a-bc45-efbbeb61ed50\",\"order_by\":2,\"name\":\"YunLong Li\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Qiqihar University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"YunLong\",\"middleName\":\"\",\"lastName\":\"Li\",\"suffix\":\"\"},{\"id\":524193099,\"identity\":\"214a12e2-be7f-425f-92dd-b9f9e6061227\",\"order_by\":3,\"name\":\"Jin Wang\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Qiqihar University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Jin\",\"middleName\":\"\",\"lastName\":\"Wang\",\"suffix\":\"\"},{\"id\":524193100,\"identity\":\"468e30f3-7a89-4850-b7ca-d532e70d4a99\",\"order_by\":4,\"name\":\"Zheng Wang\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Qiqihar University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Zheng\",\"middleName\":\"\",\"lastName\":\"Wang\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-08-10 13:53:18\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-7339213/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-7339213/v1\",\"draftVersion\":[],\"editorialEvents\":[],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":92823265,\"identity\":\"d7cd0034-6c17-4842-b727-76a1474b386f\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"docx\",\"order_by\":0,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":809076,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"plV1.2.docx\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/2d607243401ed801690bcf13.docx\"},{\"id\":92823264,\"identity\":\"fe763c91-c554-40f9-9d68-e4e4fcaf3148\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"json\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":7786,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"cdc862e058c546a8815aadc844f6394a.json\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/7d6190a375321ff3008b5877.json\"},{\"id\":92823267,\"identity\":\"d468bf26-de58-419e-9e04-7c989f9208dd\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"xml\",\"order_by\":2,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":79458,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"cdc862e058c546a8815aadc844f6394a1enriched.xml\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/f7dbae1a24f2ddeaf1c43821.xml\"},{\"id\":92823524,\"identity\":\"44576c31-c501-4711-a325-6c90bf1511da\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:26:04\",\"extension\":\"png\",\"order_by\":3,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":423,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage1.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/dc6b940c8afa12cd092b181d.png\"},{\"id\":92823609,\"identity\":\"862bc52c-a02b-4346-b426-3af806ed00f0\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:34:04\",\"extension\":\"png\",\"order_by\":4,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":72739,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage2.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/2e857941f67bb8d3deec5030.png\"},{\"id\":92823268,\"identity\":\"7e53f6d5-cd96-4169-9446-c2da4ef5e8e4\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":5,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":240377,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage3.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/030cc6904347711a295dd38a.png\"},{\"id\":92823266,\"identity\":\"610b4dc4-bffe-49ae-b0bf-5451c8f27236\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":6,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":114602,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage4.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/a7a9c8c53e1a8356f10ca204.png\"},{\"id\":92823271,\"identity\":\"756fff40-af8b-45e6-958d-0c21a1d86c46\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"jpeg\",\"order_by\":7,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":131995,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage5.jpeg\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/3389457800c0797651b7900d.jpeg\"},{\"id\":92823277,\"identity\":\"b57dc7b8-472d-4a2e-b105-f8ca486d51c5\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"jpeg\",\"order_by\":8,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":110651,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage6.jpeg\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/70ca6b2e9e964261804f69b6.jpeg\"},{\"id\":92823276,\"identity\":\"023a1bf9-4831-49bb-b38a-2ceb147e2d80\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"jpeg\",\"order_by\":9,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":194345,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage7.jpeg\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/0c4a7eda23b60cd158b2236b.jpeg\"},{\"id\":92823284,\"identity\":\"87623650-c375-4f03-9c6f-9726204acf07\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"jpeg\",\"order_by\":10,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":2301892,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage8.jpeg\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/1937dda98dde2a8f5b5887b1.jpeg\"},{\"id\":92823526,\"identity\":\"a6d5a5c3-3715-4a1f-8279-0cf925330973\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:26:04\",\"extension\":\"jpeg\",\"order_by\":11,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":4888,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"floatimage9.jpeg\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/25a76a44405b355b8c8f8de7.jpeg\"},{\"id\":92823278,\"identity\":\"27719eff-f56e-4250-9f41-623353621f61\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":12,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":405,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage1.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/a952858d1a8e78fd6c61a0fd.png\"},{\"id\":92823279,\"identity\":\"e5e31100-9ea3-4bf5-8479-482a17d3c08d\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":13,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":43816,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage2.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/a63c7e954db4e1815bb6725b.png\"},{\"id\":92823272,\"identity\":\"5e404f30-3a46-40b1-b77f-c12f35a854a7\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":14,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":41719,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage3.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/8c882d66e325fefb995b65be.png\"},{\"id\":92823282,\"identity\":\"9abb3226-3d66-4bac-8a69-d5640d763b3e\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":15,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":27147,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage4.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/ea367ee57a1282100792ea97.png\"},{\"id\":92823283,\"identity\":\"40012032-6558-4cf2-b36e-7cc4914c74f6\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":16,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":207840,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage5.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/7385ad066cfdbbf33f95c363.png\"},{\"id\":92823610,\"identity\":\"b5e71a3b-cee8-443b-8907-1cc60f224bd2\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:34:04\",\"extension\":\"png\",\"order_by\":17,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":27975,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage6.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/4e061825d5094759f8188326.png\"},{\"id\":92823281,\"identity\":\"5b293e88-85ee-4656-b57b-3db70f0e70af\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":18,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":53042,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage7.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/d3eee8e49ca5d938615a4007.png\"},{\"id\":92823285,\"identity\":\"5ec441c7-c041-4e07-bfd2-e3c3abc9d39b\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":19,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":299999,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage8.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/03b23d65dbf9e7a1d7f9d6d6.png\"},{\"id\":92823274,\"identity\":\"1c6eed5f-4c9b-4797-87f3-ba27f63cdae7\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:18:04\",\"extension\":\"png\",\"order_by\":20,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":5719,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Onlinefloatimage9.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/303e4839616eb61c6bd6a256.png\"},{\"id\":92823528,\"identity\":\"aea3037f-b7a0-4c81-aa27-e330f4f0ceb4\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:26:04\",\"extension\":\"xml\",\"order_by\":21,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":77015,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"cdc862e058c546a8815aadc844f6394a1structuring.xml\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/cec4575bfec9a707f8072c43.xml\"},{\"id\":92823529,\"identity\":\"709a43da-ec9f-4a94-a907-1e3e1b1364e6\",\"added_by\":\"auto\",\"created_at\":\"2025-10-06 03:26:04\",\"extension\":\"html\",\"order_by\":22,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"acdc-reference\",\"size\":86919,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"earlyproof.html\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1/fe798a7a32c5590f9ea4e3ab.html\"},{\"id\":102297807,\"identity\":\"f8b01dea-07e9-453d-a27b-32e0df57cea7\",\"added_by\":\"auto\",\"created_at\":\"2026-02-10 10:29:16\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":809986,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"plV1.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-7339213/v1_covered_195266ac-d2d9-46e4-8413-20d2db850751.pdf\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Research on multimodal fatigue driving detection method based on bidirectional temporal modeling and cross-attention mechanism\",\"fulltext\":[],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":false,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":true,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":false,\"isAuthorSuppliedPdf\":true,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":true,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true},\"keywords\":\"Fatigue detection, MediaPipe, MobileNetV3, CrossAttention, BiGRU\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-7339213/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-7339213/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"\\u003cp\\u003eTraffic safety issues caused by driver fatigue are becoming increasingly prominent, necessitating an efficient, robust, and deployable fatigue detection assistance system. Traditional geometric feature-based fatigue detection methods typically rely on manually set fixed thresholds and judge fatigue status using single or fused facial geometric features. These methods lack adaptability to individual differences and struggle to ensure stability and accuracy in complex scenarios. While purely image feature-driven methods can capture rich visual details, they often overlook the temporal evolution of keypoints, limiting their ability to model the dynamics of fatigue status. To overcome these challenges, this paper proposes a multimodal feature fusion-based fatigue detection model, MM-DMBICA (MediaPipe MobileNetV3-Dual Modal BiGRU CrossAttention). This model employs a dual-branch architecture: the geometry branch utilizes MediaPipe to extract facial keypoint coordinate sequences and models their temporal dynamics using a bidirectional GRU. The image branch employs MobileNetV3 as a frame-level feature extractor, combined with a BiGRU to capture temporal dependencies between video frames. Furthermore, a bidirectional CrossAttention mechanism, called CrossAttention, is introduced, leveraging a learnable query vector to enhance the interaction between the geometric and image modalities, enabling each modality to focus on the other's important temporal information. Finally, a gated fusion mechanism adaptively integrates the bimodal attention outputs, dynamically balancing the contributions of different features and improving classification robustness. Experiments demonstrate that this model effectively integrates spatial visual details with temporal behavioral patterns, significantly enhancing the ability to discriminate fatigue states in complex environments and providing a highly accurate solution for real-time fatigue monitoring in assisted driving systems.\\u003c/p\\u003e\",\"manuscriptTitle\":\"Research on multimodal fatigue driving detection method based on bidirectional temporal modeling and cross-attention mechanism\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-10-06 03:17:59\",\"doi\":\"10.21203/rs.3.rs-7339213/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"1f118f59-fb2d-415e-9c84-8e32cef824f8\",\"owner\":[],\"postedDate\":\"October 6th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"posted\",\"subjectAreas\":[{\"id\":55716925,\"name\":\"Physical sciences/Engineering\"},{\"id\":55716926,\"name\":\"Physical sciences/Mathematics and computing\"}],\"tags\":[],\"updatedAt\":\"2026-02-10T04:40:24+00:00\",\"versionOfRecord\":[],\"versionCreatedAt\":\"2025-10-06 03:17:59\",\"video\":\"\",\"vorDoi\":\"\",\"vorDoiUrl\":\"\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-7339213\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-7339213\",\"identity\":\"rs-7339213\",\"version\":[\"v1\"]},\"buildId\":\"8U1c8b4HqxoKbykW_rLl7\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}