Integrating deep CNN models for multilingual Sign Language recognition: A SignLink-based approach for Bengali and English

Integrating deep CNN models for multilingual Sign Language recognition: A SignLink-based approach for Bengali and English | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Integrating deep CNN models for multilingual Sign Language recognition: A SignLink-based approach for Bengali and English Niamul Hassan Samin, Mustahidul Islam Ferdous, Renu Akter Suity, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7208431/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Sign language serves as a vital medium of communication for individuals with hearing or speech impairments. While considerable research has focused on the recognition of widely used sign languages such as American Sign Language (ASL) and British Sign Language (BSL), Bengali Sign Language (BdSL) remains significantly underexplored. To address this gap, a bilingual Bengali-English sign language recognition system is proposed, leveraging deep learning techniques for enhanced multilingual gesture interpretation. Two publicly available datasets were combined—one comprising English letters (A–Z), digits (0–9), and a space symbol, and the other consisting of 38 Bengali alphabet gestures. Following dataset merging and augmentation to balance class distributions, the final dataset contained 75 classes and a total of 112,493 images. Eight deep learning models were evaluated, including six pre-trained architectures and two custom-designed networks. Among these, AlexNet demonstrated the highest standalone test accuracy of 96.99%. To further improve performance and mitigate overfitting, a hybrid model named SignLink was developed by integrating AlexNet with MobileNetV2 and Xception—two models exhibiting strong generalization. The hybrid system achieved a test accuracy of 98.93%, outperforming all individual architectures. The proposed system demonstrates robust bilingual sign language recognition capabilities, contributing to inclusive and accessible communication technologies in linguistically diverse and low-resource settings. Artificial Intelligence and Machine Learning Bilingual Sign Language Recognition Deep Learning Hand Gesture Classification Convolutional Neural Network (CNN) Computer Vision Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7208431","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":490506022,"identity":"c716c4d6-8ccb-4e27-b2cc-f891923e8bd9","order_by":0,"name":"Niamul Hassan Samin","email":"","orcid":"","institution":"Bangladesh University of Business and Technology","correspondingAuthor":false,"prefix":"","firstName":"Niamul","middleName":"Hassan","lastName":"Samin","suffix":""},{"id":490506023,"identity":"e936ed29-c428-4683-ba99-98d577c9a9cb","order_by":1,"name":"Mustahidul Islam Ferdous","email":"","orcid":"","institution":"Bangladesh University of Business and Technology","correspondingAuthor":false,"prefix":"","firstName":"Mustahidul","middleName":"Islam","lastName":"Ferdous","suffix":""},{"id":490506024,"identity":"dcca6a32-371e-448b-b197-8b7f29fa3394","order_by":2,"name":"Renu Akter Suity","email":"","orcid":"","institution":"Bangladesh University of Business and Technology","correspondingAuthor":false,"prefix":"","firstName":"Renu","middleName":"Akter","lastName":"Suity","suffix":""},{"id":490506025,"identity":"5c504d17-8d4a-4a5c-b74f-e22d4996bca9","order_by":3,"name":"Md. Fakrul Islam Rafsan","email":"","orcid":"","institution":"Bangladesh University of Business and Technology","correspondingAuthor":false,"prefix":"","firstName":"Md.","middleName":"Fakrul Islam","lastName":"Rafsan","suffix":""},{"id":490506026,"identity":"33b773ec-3557-4285-83c8-fca47d9d6a33","order_by":4,"name":"Md Naimur Rahman","email":"","orcid":"","institution":"Bangladesh University of Business and Technology","correspondingAuthor":false,"prefix":"","firstName":"Md","middleName":"Naimur","lastName":"Rahman","suffix":""},{"id":490506027,"identity":"89b93b2e-6299-4c1b-b919-4ad4d46bb57c","order_by":5,"name":"Al Bashir","email":"","orcid":"","institution":"Texas A\u0026M University","correspondingAuthor":false,"prefix":"","firstName":"Al","middleName":"","lastName":"Bashir","suffix":""},{"id":490506028,"identity":"532a6482-5256-47ac-bc7c-c63437fb2024","order_by":6,"name":"Md Mahbubur Rahman","email":"","orcid":"","institution":"University of Wyoming, Laramie","correspondingAuthor":false,"prefix":"","firstName":"Md","middleName":"Mahbubur","lastName":"Rahman","suffix":""},{"id":490506029,"identity":"8a247488-a3c4-43fe-8a85-5d5990b6365d","order_by":7,"name":"Ali Torabi","email":"","orcid":"","institution":"University of Wyoming, Laramie","correspondingAuthor":false,"prefix":"","firstName":"Ali","middleName":"","lastName":"Torabi","suffix":""},{"id":490506030,"identity":"a2190dcf-ffe1-4717-8075-8ba610a1ffcd","order_by":8,"name":"Sanjog Gaihre","email":"","orcid":"","institution":"University of Wyoming, Laramie","correspondingAuthor":false,"prefix":"","firstName":"Sanjog","middleName":"","lastName":"Gaihre","suffix":""},{"id":490506031,"identity":"7fd36043-6e8c-4ec4-a0dc-df15601c3085","order_by":9,"name":"Yaqoob Majeed","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxElEQVRIiWNgGAWjYJACAwYGZjkom5l4LcakaQGpTGwgWovBtcMPCj5UWKdvuJFj+IChwhqmF4+W22kGhjPOpOcCtRgbMJxJJ0ZLgoExb9vh3A23c8wkGNsOE6Ml/YPx33+H0w3AWv4RpSXHwJix4XACREsDEVokb+cUGPYcSzecef9ZsUHCsXRjglr4bqdvM/hRYy3Pd+bwxgcfaqxlCWpROMDAZgBhchgwJBBSDgLyDQzMDyBM9gfEaBgFo2AUjIIRCACkxUQaVwFSZQAAAABJRU5ErkJggg==","orcid":"","institution":"University of Wyoming, Laramie","correspondingAuthor":true,"prefix":"","firstName":"Yaqoob","middleName":"","lastName":"Majeed","suffix":""}],"badges":[],"createdAt":"2025-07-24 19:45:44","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7208431/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7208431/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":87683944,"identity":"7f13f1a7-7b6c-48d5-8564-25e6315237e6","added_by":"auto","created_at":"2025-07-28 01:29:41","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5360286,"visible":true,"origin":"","legend":"","description":"","filename":"MultilingualSignLanguagerecognitionUpdated240725.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7208431/v1_covered_8bed6799-6076-4974-8e89-01a977593b0e.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eIntegrating deep CNN models for multilingual Sign Language recognition: A SignLink-based approach for Bengali and English\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Wyoming","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Bilingual Sign Language Recognition, Deep Learning, Hand Gesture Classification, Convolutional Neural Network (CNN), Computer Vision","lastPublishedDoi":"10.21203/rs.3.rs-7208431/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7208431/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eSign language serves as a vital medium of communication for individuals with hearing or speech impairments. While considerable research has focused on the recognition of widely used sign languages such as American Sign Language (ASL) and British Sign Language (BSL), Bengali Sign Language (BdSL) remains significantly underexplored. To address this gap, a bilingual Bengali-English sign language recognition system is proposed, leveraging deep learning techniques for enhanced multilingual gesture interpretation. Two publicly available datasets were combined—one comprising English letters (A–Z), digits (0–9), and a space symbol, and the other consisting of 38 Bengali alphabet gestures. Following dataset merging and augmentation to balance class distributions, the final dataset contained 75 classes and a total of 112,493 images. Eight deep learning models were evaluated, including six pre-trained architectures and two custom-designed networks. Among these, AlexNet demonstrated the highest standalone test accuracy of 96.99%. To further improve performance and mitigate overfitting, a hybrid model named SignLink was developed by integrating AlexNet with MobileNetV2 and Xception—two models exhibiting strong generalization. The hybrid system achieved a test accuracy of 98.93%, outperforming all individual architectures. The proposed system demonstrates robust bilingual sign language recognition capabilities, contributing to inclusive and accessible communication technologies in linguistically diverse and low-resource settings.\u003c/p\u003e","manuscriptTitle":"Integrating deep CNN models for multilingual Sign Language recognition: A SignLink-based approach for Bengali and English","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-28 01:21:27","doi":"10.21203/rs.3.rs-7208431/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e84c4f42-9126-4310-9c67-13c56db45751","owner":[],"postedDate":"July 28th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":52083325,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-07-28T01:21:27+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-28 01:21:27","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7208431","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7208431","identity":"rs-7208431","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}