An Improved Facial Emotion Recognition System Using Convolutional Neural Network for the Optimization of Human Robot Interaction

An Improved Facial Emotion Recognition System Using Convolutional Neural Network for the Optimization of Human Robot Interaction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An Improved Facial Emotion Recognition System Using Convolutional Neural Network for the Optimization of Human Robot Interaction Ravi Raj, Ilker Demirkol This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6890226/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Artificial intelligence (AI) has been effectively augmenting the features of robotics applications, including surveillance, medical support, aid services for the elderly or disabled, and many more uses. Most robotics applications need a variety of human-robot interactions (HRI) for effective and accurate use. Computer vision plays a vital role in improved and precise HRI. The human facial emotion recognition (FER) technique is one of the important types of computer vision that is crucial for the enhancement of HRI. This paper presents a study that uses algorithms based on computer vision and machine learning (ML) to identify the emotional states of humans in photographs and videos while users interact with various visual objects. Using ML techniques and a digital image processing pipeline, this study aims to reveal the development of software that can recognize emotions in human gestures. This research shows that software creation and training based on emotional expressions, together with the use of convolutional neural networks (CNNs) for emotional identification, are feasible. The main objective of this study is to find the essential facial gestures that the CNN framework with FER emphasizes. Finally, the proposed model is comparatively analyzed against FER2013, the Real-world Affective Faces Database (RAF-DB), and CK+ datasets. The rate of accuracy is highest for the CK+ dataset, which is around 95% and lowest for the FER2013 dataset, which is around 64%. The findings of this research will advance our knowledge about neural networks and help enhance the efficiency of computer vision. Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6890226","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":471206914,"identity":"3ad91ed0-7e7d-4473-b8b3-71b2cb6f95dc","order_by":0,"name":"Ravi Raj","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuklEQVRIiWNgGAWjYJCCA0DMzA9hM5OgRbKBFC1gYHCAWC3m7WcMDzDusWM3vpF78ANDhXViAyEtMmdygOY/S2Y2u5GXLMFwJp2wFgmGtIQDDAeYgVpyzBgY2w4ToYX/GUhLPbPxDJCWf8RokUg+ANRymNlAAqSlgSgtjw8cSDhwnFnizLtkiYRj6cZEOCyx+cOHA9XJ/O3AEPtQYy1LUAsYJDAwJDMw8IAZxAM7sJZRMApGwSgYBdgAANhHO2c/3AcjAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0001-8073-1812","institution":"Department of Mining, Industrial and ICT Engineering (EMIT), Manresa School of Engineering (EPSEM), Universitat Politecnica de Catalunya (UPC), BarcelonaTech, 08242 Manresa, Spain","correspondingAuthor":true,"prefix":"","firstName":"Ravi","middleName":"","lastName":"Raj","suffix":""},{"id":471206915,"identity":"4988b84b-260b-4683-b4a1-5e9843e9e7aa","order_by":1,"name":"Ilker Demirkol","email":"","orcid":"https://orcid.org/0000-0002-8026-5337","institution":"Department of Mining, Industrial and ICT Engineering (EMIT), Manresa School of Engineering (EPSEM), Universitat Politecnica de Catalunya (UPC), BarcelonaTech, 08242 Manresa, Spain","correspondingAuthor":false,"prefix":"","firstName":"Ilker","middleName":"","lastName":"Demirkol","suffix":""}],"badges":[],"createdAt":"2025-06-13 18:13:36","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6890226/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6890226/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":84939601,"identity":"3aa59974-a761-48dc-86a8-063f64bf8419","added_by":"auto","created_at":"2025-06-19 04:44:14","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":943845,"visible":true,"origin":"","legend":"","description":"","filename":"tjjxcbmskrrqzkbmjndmppzcqrwyvxwc.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6890226/v1_covered_19e44b6e-5119-4d3a-9ad6-bf67bcf220c4.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eAn Improved Facial Emotion Recognition System Using Convolutional Neural Network for the Optimization of Human Robot Interaction\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Universitat Politècnica de Catalunya","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6890226/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6890226/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eArtificial intelligence (AI) has been effectively augmenting the features of robotics applications, including surveillance, medical support, aid services for the elderly or disabled, and many more uses. Most robotics applications need a variety of human-robot interactions (HRI) for effective and accurate use. Computer vision plays a vital role in improved and precise HRI. The human facial emotion recognition (FER) technique is one of the important types of computer vision that is crucial for the enhancement of HRI. This paper presents a study that uses algorithms based on computer vision and machine learning (ML) to identify the emotional states of humans in photographs and videos while users interact with various visual objects. Using ML techniques and a digital image processing pipeline, this study aims to reveal the development of software that can recognize emotions in human gestures. This research shows that software creation and training based on emotional expressions, together with the use of convolutional neural networks (CNNs) for emotional identification, are feasible. The main objective of this study is to find the essential facial gestures that the CNN framework with FER emphasizes. Finally, the proposed model is comparatively analyzed against FER2013, the Real-world Affective Faces Database (RAF-DB), and CK+ datasets. The rate of accuracy is highest for the CK+ dataset, which is around 95% and lowest for the FER2013 dataset, which is around 64%. The findings of this research will advance our knowledge about neural networks and help enhance the efficiency of computer vision.\u003c/p\u003e","manuscriptTitle":"An Improved Facial Emotion Recognition System Using Convolutional Neural Network for the Optimization of Human Robot Interaction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-19 04:36:06","doi":"10.21203/rs.3.rs-6890226/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c7010ede-023b-40b6-8a4b-169d774fb0c8","owner":[],"postedDate":"June 19th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-06-19T04:36:06+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-19 04:36:06","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6890226","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6890226","identity":"rs-6890226","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}