An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments

An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments Mahira Ali, Poma Panezai, Mahmood ul Hassan, Khalid Mahmood, Abdul Qadeer, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7441997/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract When it comes to education today, online learning tools significantly impact how students learn. Understanding and evaluating students' behavior can get them much more involved and help them learn more. This paper presents a real-time emotion detection system that can classify emotions into three main categories: engagement, confusion, and boredom. The system utilizes the DISFA dataset, which consists of video recordings of 26 subjects (each four minutes long) annotated with 12 Action Units (AUs). The data set is then preprocessed and converted into frames using FFmpeg, from which unique frames are extracted using DupeGuru. A convolutional neural network (CNN) model, created with Keras and TensorFlow, is trained to detect these emotions accurately, with a classification accuracy of 92.92% , surpassing the base model. GAN-based data augmentation was also applied to generate additional bored emotion samples, enhancing the dataset and boosting CNN accuracy to 93.46%. The system has an intuitive web interface designed with Django and integrates the Agora SDK to facilitate real-time video conferencing in virtual classes. The proposed system also offers real-time feedback on student emotions, enabling teachers to obtain important information to adapt and deliver a more effective, responsive education. The running source code is available at: https://github.com/mahiraaly/Real-Time-Emotion-Detection-Using-CNN . machine learning deep learning emotion detection online learning CNN GAN Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 13 Nov, 2025 Reviewers invited by journal 12 Nov, 2025 Editor assigned by journal 26 Aug, 2025 Submission checks completed at journal 26 Aug, 2025 First submitted to journal 23 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7441997","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":549299036,"identity":"f44c8687-0b1a-48be-a5f9-fed73600fc44","order_by":0,"name":"Mahira Ali","email":"","orcid":"","institution":"BUITEMS University,Baluchistan, Pakistan.","correspondingAuthor":false,"prefix":"","firstName":"Mahira","middleName":"","lastName":"Ali","suffix":""},{"id":549299037,"identity":"abf96b07-ea4e-44fd-9495-d49c09f183d1","order_by":1,"name":"Poma Panezai","email":"","orcid":"","institution":"BUITEMS University,Baluchistan, Pakistan.","correspondingAuthor":false,"prefix":"","firstName":"Poma","middleName":"","lastName":"Panezai","suffix":""},{"id":549299038,"identity":"5eec65e5-9a71-413a-a773-eefca3ae06f7","order_by":2,"name":"Mahmood ul Hassan","email":"","orcid":"","institution":"IIC University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Mahmood","middleName":"ul","lastName":"Hassan","suffix":""},{"id":549299041,"identity":"1755fa63-41f2-40ba-b56b-3c009ae384d5","order_by":3,"name":"Khalid Mahmood","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxklEQVRIiWNgGAWjYBACCTBZkWCA4BCn5QzJWhjbSNEi2X788evKeWnGBgeYD97mYaiTJ6hFmifHzPLsthwzgwNsydY8DIcNGwhpkWPIYTNs3FZhY3CAx0yah+EAI2Et/M+fGTbOAWnh/wbUUmdPUIu0RILxw8YGkMN42IBamBMJapGc8caMseFYmrHkYTZjyzkGh5MJapE4n/74Y0NNsmHf8eaHN95U1NkS1AIEbJDoYAYRBkSoB6n9QJy6UTAKRsEoGLEAAOuPN+9suiOmAAAAAElFTkSuQmCC","orcid":"","institution":"King Khalid University, Engineering and Technical Specializations Unit Applied College Muhayil Aseer, Kingdom of Saudi Arabia.","correspondingAuthor":true,"prefix":"","firstName":"Khalid","middleName":"","lastName":"Mahmood","suffix":""},{"id":549299042,"identity":"e946f1fe-a264-4a28-a178-ac38370a5a07","order_by":4,"name":"Abdul Qadeer","email":"","orcid":"","institution":"Balochistan University of Engineering and Technology, Khuzdar","correspondingAuthor":false,"prefix":"","firstName":"Abdul","middleName":"","lastName":"Qadeer","suffix":""},{"id":549299043,"identity":"5aab07ae-874a-4409-874e-dfd9123e6ceb","order_by":5,"name":"Muhammad Wasim Javed","email":"","orcid":"","institution":"King Khalid University, Engineering and Technical Specializations Unit Applied College Muhayil Aseer, Kingdom of Saudi Arabia.","correspondingAuthor":false,"prefix":"","firstName":"Muhammad","middleName":"Wasim","lastName":"Javed","suffix":""}],"badges":[],"createdAt":"2025-08-23 14:53:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7441997/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7441997/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":96671088,"identity":"8a57a473-4dea-4f8b-9014-8302c27afe5b","added_by":"auto","created_at":"2025-11-24 23:11:20","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8412,"visible":true,"origin":"","legend":"","description":"","filename":"9dec88e815854e4b98aa6998ef2493d2.json","url":"https://assets-eu.researchsquare.com/files/rs-7441997/v1/eddc03374b44aba5da8a6988.json"},{"id":96710659,"identity":"14904178-d19c-43da-8d50-ca066143cfff","added_by":"auto","created_at":"2025-11-25 10:11:02","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":828478,"visible":true,"origin":"","legend":"","description":"","filename":"GANbasedemotiondetection.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7441997/v1_covered_ac1674cb-8978-405b-8f5f-bcd045963e05.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"user-modeling-and-user-adapted-interaction","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"umui","sideBox":"Learn more about [User Modeling and User-Adapted Interaction](http://link.springer.com/journal/11257)","snPcode":"11257","submissionUrl":"https://submission.nature.com/new-submission/11257/3","title":"User Modeling and User-Adapted Interaction","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"machine learning, deep learning, emotion detection, online learning, CNN, GAN","lastPublishedDoi":"10.21203/rs.3.rs-7441997/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7441997/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"When it comes to education today, online learning tools significantly impact how students learn. Understanding and evaluating students' behavior can get them much more involved and help them learn more. This paper presents a real-time emotion detection system that can classify emotions into three main categories: engagement, confusion, and boredom. The system utilizes the DISFA dataset, which consists of video recordings of 26 subjects (each four minutes long) annotated with 12 Action Units (AUs). The data set is then preprocessed and converted into frames using FFmpeg, from which unique frames are extracted using DupeGuru. A convolutional neural network (CNN) model, created with Keras and TensorFlow, is trained to detect these emotions accurately, with a classification accuracy of 92.92\\% , surpassing the base model. GAN-based data augmentation was also applied to generate additional bored emotion samples, enhancing the dataset and boosting CNN accuracy to 93.46\\%. The system has an intuitive web interface designed with Django and integrates the Agora SDK to facilitate real-time video conferencing in virtual classes. The proposed system also offers real-time feedback on student emotions, enabling teachers to obtain important information to adapt and deliver a more effective, responsive education. The running source code is available at: https://github.com/mahiraaly/Real-Time-Emotion-Detection-Using-CNN.","manuscriptTitle":"An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-24 23:11:15","doi":"10.21203/rs.3.rs-7441997/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"117067426153185361009764291958822667994","date":"2025-11-13T08:10:51+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-11-13T04:55:31+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-26T10:16:02+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-26T10:15:17+00:00","index":"","fulltext":""},{"type":"submitted","content":"User Modeling and User-Adapted Interaction","date":"2025-08-23T14:51:39+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"user-modeling-and-user-adapted-interaction","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"umui","sideBox":"Learn more about [User Modeling and User-Adapted Interaction](http://link.springer.com/journal/11257)","snPcode":"11257","submissionUrl":"https://submission.nature.com/new-submission/11257/3","title":"User Modeling and User-Adapted Interaction","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"38962b51-f75f-4205-97a4-3bfac6221fe9","owner":[],"postedDate":"November 24th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-11-24T23:11:15+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-24 23:11:15","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7441997","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7441997","identity":"rs-7441997","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}