Comparing Cost-Sensitive and Data-Level Strategies to Address Extreme Class Imbalance in Educational Review Sentiment Analysis

Comparing Cost-Sensitive and Data-Level Strategies to Address Extreme Class Imbalance in Educational Review Sentiment Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Comparing Cost-Sensitive and Data-Level Strategies to Address Extreme Class Imbalance in Educational Review Sentiment Analysis Nagwa Elmobark, Sara Elhishi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8294275/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: Educational review sentiment analysis poses serious challenges due to class imbalance, where positive reviews (4-5 stars) predominate, and the dataset lacks critical negative feedback. The result of this imbalance is the creation of models that fail to recognize the critical aspect of minority views, thereby limiting their usefulness in measuring the quality of education. Objective: The paper aims to explore and compare various class imbalance management strategies to enhance sentiment classification accuracy, particularly for minority classes in educational review data. Methods: A large-scale dataset of 107,018 reviews of educational courses was used to conduct a thorough analysis that revealed a severe imbalance (32:1 between the majority and minority classes). Five imbalance handling methods were systematically compared: classification of the baseline, weighted learning on cost-sensitive classification, oversampling by human intervention, weighted learning by ensemble, and combined resampling methods. It was evaluated on macro F1-score, per-class F1-scores, and statistical significance. Results: The weighted Logistic Regression was found to be the best method, with the greatest percentage change in macro F1-score (0.3691 to 0.4087) compared to the traditional methods (10.7). The strategy showed significant improvement across the minority classes: 39.3% in 2-star reviews, 40.9% in 3-star reviews, 49.7% in 4-star reviews, and 99.9% in 5-star reviews. The statistical analysis revealed significant improvements across all underrepresented classes. Conclusions: This study demonstrates that basic cost-conscious learning strategies can effectively counteract extreme class imbalance in educational sentiment analysis, eliminating the need for complicated resampling techniques or ensemble analyses. The results are beneficial, offering feasible recommendations for creating more balanced and reliable sentiment analysis systems for academic use, enabling the effective identification of the critical feedback needed to enhance the quality of education. Artificial Intelligence and Machine Learning class imbalance sentiment analysis educational data mining cost-sensitive learning text classification machine learning Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8294275","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":556203533,"identity":"bf724d4b-d17e-45e9-ba28-a8f8ef4dff0b","order_by":0,"name":"Nagwa Elmobark","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABHElEQVRIiWNgGAWjYHAD5gYGxgYGBgn2BgZmIrUwQrXwHCBZi0QCfi380u0XmAtqahP72xtbN3zcYZcvOfON4eeCChsG/vbuBGxaJOecKWCecex44owzB9tuzjyTbDlbOsdYesaZNAaJM2c3YNNicCMngZmH7Vhuw43Ettu8bcwGctI5BtK8bYcZDCRysWqxB2v5dyx3/v2HIC31BnKSZ4x/49NiIJF+gJm3rSZ3ww1GkJbDBtISPGZ4bZG4kcPAzNt3oH7jmUSQX44bSPaklVnznEnjweUX/hnpD5h5vtUZyx0/fOzGxx3VBhLHD2++zVNhI8ff3otVCwMDj/kPBobDyCIcBmBx7MpBgP0BkKjDEBkFo2AUjIJRAAcAtsxn3cU0SlcAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-6757-088X","institution":"Mansoura University","correspondingAuthor":true,"prefix":"","firstName":"Nagwa","middleName":"","lastName":"Elmobark","suffix":""},{"id":556203567,"identity":"cb3bc602-4892-4f14-89b5-868954ab7f5b","order_by":1,"name":"Sara Elhishi","email":"","orcid":"","institution":"Arab Open University of Egypt","correspondingAuthor":false,"prefix":"","firstName":"Sara","middleName":"","lastName":"Elhishi","suffix":""}],"badges":[],"createdAt":"2025-12-06 11:25:38","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8294275/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8294275/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97894713,"identity":"90062a3d-e400-4afb-ab88-fed2d2545f46","added_by":"auto","created_at":"2025-12-10 15:32:55","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":825370,"visible":true,"origin":"","legend":"","description":"","filename":"ComparingCostSensitiveandDataLevelStrategiestoAddressExtremeClassImbalanceinEducationalReviewSentimentAnalysis.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8294275/v1_covered_8594b19b-6c56-456c-aef1-4197667bc5b4.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eComparing Cost-Sensitive and Data-Level Strategies to Address Extreme Class Imbalance in Educational Review Sentiment Analysis\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Mansoura University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"class imbalance, sentiment analysis, educational data mining, cost-sensitive learning, text classification, machine learning","lastPublishedDoi":"10.21203/rs.3.rs-8294275/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8294275/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Educational review sentiment analysis poses serious challenges due to class imbalance, where positive reviews (4-5 stars) predominate, and the dataset lacks critical negative feedback. The result of this imbalance is the creation of models that fail to recognize the critical aspect of minority views, thereby limiting their usefulness in measuring the quality of education.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eObjective:\u003c/strong\u003e The paper aims to explore and compare various class imbalance management strategies to enhance sentiment classification accuracy, particularly for minority classes in educational review data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e \u0026nbsp;A large-scale dataset of 107,018 reviews of educational courses was used to conduct a thorough analysis that revealed a severe imbalance (32:1 between the majority and minority classes). Five imbalance handling methods were systematically compared: classification of the baseline, weighted learning on cost-sensitive classification, oversampling by human intervention, weighted learning by ensemble, and combined resampling methods. It was evaluated on macro F1-score, per-class F1-scores, and statistical significance.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e The weighted Logistic Regression was found to be the best method, with the greatest percentage change in macro F1-score (0.3691 to 0.4087) compared to the traditional methods (10.7). The strategy showed significant improvement across the minority classes: 39.3% in 2-star reviews, 40.9% in 3-star reviews, 49.7% in 4-star reviews, and 99.9% in 5-star reviews. The statistical analysis revealed significant improvements across all underrepresented classes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e This study demonstrates that basic cost-conscious learning strategies can effectively counteract extreme class imbalance in educational sentiment analysis, eliminating the need for complicated resampling techniques or ensemble analyses. The results are beneficial, offering feasible recommendations for creating more balanced and reliable sentiment analysis systems for academic use, enabling the effective identification of the critical feedback needed to enhance the quality of education.\u003c/p\u003e","manuscriptTitle":"Comparing Cost-Sensitive and Data-Level Strategies to Address Extreme Class Imbalance in Educational Review Sentiment Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-09 03:05:13","doi":"10.21203/rs.3.rs-8294275/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3f619afd-723b-452f-88df-5554c9393864","owner":[],"postedDate":"December 9th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":59201476,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-12-09T03:05:13+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-09 03:05:13","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8294275","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8294275","identity":"rs-8294275","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}