Knowledge Distillation with Applications to Interpretable Arabic Sentiment Analysis

Knowledge Distillation with Applications to Interpretable Arabic Sentiment Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Knowledge Distillation with Applications to Interpretable Arabic Sentiment Analysis Arwa Diwali, Kawther Saeedi, Kia Dashtipour, Mandar Gogate, Amir Hussain This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5356825/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Sentiment analysis stands as a focal point in the current landscape of natural language processing research with deep neural network models as being prevalent tools of choice. While these models have exhibited noteworthy performance, their intricate nature frequently renders them akin to black boxes, resulting in a lack of transparency regarding the internal mechanisms of the sentiment classification process. The lack of interpretability in such models raises concerns regarding the reliance on outcomes from opaque systems. This study introduces an approach for distilling knowledge from complex deep neural network models into simpler and more interpretable ones while maintaining performance and ensuring global interpretability. Three distinct knowledge distillation pipelines are proposed to transfer the knowledge acquired by teacher models, including Long Short-Term Memory, Bidirectional Long Short-Term Memory, Convolutional Neural Network and AraBERT into Logistic Regression and Decision Tree models. Conducting thorough assessments across three separate datasets for Arabic sentiment analysis, the study’s proposed approach consistently demonstrates performance levels that surpass those of complex models. Sentiment Analysis Knowledge Distillation Predictive Models Deep Learning Interpretability Full Text Additional Declarations No competing interests reported. Supplementary Files 2024SupplementaryMaterial.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5356825","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":375327555,"identity":"46bf2616-ef23-49e7-be39-0bf90996ef8c","order_by":0,"name":"Arwa Diwali","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAz0lEQVRIiWNgGAWjYFCCxAYGhgIJOQkwh41oLQYSxqRoSQBiA4bEGURr4W9PbpP4YWCRPrP9jAHDhzIGe/4GAlokzjxsk+wxkMidzZNjwDjjHNC6A4SsuZHYdoMHqGUeQ44BM28b0J2EtMgDtdz8YyCRLsf/xoD5bxuDvTwhLQZALbeBtiRISwBtYWxjYNxASIvhmYftv2UMJAxnznhWcLDnnETiRkJa5I6nPzZ8U1EnL3E+eeODH2U29nKEtKAAoGIJUtSPglEwCkbBKMAFAM0dPqDx4lT7AAAAAElFTkSuQmCC","orcid":"","institution":"King Abdulaziz University","correspondingAuthor":true,"prefix":"","firstName":"Arwa","middleName":"","lastName":"Diwali","suffix":""},{"id":375327556,"identity":"c96d9379-7d78-403f-9cfd-30a922d8b8a4","order_by":1,"name":"Kawther Saeedi","email":"","orcid":"","institution":"King Abdulaziz University","correspondingAuthor":false,"prefix":"","firstName":"Kawther","middleName":"","lastName":"Saeedi","suffix":""},{"id":375327557,"identity":"c8ccc867-2975-4f9b-a93c-420ab9670e07","order_by":2,"name":"Kia Dashtipour","email":"","orcid":"","institution":"Edinburgh Napier University","correspondingAuthor":false,"prefix":"","firstName":"Kia","middleName":"","lastName":"Dashtipour","suffix":""},{"id":375327558,"identity":"8d8dfdb2-36b9-48b7-9ee1-0c00466c3cde","order_by":3,"name":"Mandar Gogate","email":"","orcid":"","institution":"Edinburgh Napier University","correspondingAuthor":false,"prefix":"","firstName":"Mandar","middleName":"","lastName":"Gogate","suffix":""},{"id":375327560,"identity":"7b6e465d-b906-4198-83f8-05005711bcfd","order_by":4,"name":"Amir Hussain","email":"","orcid":"","institution":"Edinburgh Napier University","correspondingAuthor":false,"prefix":"","firstName":"Amir","middleName":"","lastName":"Hussain","suffix":""}],"badges":[],"createdAt":"2024-10-29 20:23:09","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5356825/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5356825/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":82673243,"identity":"ddc3f4be-b938-452f-8257-ce20579dcb68","added_by":"auto","created_at":"2025-05-14 03:23:52","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":657753,"visible":true,"origin":"","legend":"","description":"","filename":"1KnowledgeDistillation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5356825/v1_covered_b19a4067-2817-4ca3-921f-ee74ba91f4bd.pdf"},{"id":69019483,"identity":"0cdf883f-698f-47c0-9809-7c9f7dfba9e3","added_by":"auto","created_at":"2024-11-14 15:35:08","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":549117,"visible":true,"origin":"","legend":"","description":"","filename":"2024SupplementaryMaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-5356825/v1/49dd537242a8a72d79dcd754.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Knowledge Distillation with Applications to Interpretable Arabic Sentiment Analysis","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Sentiment Analysis, Knowledge Distillation, Predictive Models, Deep Learning, Interpretability","lastPublishedDoi":"10.21203/rs.3.rs-5356825/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5356825/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eSentiment analysis stands as a focal point in the current landscape of natural language processing research with deep neural network models as being prevalent tools of choice. While these models have exhibited noteworthy performance, their intricate nature frequently renders them akin to black boxes, resulting in a lack of transparency regarding the internal mechanisms of the sentiment classification process. The lack of interpretability in such models raises concerns regarding the reliance on outcomes from opaque systems. This study introduces an approach for distilling knowledge from complex deep neural network models into simpler and more interpretable ones while maintaining performance and ensuring global interpretability. Three distinct knowledge distillation pipelines are proposed to transfer the knowledge acquired by teacher models, including Long Short-Term Memory, Bidirectional Long Short-Term Memory, Convolutional Neural Network and AraBERT into Logistic Regression and Decision Tree models. Conducting thorough assessments across three separate datasets for Arabic sentiment analysis, the study\u0026rsquo;s proposed approach consistently demonstrates performance levels that surpass those of complex models.\u003c/p\u003e","manuscriptTitle":"Knowledge Distillation with Applications to Interpretable Arabic Sentiment Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-14 15:35:03","doi":"10.21203/rs.3.rs-5356825/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5d41d8a2-b505-448d-803f-12657c7f3458","owner":[],"postedDate":"November 14th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-05-14T03:23:19+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-14 15:35:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5356825","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5356825","identity":"rs-5356825","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}