User-based Evaluation of Explainability Techniques for Misogyny Detection in Code-mixed Hindi-English

User-based Evaluation of Explainability Techniques for Misogyny Detection in Code-mixed Hindi-English | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 22 September 2025 V1 Latest version Share on User-based Evaluation of Explainability Techniques for Misogyny Detection in Code-mixed Hindi-English Authors : Sargam Yadav 0000-0001-8115-6741 [email protected] , Abhishek Kaushik , and Kevin McDaid Authors Info & Affiliations https://doi.org/10.22541/au.175856176.63643017/v1 Published Applied AI Letters Version of record Peer review timeline 115 views 119 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Although Artificial Intelligence (AI) models have demonstrated great success and efficiency in automatically countering online hate speech and misogyny, they suffer from a lack of explainability and transparency. Explainable Artificial Intelligence (XAI) is an apparent solution to make opaque black-box models more explainable, but remains under-explored for low resource and code-mixed languages. In this study, user-based evaluation of 3 explainability techniques for a misogyny classifier has been performed through simulatability, what-if explainability, and a qualitative feedback that covers several key dimensions. The participants were also required to mark the rationales to justify their predictions, and required to answer an open-ended question regarding their opinion on the explanations. Analysis of the results from 10 participants highlights that Integrated Gradients (IG) was more consistent and plausible than other techniques, and their preference for token attribution heatmaps and bar plots over Natural Language Explanations (NLE). Supplementary Material File (code_mixed_misogyny_xai_user_evaluation-2.pdf) Download 886.78 KB Information & Authors Information Version history V1 Version 1 22 September 2025 Peer review timeline Published Applied AI Letters Version of Record 11 Mar 2026 Published Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords code-mixing explainability hate speech hinglish natural language processing Authors Affiliations Sargam Yadav 0000-0001-8115-6741 [email protected] Dundalk Institute of Technology View all articles by this author Abhishek Kaushik Dundalk Institute of Technology View all articles by this author Kevin McDaid Dundalk Institute of Technology View all articles by this author Metrics & Citations Metrics Article Usage 115 views 119 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Sargam Yadav, Abhishek Kaushik, Kevin McDaid. User-based Evaluation of Explainability Techniques for Misogyny Detection in Code-mixed Hindi-English. Authorea . 22 September 2025. DOI: https://doi.org/10.22541/au.175856176.63643017/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.175856176.63643017/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9febef6dfe19f047',t:'MTc3OTI4NjMzNQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();