Hate Speech Detection in Roman Urdu English Tweets Through Data Pre-processing

Hate Speech Detection in Roman Urdu English Tweets Through Data Pre-processing | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Hate Speech Detection in Roman Urdu English Tweets Through Data Pre-processing Muhammad Asif Khan, Jazib e nazar, GuohHua Liu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6345769/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Hate speech detection enhances internet safety by recognizing and reducing harmful or objectionable information. The increasing use of social media has made it more difficult to control and moderate hate speech. The casual and varied nature of material on Twitter presents a particular difficulty for hate speech identification because of its diversified and multilingual user base that includes code-mixed languages like Roman Urdu-English. To tackle the issue of Hate Speech in code mixed Roman Urdu-English little amount of research has been done by the NLP and machine learning community. To solve this problem, this article looks at how data pre-processing affects the ability to identify hate speech in tweets that combine Roman Urdu and English codes. We used a comprehensive 10-step data cleaning procedure followed by the Multilingual BERT (mBERT) model for Hate Speech detection. The methodology includes optimizing hyper parameters and carrying out comprehensive tests to evaluate model's accuracy. The results showed the proposed data preprocessing approach considerably increases accuracy. Compared to previous techniques, the mBERT model showed about 9.12% gain in accuracy. This demonstrates how well our pre-processing methods work and how powerful mBERT is in enhancing hate speech detection on social media networks. Biological sciences/Computational biology and bioinformatics Health sciences/Signs and symptoms code-mixed code-switched neural networks speech detection hatred speech multilingual Full Text Additional Declarations No competing interests reported. Supplementary Files linkofgithubResData.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6345769","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":451772990,"identity":"d1bd50e8-25f6-45c7-ba9f-20888240fc30","order_by":0,"name":"Muhammad Asif Khan","email":"","orcid":"","institution":"Technology Donghua University","correspondingAuthor":false,"prefix":"","firstName":"Muhammad","middleName":"Asif","lastName":"Khan","suffix":""},{"id":451772991,"identity":"f37973c3-9318-4ac8-b259-3c3cb757c3da","order_by":1,"name":"Jazib e nazar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGElEQVRIiWNgGAWjYBAC9gYGhsNQNuMDBgYJIJ3AwMDDAGFiA4xIWpgNQOp4iNHCDGWzgRUR1tJ+/OHhgoo78vLtZ59V/GyzkLdnT2B88LaNIU+yAYeWnoSEwzPOPDNs7Ek3u9nbJmHYw/OA2XBuG0OxNE6HJRw4zNt2mLGZIY3tNmObBGOPRAKbNG8bQ+I8XFr6HzYc5v132L6N/xlbMVCLPVAL+298WgRnJDMc5m04nNgjkcbGDNSSCLKFGaRlNg4t0hLPGA7POHY4eYbEM2bJnnMSyT1nHjZLzjknUYzL+3z86Y8/F9Qctp3fn8b44UdZnW17e/LBD2/KbPIkDuCwBmuQAAmJBOI1wAAZWkbBKBgFo2CYAgChKlkx0onhuwAAAABJRU5ErkJggg==","orcid":"","institution":"Comsats University","correspondingAuthor":true,"prefix":"","firstName":"Jazib","middleName":"e","lastName":"nazar","suffix":""},{"id":451772992,"identity":"740b05ee-c066-4572-ba96-d1af4244824b","order_by":2,"name":"GuohHua Liu","email":"","orcid":"","institution":"Technology Donghua University","correspondingAuthor":false,"prefix":"","firstName":"GuohHua","middleName":"","lastName":"Liu","suffix":""}],"badges":[],"createdAt":"2025-03-31 14:38:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6345769/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6345769/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":96249304,"identity":"279f8a1b-418e-4360-81d8-9b9ef3a02824","added_by":"auto","created_at":"2025-11-19 07:32:55","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":634965,"visible":true,"origin":"","legend":"","description":"","filename":"HateSpeechDetection.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6345769/v1_covered_23c29fc0-5eb7-4df9-88d0-676a30ad03a3.pdf"},{"id":82156090,"identity":"5c2b22b8-67d3-4661-8032-9b70feaf0e1d","added_by":"auto","created_at":"2025-05-07 07:42:35","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":13385,"visible":true,"origin":"","legend":"","description":"","filename":"linkofgithubResData.docx","url":"https://assets-eu.researchsquare.com/files/rs-6345769/v1/3a4a34f536d1bcc6ddc789b5.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Hate Speech Detection in Roman Urdu English Tweets Through Data Pre-processing","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"code-mixed, code-switched, neural networks, speech detection, hatred speech, multilingual","lastPublishedDoi":"10.21203/rs.3.rs-6345769/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6345769/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eHate speech detection enhances internet safety by recognizing and reducing harmful or objectionable information. The increasing use of social media has made it more difficult to control and moderate hate speech. The casual and varied nature of material on Twitter presents a particular difficulty for hate speech identification because of its diversified and multilingual user base that includes code-mixed languages like Roman Urdu-English. To tackle the issue of Hate Speech in code mixed Roman Urdu-English little amount of research has been done by the NLP and machine learning community. To solve this problem, this article looks at how data pre-processing affects the ability to identify hate speech in tweets that combine Roman Urdu and English codes. We used a comprehensive 10-step data cleaning procedure followed by the Multilingual BERT (mBERT) model for Hate Speech detection. The methodology includes optimizing hyper parameters and carrying out comprehensive tests to evaluate model's accuracy. The results showed the proposed data preprocessing approach considerably increases accuracy. Compared to previous techniques, the mBERT model showed about 9.12% gain in accuracy. This demonstrates how well our pre-processing methods work and how powerful mBERT is in enhancing hate speech detection on social media networks.\u003c/p\u003e","manuscriptTitle":"Hate Speech Detection in Roman Urdu English Tweets Through Data Pre-processing","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-05-07 07:26:31","doi":"10.21203/rs.3.rs-6345769/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1cde33b2-9d90-4ff7-873b-ed1bc44984bd","owner":[],"postedDate":"May 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":48049492,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":48049493,"name":"Health sciences/Signs and symptoms"}],"tags":[],"updatedAt":"2025-11-18T06:54:03+00:00","versionOfRecord":[],"versionCreatedAt":"2025-05-07 07:26:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6345769","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6345769","identity":"rs-6345769","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}