Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar

Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar Aung Kyaw Htet, Mark Dras This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4329843/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 06 Jun, 2025 Read the published version in Language Resources and Evaluation → Version 1 posted 10 You are reading this latest preprint version Abstract Despite dramatic recent progress in NLP, it is still a major challenge to apply Large Language Models (LLM) to low-resource languages. This is made visible in benchmarks such as Cross-Lingual Natural Language Inference (XNLI), a key task that demonstrates cross-lingual capabilities of NLP systems across a set of 15 languages. In this paper, we extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages, and make three core contributions. First, we build a dataset called Myanmar XNLI (myXNLI) using community crowd-sourced methods, as an extension to the existing XNLI corpus. This involves a two-stage process of community-based construction followed by expert verification; through an analysis, we demonstrate and quantify the value of the expert verification stage in the context of community-based construction for low-resource languages. We make the myXNLI dataset available to the community for future research. Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance. Our data-augmentation methods improve model accuracy by up to 2 percentage points for Myanmar, while uplifting other languages at the same time.Third, we investigate how well these data-augmentation methods generalise to other low-resource languages in the XNLI dataset. Low-Resource Natural Language Inference Burmese Myanmar Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 06 Jun, 2025 Read the published version in Language Resources and Evaluation → Version 1 posted Editorial decision: Revision requested 04 Mar, 2025 Reviews received at journal 28 Nov, 2024 Reviews received at journal 20 Nov, 2024 Reviewers agreed at journal 27 Oct, 2024 Reviewers agreed at journal 25 Oct, 2024 Reviewers agreed at journal 24 Oct, 2024 Reviewers invited by journal 24 Oct, 2024 Editor assigned by journal 26 Jun, 2024 Submission checks completed at journal 29 Apr, 2024 First submitted to journal 26 Apr, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4329843","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":296512270,"identity":"8251f139-0cfe-4ea5-9cb1-e6b310a092cc","order_by":0,"name":"Aung Kyaw Htet","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuklEQVRIiWNgGAWjYDACHh4QaQPEjI0HiNXC2MCQkAbS0kCSlsNgNnFa+HvOHn/w8cd5u7Xth4G21NhEE9QicbYvsXFGwu3kbWcSgVqOpeU2ENRznsewmQeoxewAUAtjw2HCWuRBWv4knEs2O/+QSC0GZ3sMmxkSDtiZ3SDWFsMzZwxn9qQlJ5jdANqSQIxf5M7kGHz4YWNnb3Y+/eGDDzU2RHgfChLBKhOIVQ4C9qQoHgWjYBSMghEGAGNbSjBEEwYtAAAAAElFTkSuQmCC","orcid":"","institution":"Macquarie University","correspondingAuthor":true,"prefix":"","firstName":"Aung","middleName":"Kyaw","lastName":"Htet","suffix":""},{"id":296512271,"identity":"ed65f5f5-27a8-4221-809e-e2520f1d2087","order_by":1,"name":"Mark Dras","email":"","orcid":"","institution":"Macquarie University","correspondingAuthor":false,"prefix":"","firstName":"Mark","middleName":"","lastName":"Dras","suffix":""}],"badges":[],"createdAt":"2024-04-26 13:23:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4329843/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4329843/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10579-025-09844-1","type":"published","date":"2025-06-06T15:57:09+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":84243126,"identity":"49a53f0c-872c-45d8-bffa-98e7ee4a0b25","added_by":"auto","created_at":"2025-06-09 16:12:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1436990,"visible":true,"origin":"","legend":"","description":"","filename":"myxnlispringer.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4329843/v1_covered_8cf2c665-8f4c-4f29-97f9-88fcb7d03fac.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"language-resources-and-evaluation","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"lrev","sideBox":"Learn more about [Language Resources and Evaluation](http://link.springer.com/journal/10579)","snPcode":"10579","submissionUrl":"https://submission.nature.com/new-submission/10579/3","title":"Language Resources and Evaluation","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Low-Resource, Natural Language Inference, Burmese, Myanmar","lastPublishedDoi":"10.21203/rs.3.rs-4329843/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4329843/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Despite dramatic recent progress in NLP, it is still a major challenge to apply Large Language Models (LLM) to low-resource languages. This is made visible in benchmarks such as Cross-Lingual Natural Language Inference (XNLI), a key task that demonstrates cross-lingual capabilities of NLP systems across a set of 15 languages.\nIn this paper, we extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages, and make three core contributions. First, we build a dataset called Myanmar XNLI (myXNLI) using community crowd-sourced methods, as an extension to the existing XNLI corpus. This involves a two-stage process of community-based construction followed by expert verification; through an analysis, we demonstrate and quantify the value of the expert verification stage in the context of community-based construction for low-resource languages. We make the myXNLI dataset available to the community for future research. Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance. Our data-augmentation methods improve model accuracy by up to 2 percentage points for Myanmar, while uplifting other languages at the same time.Third, we investigate how well these data-augmentation methods generalise to other low-resource languages in the XNLI dataset.","manuscriptTitle":"Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-03 05:28:27","doi":"10.21203/rs.3.rs-4329843/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-03-04T10:19:48+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-29T03:52:09+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-20T09:33:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"40375637867105144522693366034769238052","date":"2024-10-27T13:10:54+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"336012682900763365422403627390248102252","date":"2024-10-25T06:35:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"189676171809634384989411727790430692644","date":"2024-10-24T11:16:51+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-10-24T11:00:27+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-06-26T11:46:17+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-04-29T04:05:54+00:00","index":"","fulltext":""},{"type":"submitted","content":"Language Resources and Evaluation","date":"2024-04-26T12:58:30+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"language-resources-and-evaluation","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"lrev","sideBox":"Learn more about [Language Resources and Evaluation](http://link.springer.com/journal/10579)","snPcode":"10579","submissionUrl":"https://submission.nature.com/new-submission/10579/3","title":"Language Resources and Evaluation","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"e6bb35c3-2696-424b-a182-ae47a3d94e9e","owner":[],"postedDate":"May 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-06-09T16:08:42+00:00","versionOfRecord":{"articleIdentity":"rs-4329843","link":"https://doi.org/10.1007/s10579-025-09844-1","journal":{"identity":"language-resources-and-evaluation","isVorOnly":false,"title":"Language Resources and Evaluation"},"publishedOn":"2025-06-06 15:57:09","publishedOnDateReadable":"June 6th, 2025"},"versionCreatedAt":"2024-05-03 05:28:27","video":"","vorDoi":"10.1007/s10579-025-09844-1","vorDoiUrl":"https://doi.org/10.1007/s10579-025-09844-1","workflowStages":[]},"version":"v1","identity":"rs-4329843","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4329843","identity":"rs-4329843","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}