A Robust Inpage-to-Unicode Encoding Converter for Kashmiri and Related Languages: Addressing a 15-Year-Old Challenge (2024)

A Robust Inpage-to-Unicode Encoding Converter for Kashmiri and Related Languages: Addressing a 15-Year-Old Challenge (2024) | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Robust Inpage-to-Unicode Encoding Converter for Kashmiri and Related Languages: Addressing a 15-Year-Old Challenge (2024) HAQ NAWAZ MALIK, Syed Murtaza Rizvi, Aadil Amin Kak, Nazima Mehdi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5737132/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract For over fifteen years, Kashmiri, Urdu, and Pashto texts composed in Inpage have faced significant barriers to digitization and wide dissemination due to Inpage’s non-Unicode nature. Although Inpage-to Unicode converters exist, most tools are tailored to Urdu and have not adequately addressed the distinctive letters and diacritics of Kashmiri script. The resulting inaccuracies have effectively “locked” a substantial body of Kashmiri literature in proprietary formats. This paper presents a new Inpage-to-Unicode converter that achieves near 98.6% accuracy for Kashmiri by combining meticulous letter-to-letter mapping, specialized diacritic handling, and an innovative chunk-based text processing architecture that keeps user interfaces responsive. We further incorporate a dictionary-based post-processing step to resolve rare or archaic word-forms. Empirical evaluations on a diverse set of Kashmiri texts demonstrate the converter’s efficacy and establish its potential to unlock decades of cultural and academic material currently confined within Inpage documents. Theoretical Computer Science Linguistics Inpage Unicode Kashmiri script diacritical correction dictionary-based refinement. Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5737132","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":395926245,"identity":"6dd66418-b743-4828-8089-303a48f2348c","order_by":0,"name":"HAQ NAWAZ MALIK","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyklEQVRIiWNgGAWjYBACAxiDjb0BSBYwMEgQr4XnAAPDAQNStDBIJBCpxVy6O4Hhxx8GeT7Jx8ekPxjYyEk2MD98dAOPFss5Zzcw9rYxGLZJp6VJHDBIM5ZmYDM2zsHnsBu5G5gZGxgY26RzzIBaDifOY+BhkyaoheEPg32b5BmStLAxJLZJ8EC0zCZGy8HeNonkNp60ZIszQL9INhP2y8YHP/7Y2M5vP3zwRkWFjZzE8eaHj/FpAYEDqJHBTED5KBgFo2AUjALCAADQ6ETGJrEG6AAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0009-0003-1994-7640","institution":"Birla Institute of Technology And Science, Pilani (BITS Pilani)","correspondingAuthor":true,"prefix":"","firstName":"HAQ","middleName":"NAWAZ","lastName":"MALIK","suffix":""},{"id":395926246,"identity":"064987cb-55c1-45b6-adc6-72be6346e3bc","order_by":1,"name":"Syed Murtaza Rizvi","email":"","orcid":"https://orcid.org/0009-0005-3738-4874","institution":"Birla Institute of Technology And Science, Pilani (BITS Pilani)","correspondingAuthor":false,"prefix":"","firstName":"Syed","middleName":"Murtaza","lastName":"Rizvi","suffix":""},{"id":395926247,"identity":"eb1b0009-e0f4-42ca-a897-214a1d32be52","order_by":2,"name":"Aadil Amin Kak","email":"","orcid":"https://orcid.org/0009-0002-7663-5531","institution":"University of Kashmir","correspondingAuthor":false,"prefix":"","firstName":"Aadil","middleName":"Amin","lastName":"Kak","suffix":""},{"id":395926248,"identity":"3eb67038-bfd0-433a-8f98-092dde75cd70","order_by":3,"name":"Nazima Mehdi","email":"","orcid":"","institution":"University of Kashmir","correspondingAuthor":false,"prefix":"","firstName":"Nazima","middleName":"","lastName":"Mehdi","suffix":""}],"badges":[],"createdAt":"2024-12-30 17:23:07","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-5737132/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5737132/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":72662053,"identity":"03a81636-d260-4617-a89b-90e5c9d33f58","added_by":"auto","created_at":"2024-12-31 02:19:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4872818,"visible":true,"origin":"","legend":"","description":"","filename":"InpagetoUnicodeConverter16.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5737132/v1_covered_44eff140-24d4-4c87-80b6-cdb633baf42b.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eA Robust Inpage-to-Unicode Encoding Converter for Kashmiri and Related Languages: Addressing a 15-Year-Old Challenge (2024)\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Birla Institute of Technology and Science, Pilani","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Inpage, Unicode, Kashmiri script, diacritical correction, dictionary-based refinement.","lastPublishedDoi":"10.21203/rs.3.rs-5737132/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5737132/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eFor over fifteen years, Kashmiri, Urdu, and Pashto texts composed in Inpage have faced significant barriers to digitization and wide dissemination due to Inpage’s non-Unicode nature. Although Inpage-to Unicode converters exist, most tools are tailored to Urdu and have not adequately addressed the distinctive letters and diacritics of Kashmiri script. The resulting inaccuracies have effectively “locked” a substantial body of Kashmiri literature in proprietary formats. This paper presents a new Inpage-to-Unicode converter that achieves near 98.6% accuracy for Kashmiri by combining meticulous letter-to-letter mapping, specialized diacritic handling, and an innovative chunk-based text processing architecture that keeps user interfaces responsive. We further incorporate a dictionary-based post-processing step to resolve rare or archaic word-forms. Empirical evaluations on a diverse set of Kashmiri texts demonstrate the converter’s efficacy and establish its potential to unlock decades of cultural and academic material currently confined within Inpage documents.\u003c/p\u003e","manuscriptTitle":"A Robust Inpage-to-Unicode Encoding Converter for Kashmiri and Related Languages: Addressing a 15-Year-Old Challenge (2024)","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-31 02:11:05","doi":"10.21203/rs.3.rs-5737132/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cf56c32c-2224-437f-ad53-adb6f609c15e","owner":[],"postedDate":"December 31st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":42178124,"name":"Theoretical Computer Science"},{"id":42178125,"name":"Linguistics"}],"tags":[],"updatedAt":"2024-12-31T02:11:05+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-31 02:11:05","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5737132","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5737132","identity":"rs-5737132","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}