Improving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability

Improving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Improving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability Anandita Jamwal, Manikandan Ravikiran, Dinesh Singh, Rohit Saluja This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8121786/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Scene text recognition (STR) is crucial for traffic monitoring, surveillance, and autonomous driving.It is even more challenging to read texts in rainy weather conditions. Rain-induced distortions, suchas raindrops, streaks, and rain accumulation, can obscure essential visual features like signboards,traffic signs, and license plates, making meaningful data extraction difficult for recognition models.Recent deep learning methods to add realistic rain to scene text images include no control over thesynthesis of rain and the readability of texts. This work addresses the problem of STR in rainy scenesby proposing a controllable rain synthesis and refinement pipeline that controls rain realism and text-readability in rainy images generated from clean images. The pipeline uses an alternating-projectionrefinement technique by introducing two interpretable hyperparameters: readability suppression (α)and rain realism (w). The optimal setup help improve results on the real rainy dataset. Overall, weobserve the reduction in word error rate (WER) and character error rate (CER) by 8.44% and 9.21%respectively over state-of-the-art benchmarks. We also present the ablations of tuning the parameters(α and w) on STR performance scene text image deraining text recognition and detection Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8121786","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":557692797,"identity":"7b15855e-c2c5-4038-bfa0-5ae9adf69c1b","order_by":0,"name":"Anandita Jamwal","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABMklEQVRIie3QPUvDQBjA8SsHyfLYrFfqy1c4KFShxXyVC4F0yaB0cQoFIS4V134Mu3V8ymFdgq4tGTQImSoWulis4OWEOiTQ1eH+kNwR8uM5jhCT6T/2eI3IeLfYcv3BonqhoBdRQZKZh2cXAQFNREHoHjIPW3i1kn9E/b17VVZHwXHOn12X+eO3y8+I1G3n4X4z6RwSW76SbFIiDUQxHfHUG7Kg3xoJWRzMWtwmPSAQcOIlJcKnA5SMpwJY2G6CQE3mB7FUJw0J8eIykbWB3PInV5HTLxDR75TvgjjLajKjRF0y1oZqCgVBNUn1FFY9pZFYBfG9YZL3mxBIUKSdHsU9sFjOsYLUXz7WK7Y9d+0bf7yGbnR8cjfNF+9xx3UcP8s2ZVIKdjtLPbgfmEwmk6miH8UVZquFAbLuAAAAAElFTkSuQmCC","orcid":"","institution":"Indian Institute of Technology Mandi","correspondingAuthor":true,"prefix":"","firstName":"Anandita","middleName":"","lastName":"Jamwal","suffix":""},{"id":557692798,"identity":"ed3198f4-062b-4d5b-976e-c8fc246abfc6","order_by":1,"name":"Manikandan Ravikiran","email":"","orcid":"","institution":"Indian Institute of Technology Mandi","correspondingAuthor":false,"prefix":"","firstName":"Manikandan","middleName":"","lastName":"Ravikiran","suffix":""},{"id":557692799,"identity":"227c0a61-5a4e-4cbe-92ec-1af2b40b6990","order_by":2,"name":"Dinesh Singh","email":"","orcid":"","institution":"Indian Institute of Technology Mandi","correspondingAuthor":false,"prefix":"","firstName":"Dinesh","middleName":"","lastName":"Singh","suffix":""},{"id":557692802,"identity":"8e1e8796-3f70-41ac-b8a7-27f0238ec672","order_by":3,"name":"Rohit Saluja","email":"","orcid":"","institution":"Indian Institute of Technology Mandi","correspondingAuthor":false,"prefix":"","firstName":"Rohit","middleName":"","lastName":"Saluja","suffix":""}],"badges":[],"createdAt":"2025-11-15 11:53:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8121786/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8121786/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97860746,"identity":"fc6c2fb6-ad4c-4de0-affc-ebdbeb2afb5d","added_by":"auto","created_at":"2025-12-10 08:44:45","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5502,"visible":true,"origin":"","legend":"","description":"","filename":"406b946d57784f70a6d152d8a46051a7.json","url":"https://assets-eu.researchsquare.com/files/rs-8121786/v1/9d13a489061234899b9c6cea.json"},{"id":99788210,"identity":"f5820d7f-08d9-4e2d-9570-556cb452929b","added_by":"auto","created_at":"2026-01-08 12:45:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1108134,"visible":true,"origin":"","legend":"","description":"","filename":"STRRainy2IJDAR1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8121786/v1_covered_d49cf015-cc94-4aa8-a2ea-aaa2c9829853.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eImproving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"scene text, image deraining, text recognition and detection","lastPublishedDoi":"10.21203/rs.3.rs-8121786/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8121786/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Scene text recognition (STR) is crucial for traffic monitoring, surveillance, and autonomous driving.It is even more challenging to read texts in rainy weather conditions. Rain-induced distortions, suchas raindrops, streaks, and rain accumulation, can obscure essential visual features like signboards,traffic signs, and license plates, making meaningful data extraction difficult for recognition models.Recent deep learning methods to add realistic rain to scene text images include no control over thesynthesis of rain and the readability of texts. This work addresses the problem of STR in rainy scenesby proposing a controllable rain synthesis and refinement pipeline that controls rain realism and text-readability in rainy images generated from clean images. The pipeline uses an alternating-projectionrefinement technique by introducing two interpretable hyperparameters: readability suppression (α)and rain realism (w). The optimal setup help improve results on the real rainy dataset. Overall, weobserve the reduction in word error rate (WER) and character error rate (CER) by 8.44% and 9.21%respectively over state-of-the-art benchmarks. We also present the ablations of tuning the parameters(α and w) on STR performance","manuscriptTitle":"Improving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-10 08:44:41","doi":"10.21203/rs.3.rs-8121786/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"85ea8f20-1e00-462e-b934-1c17ffba2354","owner":[],"postedDate":"December 10th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-12-28T05:23:38+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-10 08:44:41","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8121786","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8121786","identity":"rs-8121786","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}