Understanding Transformer-Based OCR for Medieval Manuscripts: A Systematic Ablation Study and Inspection Analysis

Understanding Transformer-Based OCR for Medieval Manuscripts: A Systematic Ablation Study and Inspection Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Understanding Transformer-Based OCR for Medieval Manuscripts: A Systematic Ablation Study and Inspection Analysis Sachin Sharma, Federico Simonetta This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8109357/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Adapting Transformer-based Optical Character Recognition (TrOCR) models to medieval manuscripts presents a significant domain gap. This work provides a systematic investigation into TrOCR fine-tuning strategies using a 14th-15th century Italian manuscript. We conduct controlled ablation studies on preprocessing, data augmentation, and encoder layer freezing. Results demonstrate that full fine-tuning of all encoder layers is critical, achieving an 11.68% Character Error Rate (CER). We also show that ContrastLimited Adaptive Histogram Equalization(CLAHE) preprocessing yields a 12.9% relative CER reduction. Our hyperparameter configuration generalized effectively, achieving 7.68% CER on the public READ-16 benchmark. As a key contribution, we perform a quantitative analysis of model localization maps. We establish that encoder-based Grad-CAM entropy and Gini impurity are a much stronger correlates of token prediction loss than decoder cross-attention. We propose its utility as a robust diagnostic for visual uncertainty. This finding has direct applications for uncertainty sampling in active learning and pseudo-label filtering in semi-supervised learning workflows. This study offers both practical guidelines for adapting TrOCR and a novel method for interpreting model adaptation. TrOCR Ablation Study Medieval Manuscripts Transfer Learning Digital Humanities Decoder Attention Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8109357","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":554635763,"identity":"ff68c3d8-573d-4b3a-a935-315d5e7e6321","order_by":0,"name":"Sachin Sharma","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6klEQVRIiWNgGAWjYHACNoYEIGkAZlcAMTMDwwEStJyBaiGghw1MgrUwtkHF8GmRbz/87MHDHQz25uxnn274Oa/WXr6dgfHwBzxaDM6kmRsknmFI3NmTbnazd9vxxA2HCTjMgCGHTSKxjSHB4EAa2w3ebccSDAj5Rb7/DViLvcH5Z2w3/845Zi/fTMj7NyC2MG64kcZ2m7ehhrGBoMNuPDMDapFI3HDjGdttmWMHgH5hbDhwBq/Dkp9J/myzATosje3mm5o6e/n+w4c/VOBzGARIwBhAVzEwNhDWgATqSFI9CkbBKBgFIwMAALF7U2hrL5pUAAAAAElFTkSuQmCC","orcid":"","institution":"Gran Sasso Science Institute","correspondingAuthor":true,"prefix":"","firstName":"Sachin","middleName":"","lastName":"Sharma","suffix":""},{"id":554635765,"identity":"f046c40d-10ff-47c3-8abd-52d4ce3263f0","order_by":1,"name":"Federico Simonetta","email":"","orcid":"","institution":"Gran Sasso Science Institute","correspondingAuthor":false,"prefix":"","firstName":"Federico","middleName":"","lastName":"Simonetta","suffix":""}],"badges":[],"createdAt":"2025-11-14 00:08:09","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8109357/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8109357/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":97481142,"identity":"b097b45d-edf4-415d-9d97-f39c024a8a21","added_by":"auto","created_at":"2025-12-04 21:54:01","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4136,"visible":true,"origin":"","legend":"","description":"","filename":"d62b470616984e7a819d12d2300aec76.json","url":"https://assets-eu.researchsquare.com/files/rs-8109357/v1/273e50a5cb55de34dee6cb73.json"},{"id":100547254,"identity":"343f53b5-d7e7-41ad-9683-a28fe26c18b2","added_by":"auto","created_at":"2026-01-19 08:15:00","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":13775965,"visible":true,"origin":"","legend":"","description":"","filename":"mczpnhcykssgztxbdgphbxnpvtwhjnwh.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8109357/v1_covered_3b123818-9ed5-4812-ae90-1ecc3c9b9999.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Understanding Transformer-Based OCR for Medieval Manuscripts: A Systematic Ablation Study and Inspection Analysis","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"TrOCR, Ablation Study, Medieval Manuscripts, Transfer Learning, Digital Humanities, Decoder Attention","lastPublishedDoi":"10.21203/rs.3.rs-8109357/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8109357/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAdapting Transformer-based Optical Character Recognition (TrOCR) models to medieval manuscripts presents a significant domain gap. This work provides a systematic investigation into TrOCR fine-tuning strategies using a 14th-15th century Italian manuscript. We conduct controlled ablation studies on preprocessing, data augmentation, and encoder layer freezing. Results demonstrate that full fine-tuning of all encoder layers is critical, achieving an 11.68% Character Error Rate (CER). We also show that ContrastLimited Adaptive Histogram Equalization(CLAHE) preprocessing yields a 12.9% relative CER reduction. Our hyperparameter configuration generalized effectively, achieving 7.68% CER on the public READ-16 benchmark.\u003c/p\u003e\n\u003cp\u003eAs a key contribution, we perform a quantitative analysis of model localization maps. We establish that encoder-based Grad-CAM entropy and Gini impurity are a much stronger correlates of token prediction loss than decoder cross-attention. We propose its utility as a robust diagnostic for visual uncertainty. This finding has direct applications for uncertainty sampling in active learning and pseudo-label filtering in semi-supervised learning workflows. This study offers both practical guidelines for adapting TrOCR and a novel method for interpreting model adaptation.\u003c/p\u003e","manuscriptTitle":"Understanding Transformer-Based OCR for Medieval Manuscripts: A Systematic Ablation Study and Inspection Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-04 21:53:57","doi":"10.21203/rs.3.rs-8109357/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ab5dd9f8-3072-4199-9c9b-0f6c4087c20d","owner":[],"postedDate":"December 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-01-16T18:53:39+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-04 21:53:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8109357","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8109357","identity":"rs-8109357","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}