A Modified Vision Transformer for Kurdish Cursive RTL Handwritten Text Recognition

A Modified Vision Transformer for Kurdish Cursive RTL Handwritten Text Recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Modified Vision Transformer for Kurdish Cursive RTL Handwritten Text Recognition Faraedwn M. Salih, Abdulbasit K. Al-talabani This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9225627/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The problem of handwritten text recognition (HTR) of low-resource cursive scripts is a major problem in document image analysis. This research aims to bridge the gap of unavailable annotated data and effective text recognition systems for Central Kurdish, also known as Sorani, which is a complex script consisting of 34 letters, rich ligatures, and context-dependent diacritical marks, and it is the first time this script is studied in HTR research like our approach. In this research, we present DASNUS, a new large-scale dataset of 11,475 annotated text lines from 867 writers across the Kurdistan region of Iraq. We also propose a deep learning framework for HTR, which combines ResNet-inspired convolutional encoders and a Vision Transformer model. The model is particularly suited for addressing variability, dependencies, and ligatures, which are inherent in Sorani script. The model was trained using several techniques, including span-based masking, geometric and photometric transformations, and depth regularization of the transformer using DropPath and LayerScale. The proposed model achieved a CER of 3.47%, and a WER of 17.37%, comparable to or even better than other state-of-the-art models for Arabic, Persian, and English HTR. In this research, we have established the first benchmark for Kurdish cursive handwriting. We have also demonstrated that well-regularized transformer-based models are capable of effectively recognizing complex, low-resource cursive scripts. The results of this research are expected to pave the way for future research in multilingual OCR, writer adaptation, synthetic data, and inclusive AI. Deep Learning Computer Vision Pattern Recognition Handwriting Vision Transformers Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 1. Introduction In the fields of computer vision and pattern recognition, handwriting recognition remains one of the most technically challenging tasks [ 1 ]. Its applications vary from digitization of historical manuscripts to assistive technologies for people with disabilities [ 2 ]. Despite significant advances for well-resourced languages like English, Arabic, Chinese, and German, there is still a large gap in research and tools for a significant number of written languages. Central Kurdish (Sorani), also known as CKB, is one of them. Sorani is used by 12–14 million people in Iraq and Iran [ 3 ]. It is written with a modified Arabic alphabet and has the same cursive writing direction and characters connected to each other from right to left like Arabic and Persian. However, it also contains unique characters for distinct sounds of the Kurdish language: ڤ, ڵ, پ, چ, ژ, ڕ, ۆ, and گ [ 4 , 5 ]. As a result, the Sorani alphabet contains 34 characters compared to 28 for Arabic and 32 for Persian. This makes handwriting recognition for Sorani significantly more difficult than for Latin and roughly Arabic scripts. Some of these difficulties include context-dependent characters with up to four different forms for some of them [ 6 ], long horizontal ligatures where two or more characters are connected to each other to form a single unit, and diacritical marks, which tend to be faint and misplaced in handwriting. Additionally, there is a lot of variation between writers in terms of pressure and stroke width and spacing (see Fig. 1 ). Significant advancements in handwriting recognition have been made on languages with large annotated datasets. The advancements have evolved from Convolutional Neural Network – Recurrent Neural Network (CNN-RNN) architectures with Connectionist Temporal Classification (CTC) loss functions [ 7 , 8 ] to Vision Transformer (ViT) architectures with self-attention mechanisms for connected cursive scripts [ 9 , 10 ]. However, as far as we know, no publicly available annotated dataset for Central Kurdish handwriting at the line level was available before this research, and no previous research utilized a ViT-based architecture for handwriting recognition on the Sorani script at the line level. This paper contributes to filling these research gaps in two aspects: a dataset and a model. First, we introduce a large-scale dataset for Central Kurdish handwriting. Second, we propose a modified ViT-based model for handwriting recognition on the Sorani script and also works well for other cursive scripts, achieving a Character Error Rate (CER) of 3.47% and a Word Error Rate (WER) of 17.37% on our proposed dataset, which are comparable with or even better than the state-of-the-art results for Arabic, Persian, and English handwriting recognition. The rest of the paper is structured as follows: Section 2 reviews related works on handwriting recognition and transformer-based architectures for handwriting recognition. In Section 3, we introduce the proposed dataset for Central Kurdish handwriting recognition, named DASNUS. In Section 4, we explain the proposed model for handwriting recognition on the Sorani script, and in Section 5, we report the experimental results on the proposed dataset and some fair comparisons. In Section 6, we conclude this paper. 2. Related Work 2.1. Datasets for cursive script handwriting recognition The development of handwriting recognition systems has largely depended on the availability of standardized datasets. For Arabic, the IFN/ENIT database [ 11 ] is one of the most widely used, consisting of 26,459 handwritten Tunisian town and village names from 411 writers. This dataset has become a benchmark for Arabic handwriting recognition, enabling CNN–BLSTM–CTC systems to achieve recognition rates above 92%. Another critical resource is the KHATT dataset [ 12 ], which contains 165,890 words, 589,924 characters, and 9,326 lines of text contributed by 1,000 different writers, with annotations at both the word and line level. The training of deep models that are resistant to handwriting variations has been facilitated by the availability of such datasets. Curated resources have also been advantageous for Persian handwriting recognition. The dataset introduced by the authors of [ 13 ] allowed CNN–BLSTM–CTC models to obtain a 99.35% accuracy in character recognition. In the same vein, Urdu recognition has progressed with datasets of handwritten text lines, such as the 6,000-line dataset utilized in the research paper of [ 14 ], which facilitated the development of segmentation-free sequence modeling approaches. Conversely, Kurdish continues to be underfunded. The KRDOH dataset [ 15 ] comprises 4,304 scanned pages, which amount to 93,612 words and 17,466 text lines from 1,076 authors. Although this is a significant contribution, its potential for end-to-end recognition is restricted by the absence of word-level annotation. Other Kurdish resources concentrate on isolated characters or numerals. More than 315,000 images of digits and isolated characters are included in the K-ZHMARA and K-PIT datasets [ 5 ], while the dataset in [ 16 ] contains 40,940 images of individual Kurdish letters that are evenly distributed across the alphabet. These datasets are indispensable for character-level recognition; however, they fail to address the obstacle of continuous, cursive handwriting recognition. At the word level, the KHWD dataset [ 17 ] provided a notable improvement at the word level by obtaining approximately 400,000 handwritten word images from about 8,000 writers containing 10,000 Sorani Kurdish vocabulary words; this new approach provides evidence supporting the significantly greater challenge in cursive recognition of handwritten words versus isolated characters—the KHWD produced a best WER of 45.68% using an enhanced version of the MobileNetV2 model. 2.2. CNN–RNN architectures In Arabic, Persian, and Urdu, the use of hybrid architectures of CNN, BLSTM, and CTC has been highly significant in the field of Handwritten Text Recognition (HTR). Hassan et al. [ 14 ] proposed a CNN-BLSTM architecture with CTC loss for the case of Urdu, which attained 83.69% accuracy in recognizing characters without explicit segmentation. However, the system faced problems due to the high similarity between classes and the complex placement of diacritic marks, which also applies in the case of Kurdish text. Maalej and Kherallah [ 18 ] also attained high accuracy in recognizing characters by using a combination of CNNs for feature extraction and BLSTM for sequence modeling on the IFN/ENIT dataset for Arabic. In a subsequent study, Mutawa et al. [ 19 ] improved the accuracy of this approach by using a combination of ResNet for feature extraction and BLSTM with CTC for sequence modeling, which attained a CER of 13.2% and a WER of 27.31% on the KHATT dataset. This also proves the effectiveness of complex architectures of CNN-RNN, though with high dependence on large datasets. In the paper [ 13 ], the authors were able to prove that a CNN-BLSTM-CTC architecture can attain a high accuracy of 99.35% in the case of Persian, thereby eliminating the need for explicit segmentation in the case of Persian text. In Kurdish, however, attempts have been limited. Ahmed et al. [ 20 ] proposed a CNN model trained on 40,940 isolated characters, achieving 83% test accuracy, but the model did not extend to word- or line-level recognition. Similarly, the authors in reference [ 16 ] focused only on character-level modeling, leaving the sequential dependencies of connected text largely unaddressed. 2.3. Transformer-based approaches Transformer-based models have also been considered in HTR in light of learning long-range dependencies in connection with recurrent connections. In the case of Arabic, the researchers in reference [ 21 ] proposed a CNN–Transformer architecture hybrid (OCFormer) based on convolutional embeddings and self-attention. Performance was high, but it was dependent on a humongous set of synthetic data of over 30 million images, which posed applicability issues in hard-resource regimes. More recent research has investigated applying ViTs to handwriting text recognition (HTR). In [ 10 ] span feature masking was added to the boosting of contextual learning, and the architecture proposed was able to surpass CNN–BLSTM baselines on English and German data, illustrating the promise of ViTs in handwriting recognition. In another research paper [ 22 ], it was suggested substituting CNN–RNN-based architecture with full Transformer-based architecture and achieving the best results on the KHATT dataset. This, however, needed to have been done with much pretrained knowledge and computational power. Attention-based sequence-to-sequence models have also been tried. In [ 23 ] the authors showed that they perform well on handwriting recognition but showed substantial drops in performance on dense cursive scripts where only small amounts of training data could be had. All these works point to the conclusion that while Transformers give state-of-the-art performance, they require high amounts of labeled or synthesized data — which happens to be a big concern in Kurdish, where such data still do not exist. 2.4. Comparative analysis: Arabic, Persian, Urdu vs. Kurdish Comparison between related scripts finds both similarities and exigent challenges. In Urdu, authors of paper [ 14 ] exemplified that segmentation-free CNN–BLSTM model-based systems could process at the stroke level while being tripped up by dot placement and letter likeness, both of which also appear in Kurdish. In Arabic scripts, IFN/ENIT [ 11 ] and KHATT [ 12 ] datasets allowed CNN–BLSTM–CTC-based systems to break the 92%-threshold in terms of accuracy. The authors in [ 24 ] presented the AHDB dataset in highlighting the value of standardized corpora in the development of the field in Arabic HTR. Mezghani et al. [ 25 ] similarly presented the multi-writer corpus in the form of AHTID/MW to accentuate its value in real-world writer-independent identification. Persian-based research in reference [ 13 ] presented that with deeply learned CNN–BLSTM–CTC systems, it was possible to get close to perfect accuracy upon being trained on finely curated datasets. Kurdish, on the other hand, poses unique challenges. While in comparison to Arabic or Persian, the script of Kurdish excludes some characters such as ڕ, ڵ, ۆ, and ژ that exist in others related scripts. More seriously, no big-scale, word-level, or line-level dataset comparable to IFN/ENIT or KHATT is available. Although KRDOH [ 15 ] represents a progression, it has no word annotation, and data such as K-ZHMARA [ 5 ] and the one in reference [ 16 ] contain only isolated symbol-based data. Therefore, Kurdish significantly trails Arabic, Persian and even Urdu in available dataset richness and benchmark material. 2.5. Motivation and research gap From the above review, several conclusions emerge: CNN–BLSTM–CTC architectures [ 13 , 14 , 18 , 19 ] have proven highly effective for cursive handwriting recognition, but their performance depends on large, well-curated datasets. Transformer-based approaches outperform CNN-RNN architectures for long sequence handling but need enormous amounts of training data. Arabic, Persian, and Urdu have the advantage of standardized datasets like IFN/ENIT [ 11 ], KHATT [ 12 ], and AHDB [ 24 ], while Kurdish only has character- or word-based datasets. There are script-specific challenges for Central Kurdish handwriting recognition: more characters, diacritic marks, and positional variations. This establishes a distinct research gap. Existing CNN–RNN or Transformer approaches cannot be directly applied to Kurdish due to its low-resource status and script-specific complexity. Our work addresses this by developing a ViT-based tailored for Kurdish text-line recognition, and contributing toward dataset expansion, thereby bridging a critical gap in low-resource handwriting recognition research. 3. The DASNUS Dataset The DASNUS dataset was collected as a part of this study and to the best of our knowledge, DASNUS (دەستنووس — Central Kurdish for handwriting manuscript) is the largest publicly available dataset for recognizing handwritten Central Kurdish (Sorani) text. We made the dataset open to the public 1 . 3.1. Data collection A total of 867 writers from the four main governorates of the Kurdistan Region of Iraq volunteered to write, and there were no restrictions on the type of pen or style of writing. Table 1 show a summary of the demographic information. Each writer filled out a structured four-page form (Fig. 2 ). This included a demographic page, a fixed paragraph that make sure all the letters of the Sorani alphabet, digits, and some common symbols were covered, and two randomly assigned paragraphs from a pool of 2,500 distinct Kurdish writings in page 3, and a free-writing portion in page 4. Out of 1,250 forms sent out, 867 were filled out completely and sent back, which made 3,468 pages. Table 1 Demographic distribution of DASNUS writers (N = 867). Category Subgroup Count Percentage (%) Age Below 15 96 11.1 16–25 591 68.2 26–50 156 18.0 More than 50 24 2.8 Address Erbil 305 35.2 Sulaymaniyah 507 58.5 Kirkuk 10 1.2 Duhok 45 5.2 Education Level Primary 33 3.8 Secondary 113 13.0 High-School 163 18.8 University 558 64.4 Gender Male 538 62.1 Female 329 37.9 Handedness Right-handed 765 88.2 Left-handed 102 11.8 3.2. Scanning and preprocessing To keep the delicate nuances of the script, forms were scanned at 600 DPI. Using Otsu's thresholding [ 26 ], morphological closing, contour detection, and skew correction through projection profile analysis [ 27 ], an OpenCV-based pipeline subsequently binarized the data and created 11,475 annotated line pictures. The process is visually presented in Fig. 3 . 3.3. Annotation and quality control Native speakers of the Sorani language manually transcribed each line image verbatim into UTF-8. Inter-annotator agreement was 99.1% (character-level) and 97.6% (word-level) in the double-annotation of 500 randomly sampled lines. All disagreements were resolved by an expert annotator, resulting in an overall transcription accuracy that surpassed 98%. 3.4. Dataset splits A writer-disjoint strategy is employed to divide DASNUS into training, validation, and test sets (Table 2 ). The agglutinative morphology of Sorani Kurdish is reflected in the out-of-vocabulary (OOV) rate of approximately 47% for both evaluation splits, which explains the greater WER disparity that was observed in our experiments. Table 2 Dataset splits and vocabulary statistics. Split #Writers #Lines #Unique Words Vocabulary Size OOV (%) Train 606 8042 10702 10702 - Validation 131 1700 3298 12258 47.18 Test 130 1733 3532 13817 47.28 4. Methodology We have modified a handwriting recognition model that is based on ViT and was expressly designed for the recognition of Central Kurdish (Sorani) cursive text lines. This model can also be used to train on other cursive scripts. An architecture composed of three components: a CNN feature extractor inspired by ResNet, a transformer encoder, and a CTC decoder. The complete architecture is depicted in Fig. 4 . 4.1. CNN feature extractor We utilize a custom CNN front-end that is based on the residual learning framework of He et al. [ 28 ], as opposed to the conventional patch embedding of the original ViT [ 29 ]. Two factors motivate this decision: (1) pure ViTs lack the spatial inductive bias necessary for stable convergence on small datasets, as demonstrated by Li et al. [ 10 ], who demonstrated that the removal of the CNN front-end from their HTR-VT model resulted in a validation CER increase from 3.3% to 26.6% on IAM. Additionally, the Sorani script contains small diacritical marks that are essential for character disambiguation and are easily lost through aggressive downsampling. Our feature extractor generates 128 tokens with a dimension of 768 from a 64×512 grayscale image. A comprehensive description of the architecture is provided in Fig. 5 . It is distinguished from both ResNet-18 [ 28 ] and the HTR-VT variant [ 10 ] in five significant ways: There is no initial downsampling. We replace the 7×7 strided convolution and max-pool of ResNet-18 with a 5×5 convolution at stride (1,1) and omit the initial pooling layer entirely, thereby preserving the full input resolution and retaining fine diacritical details. Gradual downsampling that is asymmetric. Using stride (2,1), layers 1 and 2 reduce the height by half while maintaining the breadth. Layer 2 is followed by an intermediate max-pool with stride (2,1), and stride (2,2) is only applied by layer 3. The sequential character information along the width dimension, which is essential for CTC alignment, is preserved by this height-first strategy. Block configuration that is deeper. We employ a [ 2 , 3 , 4 , 3 ] block configuration (12 total blocks), which is more profound than both ResNet-18 [ 2 , 2 , 2 , 2 ] and HTR-VT [ 2 , 2 , 2 ]. This configuration enables us to acquire the intricate positional variants and ligature patterns of cursive Arabic-script writing. Convolutions that are dilated during the ultimate stage. We retain the fourth stage with stride (1,1) and apply dilation rate 2, rather than removing it as in HTR-VT. This expands the receptive field to capture inter-character stroke dependencies without further spatial resolution loss. Regularization of dropouts. A CNN-level regularization is provided in our low-resource context by the application of a dropout layer (p = 0.3) following the final residual stage, which is absent from both He et al. [ 28 ] and HTR-VT [ 10 ]. The expansion is delayed by the channel progression (192→192→384→768), which concentrates early layers on low-level stroke and diacritical features. The final 768-channel dimension is directly proportional to the transformer embedding size, thereby eradicating any projection layer between the two components. The final 128×768 token sequence is generated through a two-stage spatial collapse, which involves average pooling and adaptive average pooling. Table 3 provides a comprehensive comparison between our architecture and the two discussed architectures. Table 3 Comparison of CNN feature extractor architectures. Feature ResNet-18 HTR-VT Ours Input channels 3 (RGB) 1 (grayscale) 1 (grayscale) Initial conv kernel 7×7, stride (2,2) 3×3, stride (2,1) 5×5, stride (1,1) Initial pooling MaxPool, stride 2 MaxPool, stride (2,1) None Residual stages 4 3 4 Blocks per stage [ 2 , 2 , 2 , 2 ] [ 2 , 2 , 2 ] [ 2 , 3 , 4 , 3 ] Total residual blocks 8 6 12 Channel progression 64→128→256→512 192→384→768 192→192→384→768 Dilated convolutions No No Yes (layer4, d = 2) Dropout in CNN No No Yes (p = 0.3) Intermediate pooling No No Yes (after layer2) Final spatial collapse Global AvgPool MaxPool (2,1) AvgPool + AdaptiveAvgPool Output dimension 512 768 768 4.2. Transformer encoder The transformer encoder simulates long-range contextual dependencies throughout the token sequence. In comparison to ViT-Base [ 29 ] and HTR-VT [ 10 ] in Table 4 , our configuration employs 8 encoder layers and 8 attention centers with a 768-dimensional embedding. We adhere to the Pre-LayerNorm formulation [ 30 ]: \(\:{\varvec{y}}^{\varvec{n}}={\varvec{x}}^{\left(\varvec{n}-1\right)}+\varvec{D}\varvec{r}\varvec{o}\varvec{p}\varvec{P}\varvec{a}\varvec{t}\varvec{h}\left(\varvec{L}\varvec{a}\varvec{y}\varvec{e}\varvec{r}\varvec{S}\varvec{c}\varvec{a}\varvec{l}\varvec{e}\left(\varvec{M}\varvec{S}\varvec{A}\left(\varvec{L}\varvec{N}\left({\varvec{x}}^{\left(\varvec{n}-1\right)}\right)\right)\right)\right)\) (1) \(\:{\varvec{x}}^{\varvec{n}}={\varvec{y}}^{\varvec{n}}+\text{DropPath}\left(\text{LayerScale}\left(\text{FFN}\left(\text{LN}\left({\varvec{y}}^{\varvec{n}}\right)\right)\right)\right)\) (2) Table 4. Transformer encoder configuration comparison. Parameter ViT-Base HTR-VT Ours Encoder layers 12 4 8 Attention heads 12 6 8 Embedding dimension 768 768 768 FFN hidden dimension 3,072 3,072 3,072 Activation function GELU GELU GELU Positional embeddings Learned 2D sinusoidal (fixed) 2D sinusoidal (fixed) Class token Yes No No LayerScale No No Yes ( \(\:1\times\:{10}^{-5}\) ) Stochastic depth No No Yes (0→0.1) QKV bias Yes Yes Yes Patch embedding Linear projection CNN (ResNet-18) CNN (ResNet-inspired) As follows are the primary configuration decisions. The depth for the Sorani script model is doubled from the 4 layers in HTR-VT to 8, as it necessitates the resolution of multiple concurrent dependencies, including diacritic-to-base character association, four positional character forms, and flowing ligature continuity. These dependencies necessitate more successive attention passes than those in Latin-script HTR. We increase the number of attention heads from 6 to 8 to facilitate parallel specialization across these distinct relationship categories. For the sake of data efficiency, we implement fixed 2D sinusoidal positional embeddings [ 31 ] to circumvent the expense of learning positional parameters on a limited dataset. To stabilize the training of the deeper encoder, we introduce LayerScale [ 32 ] with an initialization of 1×10⁻⁵, which enables each block to progressively learn its contributing factor. We employ stochastic depth [ 33 ] with a linearly increasing drop rate of 0→0.1 to regulate the subsequent encoder blocks against overfitting. Therefore, no class token is employed, as the assignment necessitates that each spatial position contribute to the CTC output sequence. 4.3. CTC decoder A linear projection is used to map each of the 128 output tokens from the 768-dimensional embedding space to K = 56 classes, which includes the CTC blank token and 55 Sorani characters and vocabulary in DASNUS, following the transformer encoder. The CTC loss [ 7 ] is employed for both inference and training, marginalizing over all valid alignment paths between the predicted and ground-truth sequences without necessitating explicit character segmentation. This property is particularly well-suited to connected cursive script. A clear evaluation of the model's intrinsic recognition capability is achieved by employing greedy decoding during inference in the absence of an external language model. 4.4. Training strategy A summary of all hyperparameters is provided in Table 5 . AdamW, which is augmented with SAM [ 34 ], is employed to optimize for flat loss landscape minima and to significantly enhance generalization on limited datasets [ 10 ]. Over 1,000 iterations, the learning rate increases linearly from 1×10⁻⁷ to 1×10⁻³, and then decreases to zero over 100,000 total iterations, following a warm-up cosine annealing schedule. An EMA with a decay of 0.9999 is maintained for the purposes of evaluation and inference. Table 5 Training hyperparameters. Hyperparameter Value Optimiser AdamW + SAM Base LR 1 × 10⁻⁷ (warm-up start) Max LR 1 × 10⁻³ LR schedule Warm-up cosine Warm-up iterations 1,000 Total iterations 100,000 Weight decay 0.5 Batch size 8 AdamW betas (0.9, 0.99) EMA decay 0.9999 Hardware 1 × RTX 4090 (24 GB) A span feature masking strategy [ 10 ] is implemented on the input token sequence prior to positional embeddings during training. This strategy replaces contiguous spans with a shared learnable mask token. We employ a mask ratio of τ = 0.3 and a maximum span length of 4 to prevent the masking of entire words in the more densely packed Sorani script, which is smaller than the values of HTR-VT (τ = 0.4, span = 8). In Table 6 , the stochastic data augmentation pipeline (p = 0.5 per augmentation) is described in detail. The input images are resized to 64×512 grayscale. Table 6 Data augmentation pipeline with parameter ranges. Augmentation Type Parameter Range Elastic Distortion α ∈ [0.5, 1.0], σ ∈ [ 1 , 10 ], kernel ≤ 3 Perspective Warping distortion ∈ [0.0, 0.4] Morphological (Dilation/Erosion) iterations = 1, kernel ≤ 3 Color & Contrast Jitter brightness ± 0.4, contrast ± 0.4, hue ± 0.2, saturation ± 0.4 Gaussian Blur kernel ∈ [ 3 , 5 ], σ ∈ [ 3 , 5 ] Sharpening strength ∈ [0, 1], alpha ∈ [0, 1] Zoom (H/W scaling) H ∈ [0.8, 1.0], W ∈ [0.99, 1.0] Span Masking mask ratio = 0.3, max span length = 4 4.5. Regularization strategy Our model addresses the fundamental challenge of training a deep model on a limited cursive script dataset by employing a coordinated multi-level regularization strategy, as summarized in Table 7 . Each of them operates at a separate stage of the pipeline, offering complementary rather than redundant protection against overfitting. Table 7 Multi-level regularisation strategy. Technique Component Purpose Dropout (p = 0.3) CNN feature extractor Prevents co-adaptation of CNN features; reduces overfitting in the feature extraction stage Span masking (τ = 0.3, s = 4) Transformer input tokens Forces contextual learning; prevents tokens from relying solely on local information LayerScale (init = 1\times{10}^{-5}) Each transformer block Stabilises training of the 8-layer encoder; enables gradual contribution learning Stochastic depth (0→0.1) Each transformer block Regularises deeper blocks; prevents overfitting in later layers Weight decay (0.5) All parameters L2 regularisation on model weights; prevents parameter magnitude growth EMA (α = 0.9999) Model-level Temporal ensembling; smooths parameter trajectory for better generalisation Data augmentation (p = 0.5) Input images Simulates real-world variation SAM optimiser Optimisation Seeks flat minima in loss landscape; improves out-of-distribution (OOD) generalisation 5. Results and Experiments To verify the performance of our HTR‑VT adapted model, we conducted a comprehensive series of ablations on the DASNUS dataset. The ablations were aimed not only at measuring the training convergence and error rate of the model, but also its relative performance with respect to state-of-the-art HTR systems. 5.1. Progressive ablation study In order to study the relative importance of each part in our model, we have done a progressive ablation analysis. Starting with a baseline ViT architecture with normal patch embeddings and a CTC decoder, we added architectural and learning refinements incrementally. At each step, CER and WER on the validation and test set were computed. Results have been given in Table 8 . Table 8 Main ablation (progressive build-up). Variant Validation Test CER (%) WER (%) CER (%) WER (%) Baseline (ViT patch embedding only) 5.30 27.10 5.10 26.20 + CNN feature extractor 4.44 21.85 4.25 20.92 + Span masking 4.21 20.82 4.07 19.94 + Data augmentation 3.76 19.24 3.75 18.28 + Deeper transformer (6 layers) 3.82 19.17 3.76 18.32 + Deeper transformer (8 layers) 3.79 18.91 3.76 18.38 + DropPath (0.1) + LayerScale (1e–5), 4 layers 3.69 18.73 3.67 18.17 + DropPath (0.1) + LayerScale (1e–5), 8 layers 3.59 18.01 3.57 17.53 The base model achieved decent results but was not suitable for recognizing continuous strokes and ligatures common in the Sorani script, as it relied on patch embeddings. Replacing patch embeddings with our proposed feature extractor resulted in the highest improvement, as convolutional features are more suitable for preserving stroke characteristics and continuity. The model's performance was also improved using span-based masking and data augmentation. The former helped improve character reconstruction from incomplete or faded parts, while data augmentation, including distortions and contrast, improved model generalization for different styles of writing. The model's depth was increased, but it did not improve performance on test set and even resulted in slight overfitting due to the small size of the dataset. However, when using regularization techniques like DropPath and LayerScale, increasing depth was advantageous. An 8-layer model achieved the best results, demonstrating that the combination of all techniques, rather than any single technique, improved performance. 5.2. Isolated ablation study While the cumulative improvements in section 5.1 demonstrate how successive modifications contribute to the final architecture, they do not reveal the independent effect of each factor. To address this, we conducted an isolated ablation study, starting from the best-performing progressive model in section 5.1 and altering one component at a time. This allows us to disentangle the role of training stabilizers, positional encodings, masking strategies, and architectural variations in the ResNet-inspired encoder. Results are presented in Table 9 . Table 9 Isolated ablation study (one factor at a time). Variant Validation Test CER (%) WER (%) CER (%) WER (%) No EMA 3.63 18.07 3.58 17.54 Optimizer = AdamW 3.83 18.90 3.80 18.40 Learnable positional enc. 3.73 18.75 3.62 17.84 Mask ratio = 0.15 3.72 18.59 3.76 18.53 Mask ratio = 0.45 3.74 18.83 3.64 18.21 Max span length = 1 3.87 19.34 3.80 18.67 Max span length = 8 3.73 18.22 3.57 18.00 No dilation 3.83 19.26 3.76 18.45 No pool after L2 3.72 18.63 3.69 17.98 Stride = (1,1) 3.58 18.18 3.47 17.37 The performance was slightly impaired by disabling EMA, indicating its importance in stabilizing the training and improving performance through generalization. The performance was impaired by a larger margin when the SAM optimizer was replaced by AdamW. The SAM optimizer performed better because it focuses on finding flatter minima of the loss function, making the model more robust to variations of handwritten text by different writers. The performance was impaired when deterministic sinusoidal positional encodings were replaced by learnable embeddings. The sinusoidal encodings performed better because they maintain deterministic positional consistency, which is important for the ligature-rich structure of Sorani handwritten text. Experiments conducted to find the best masking approach indicated that a moderate amount of masking was necessary. Less masking was not effective because it failed to induce contextual learning. Excessive masking, however, eliminated important information. The best performance was obtained when the span lengths were moderate because they are representative of the irregularities of handwritten text. The importance of the different components of the ResNet-inspired encoder was also identified. The dilation and max pooling components of the ResNet-inspired encoder were important. The performance was impaired when either of the two was disabled. The best performance was obtained when the initial stride was (1,1) instead of (2,1). The horizontal information was important in the early stages of the network, which was necessary to recognize subtle differences such as diacritics and spacing. The best-performing configuration was a combination of a well-tuned of our CNN feature extractor, an 8-layer Transformer network, regularization, sinusoidal positional encodings, balanced masking, the SAM optimizer, and dropout. The ablation study indicates that the different components of the Sorani HTR model, both major and minor, contributed to achieving stable and accurate performance. This final setup was trained with 100,000 iterations and a scheduled learning rate. The learning rate was initially set at 0.0001 and stepped up to 0.001 during the initial 1,000 iterations so that early fast convergence was allowed (Fig. 6 a). During training, both losses on the training and validation sets had smooth and continuous decreasing trends, with loss on the training set falling from 190.69 to 7.61 and loss on the validation set from 37.81 to 9.81 (Fig. 6 b). This narrow and stable difference between these two curves also points to good generalization and little overfitting, supporting the stability in model configuration deduced in the ablation studies. Figure 6 c presents CER and WER, which also decreased with the passage of time. In iteration 1000, the CER was 19.76% and the WER was 67.74%, which decreased with the improvement of training. By iteration 50,000, the CER went below 5%, and the WER was in the mid-20%. Finally, the model had the best validation CER of 3.58% and the WER of 18.18%. 5.3. Comparison with state-of-the-art This section compares the proposed HTR model with extant state-of-the-art systems across multiple datasets. Initially, we conducted a comparison between the proposed architecture and the HTR-VT model [CITE], which served as the foundation for our methodology. HTR-VT was re-implemented and trained on the DASNUS dataset under identical conditions to guarantee a fair comparison. The baseline achieved a validation CER of 4.16% and a test CER of 4.09%, with validation and test WERs of 20.46% and 19.80%, respectively. Conversely, our modified model demonstrated superior outcomes, attaining a validation CER of 3.58%, a test CER of 3.47%, a validation WER of 18.18%, and a test WER of 17.37%. These enhancements illustrate that the modifications introduced, including the ResNet-inspired encoder with stride adjustments, span masking, data augmentation, and optimized training strategies, considerably enhance the recognition of Sorani handwriting. In order to evaluate the performance of the models for the task of generalization, the models were also tested on an additional set of 100 handwritten text lines from writers outside the DASNUS dataset. In this case, the performance of the re-implemented baseline model resulted in a CER of 7.57%, while the proposed model resulted in a CER of 6.59%. Similarly, the proposed model resulted in a WER of 31.76%, while the baseline model resulted in a WER of 35.89%. This shows that the proposed model has been revised to be more robust for the task. The results have been summarized in Table 10 . Table 10 Comparison of our adapted model with the re-implemented HTR-VT [ 10 ] on DASNUS and on an external set of 100 real-world Sorani text-lines. Our model consistently outperforms the baseline across both benchmarks. Variant Dataset Validation Test CER (%) WER (%) CER (%) WER (%) HTR-VT [ 10 ] (baseline) DASNUS 4.16 20.46 4.09 19.80 Ours (adapted model) DASNUS 3.58 18.18 3.47 17.37 HTR-VT [ 10 ] (baseline) External (100 lines) – – 7.57 35.89 Ours (adapted model) External (100 lines) – – 6.59 31.76 Additionally, we conducted a comparison between our model and a variety of Arabic handwriting recognition methods on the KHATT dataset (Table 11 ). The proposed system obtained a validation CER of 7.97% and a test CER of 5.05% without the use of any external language model or lexicon. Our proposed model attained more robust character-level accuracy while functioning as a single visual model, despite the fact that an ensemble BLSTM–CTC system [ 35 ] reported a lower WER (13.52%) as a result of language modeling and output voting. In comparison to other systems, including MDLSTM-CTC [ 36 ], CNN–BLSTM–CTC [ 19 , 37 ], and Transformer-based models [ 22 ], our proposed architecture obtained a significantly lower CER. This was primarily due to the CNN-Transformer design, which captures both local stroke features and long-range contextual dependencies. Improved computational efficiency, reduced dependence on sequential recurrence, and avoidance of handcrafted features are all advantages of this design. Further evaluation was conducted on Katib’s Pashto Text Imagebase (KPTI) [ 38 ], which showed strong cross-script generalization (Table 11 ). Our proposed model, based on the CNN, Transformer, and CTC, was found to have a validation CER of 3.04% and a test CER of 3.10%, which is a reduction of about 66% compared to the MDLSTM baseline [ 38 ]. Overall, the proposed model is shown to not only perform better than a strong baseline on the Sorani handwriting dataset but also perform state-of-the-art on a range of Arabic-script datasets, all without the need for any external linguistic resources. Table 11 Performance comparison on the KHATT and KPTI datasets. Dataset Reference Methods CER (%) KHATT [ 35 ] BLSTM + CTC 7.85% [ 36 ] MDLSTM + CTC 19.98% [ 37 ] CNN + BLSTM + CTC 19.85% [ 19 ] ResNet–BiLSTM–CTC hybrid model + 3-gram LM 13.2% [ 22 ] Transformer 18.45% Ours ViT-based model 5.05% KPTI [ 38 ] BLSTM and MDLSTM 9.22% Ours ViT-based model 3.10% 5.4. Misrecognized character analysis The character-level error analysis reveals numerous systematic defects in the model that are not apparent through aggregate CER and WER metrics alone. The model's propensity to omit characters, particularly spaces and faint letters, is one of the most significant issues. This results in merged words and structural distortions in the transcription. This behavior is plainly demonstrated in Fig. 7 a, where the model fails to detect the inter-word space in "ئامانجە سەرەکییەکە," resulting in the entire phrase being merged into a single token. The DASNUS test set frequently contains such omissions due to the highly inconsistent spacing practices of handwritten Central Kurdish, which range from wide gaps to tightly connected letters that visually resemble ligatures. Dot-based confusions between visually similar characters are another significant source of error. Letters such as ت, ن, ب, ی, and ز are primarily distinguished by the number or location of their dots, which are frequently overlooked in negligent or artistic handwriting styles. The most misrecognized characters presented in Fig. 8 , as the confusion pairs ت, ن, and ب, ی are among the most prevalent. This type of error is frequently the result of the dots being either indistinguishable or absent in the original handwriting, which renders the letters nearly identical during the handwriting process. A practical example of this type of error is illustrated in Fig. 7 b, where the indistinguishable form of ز leads to the confusion of تەرکیز with تەرکیت. Another observation is that the model overpredicts; therefore, the model introduces characters that do not exist. This is attributed to the model's inability to differentiate between character strokes and noise. The inability of the model to recognize punctuation marks, especially dots, is an indication of the model's low sensitivity to small details. It is also evident from the observations that the model can easily miss a character or add an extra character even if the model is doing extremely well. The aforementioned observations broadly classify the model's misrecognition of characters into three categories: overpredicted characters, dot-based confusions, and missing characters. Based on the aforementioned observations, it is evident that the model's misrecognition of characters is attributed to the complexity of Kurdish cursive handwriting and the model's inability. The model's accuracy can be improved by implementing the proposed solutions, which include a better context sequence model to leverage the characteristics of Kurdish words, an enhanced attention model to increase sensitivity to small details, and the incorporation of dot recognition. 5.5. Discussion of results and findings The performance obtained by our adapted HTR-VT model on the DASNUS dataset demonstrates a major improvement in the field of CKB handwriting recognition. The model not only generalizes very well on unseen text, with its CER of 3.47% and WER of 17.37% on the last test set, but also surpasses many of the current methods formulated for other cursive and semitic languages. Through these experiments, it can be inferred that certain architectural decisions along with the training process we used played a valuable role in enhancing the recognition performance, especially in reducing the number of character-level misclassifications. One of the most important contributing elements to the low CER was the our HTR-VT architecture's design, which makes use of a vision-transformer backbone configured for the specific visual and spatial characteristics of Sorani handwriting. This is in contrast to standard CNN-based models that can be prone to the minute differences in curvature, shape, and ligature position typical of cursive scripts. The transformer architecture is well-suited to handle long-range relationships as well as fine-grained differences along sequences of visual tokens. This enabled the model to attempt to better discriminate among characters that are very similar in shape but varying in position, context, or small diacritics. Another major reason for this performance is the quality and size of the DASNUS dataset. The DASNUS dataset, being a large dataset of Sorani handwritten text-line images, offered a wide range of ligatures of characters, handwriting styles, and inter-word space variability. The model was thus subjected to a wide range of writing styles. Furthermore, since the model was trained on the entire text-line images and not just the cropped characters, this approach helped to reduce confusion between morphologically ambiguous characters. The strategy of training itself was also pivotal. Early on, aggressive learning rate (up to 0.001) helped the model quickly acquire global features of the handwriting. Later on, a gradual decay of the learning rate enabled the model to refine its weights subtlety, attending to fine details without deranging previously acquired patterns. This resulted in a smooth, stable decline of both training loss and validation loss, with very little overfitting, as indicated by the close correspondence of validation and test performance. Our emphasis on CER in particular was intentional, since CER is a smaller-grained measure of recognition quality compared to WER, particularly in richly inflected languages such as CKB. In most real-world applications such as manuscript digitization or Optical Character Recognition (OCR)-based search, partial word recognition is useful even if most of the characters are correctly recognized. Thus, low CER is a good sign of usefulness in subsequent tasks such as indexing, archiving, or helping preserve languages. Compared to models we studied in 5.4 Comparison with State-of-the-Art, we can see that our architecture fits better with the structure and linguistic requirements of Sorani handwriting. Other models with good performance on Latin or Arabic were found wanting as soon as they are applied to scripts with complicated joining behavior or varied stroke representations. Our HTR-VT's use of attention mechanisms and global receptive fields addresses this by allowing the model to acquire context-sensitive character embeddings. Lastly, we highlight that transformer-based models, in conjunction with sufficient data variance, customized preprocessing, and curriculum-based methods, can redefine the benchmark for HTR in lesser-represented languages such as Kurdish. These findings not only provide evidence of our HTR-VT model's viability but also are a strong justification for future work on deep-learn-based script-specific models as well as multilingual HTR. While our writer-disjoint evaluation provides a fair measure of generalization to unseen handwriting styles, a limitation of the present study is the absence of an external OOD test set collected under different acquisition conditions (e.g., new writers, varied scanning devices, alternative prompts). Such an OOD benchmark would allow us to measure robustness under truly novel scenarios. We plan to address this in future work by expanding DASNUS with additional forms collected from different environments and contributors. While our evaluation demonstrates strong performance under writer-disjoint splits of DASNUS, broader generalization analyses remain an important direction. In particular, zero-shot testing on unseen Kurdish handwriting (e.g., new page formats, layouts, or writing prompts), one-shot and few-shot writer adaptation using Parameter-Efficient Fine-Tuning (PEFT) techniques such as adapters or LoRA, and cross-script transfer (e.g., Arabic or Persian pretraining followed by Sorani adaptation) would provide stronger evidence of robustness and adaptability. These evaluations were not included in the present work due to resource and dataset limitations, but they represent natural next steps in future research. We anticipate that such studies would reveal how much transferable structure exists across cursive scripts and whether parameter-efficient fine-tuning can further improve writer adaptation in low-resource contexts. 6. Conclusion and Future Work This paper proposed a novel deep learning method for Central Kurdish Language (CKB) handwriting recognition with a ViT-based model structure modified for the task. Addressing the long-standing challenge of low-resource script recognition, particularly for a linguistically dense and visually complex script like CKB, we developed the DASNUS dataset—a high-quality, diverse, and annotated collection of over 11,000 handwritten text-line images from 867 writers. The dataset is a groundbreaking achievement, as it is the largest public dataset that is specifically designed for Sorani handwriting recognition. Our modified model, which features a convolutional backbone inspired by ResNet and a span-masked transformer encoder, exhibited a robust ability to learn the global contextual relationships and local stroke information of CKB handwriting. We were able to effectively address ligature, position-sensitive glyph, and handwriting variation issues by structuring the model with careful consideration of horizontal continuity and span-based regularization at training time. Favorable results were obtained in these experiments, as the adapted model attained a CER of 3.47% and a WER of 17.37% on the held-out test set. These rates are almost identical to the validation results, which shows that the model generalizes the training data much more than it overfits. It is important to take these results into consideration. Though it doesn't make use of synthetic inputs nor of pretraining on enormous corpora, its accuracy still stands comparably to the state-of-the-art systems trained with other languages. That in itself testifies to the stability of the architecture—particularly in the context of low-resource settings—and belies the promise of ViTs in dealing with the idiosyncrasies of cursive script. Looking ahead, A number of avenues suggest themselves: Multilinguality: Future releases may provide support to other Kurdish dialects—or even totally related scripts such as Pashto or Uyghur—by transferring knowledge across related languages. Synthetic Data Creation: There also exists future promise in creating artificial Central Kurdish handwriting samples, potentially with GANs or with style-transfer methods, to augment the model to generalize better across diversified handwriting systems. Writer Identification and Adaptation: In the light of DASNUS's population richness, the next step shall comprise writer-adaptive identification systems or a fusion of writer identification and handwriting recognition. Multimodal Fusion: Handwriting with either oral modality or auditory data could potentially allow for more complete transcription systems to preserve the Kurdish language. In this research, we have proposed a ViT-based model of CKB handwriting recognition and have presented the DASNUS dataset. The new system produces rivaling outcomes with no dependency on linguistic priors or auxiliary lexicons. In future research, we intend to incorporate a lightweight language model or lexicon-based decoder to minimize even more the WER and enhance linguistic normality. In addition, we intend to extend comparative analyses by incorporating newer transformer-based models in an effort to put our approach in more direct competition with newer state-of-the-art models. We also intend to investigate multilingual transfer learning and cross-pretraining scripts approaches to improve recognition robustness in other cursive scripts with limited resources. Briefly, this work lays a strong foundation for Kurdish handwriting recognition with state-of-the-art deep learning approaches and offers seminal resources and approaches that can be extended by the research community. It is a significant step toward balanced coverage of low-resource languages for AI-driven document processing and pattern detection systems. Declarations Participant consent statement: All participants who contributed handwriting samples to the DASNUS dataset were informed about the purpose of the study and voluntarily agreed to participate. Written informed consent was obtained from the participants prior to data collection. The collected forms contained only demographic information and handwriting samples, and no personally identifying information is included in the published dataset or the study. The study procedures followed institutional research and ethical guidelines. Competing interests : The authors declare no competing interests. Funding: this research did not receive any funding. Author Contribution FARAEDWN M. SALIH led the study's design, data collection, methodology, deep learning implementation, experiments, software, and manuscript drafting. ABDULBASIT K. AL-TALABANI contributed to data collection, validated and analyzed results, supervised the research, and reviewed the manuscript. Both authors approved the final version. Data Availability The dataset used in this study is publicly available at: https://data.mendeley.com/datasets/xdj9f55rkm/1 References Afkari-Fahandari, A., Shabaninia, E., Asadi-Zeydabadi, F., Nezamabadi-Pour, H.: A Comprehensive Survey of Transformers in Text Recognition: Techniques, Challenges, and Future Directions. ACM Comput. Surv. 58 (1–111), 42 (2025). https://doi.org/10.1145/3771273 Romein, C.A., Rabus, A., Leifert, G., Ströbel, P.B.: Assessing advanced handwritten text recognition engines for digitizing historical documents. Int. J. Digit. Humanit. 7 , 115–134 (2025). https://doi.org/10.1007/s42803-025-00100-0 Bamoki, M., Wady, S.H., Badawi, S.: Data Brief. 60 , 111533 (2025). https://doi.org/10.1016/j.dib.2025.111533 Holy Quran Kurdish Sorani translation dataset for language modelling Abdalla, P.A., Shakor, M.Y., Ameen, A.K., Hassan, D.A.: Critical review of the model description in ‘Kurdish handwritten character recognition using deep learning techniques’. Gene Expr. Patterns. 56 , 119399 (2025). https://doi.org/10.1016/j.gep.2025.119399 Abdalla, P.A., Qadir, A.M., Shakor, M.Y., Saeed, A.M., Jabar, A.T., Salam, A.A., Amin, H.H.H.: A vast dataset for Kurdish handwritten digits and isolated characters recognition. Data Brief. 47 , 109014 (2023). https://doi.org/10.1016/j.dib.2023.109014 Qader, A., Rashid, I.: T.A.: A Comprehensive Dataset of Complete Kurdish Handwritten Characters and Digits. 1, (2024). https://doi.org/10.17632/shny5k9f4b.1 Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369–376. Association for Computing Machinery, New York, NY, USA (2006) Kizilirmak, F., Yanikoglu, B.: CNN-BiLSTM model for English Handwriting Recognition: Comprehensive Evaluation on the IAM Dataset, (2023). http://arxiv.org/abs/2307.00664 Chan, A., Mijar, A., Saeed, M., Wong, C.-W., Khater, A.: HATFormer: Historic Handwritten Arabic Text Recognition with Transformers, (2025). http://arxiv.org/abs/2410.02179 Li, Y., Chen, D., Tang, T., Shen, X.: HTR-VT: Handwritten text recognition with vision transformer. Pattern Recogn. 158 , 110967 (2025). https://doi.org/10.1016/j.patcog.2024.110967 Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT - DATABASE OF HANDWRITTEN ARABIC WORDS. In Proc. of CIFED. 127–136 (2002) Mahmoud, S.A., Ahmad, I., Al-Khatib, W.G., Alshayeb, M., Tanvir Parvez, M., Märgner, V., Fink, G.A.: KHATT: An open Arabic offline handwritten text database. Pattern Recogn. 47 , 1096–1112 (2014). https://doi.org/10.1016/j.patcog.2013.08.009 Safarzadeh, V.M., Jafarzadeh, P.: Offline Persian Handwriting Recognition with CNN and RNN-CTC. In: 2020 25th International Computer Conference, Computer Society of Iran (CSICC). pp. 1–10 (2020) Hassan, S., Irfan, A., Mirza, A., Siddiqi, I.: Cursive Handwritten Text Recognition using Bi-Directional LSTMs: A Case Study on Urdu Handwriting. In: 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications (Deep-ML). pp. 67–72 (2019) Mohammed, T.L., Ahmed, A.A., Al-Sanjary, O.I.: KRDOH: Kurdish Offline Handwritten Text Database. In: 2019 IEEE 7th Conference on Systems, Process and Control (ICSPC). pp. 86–89 (2019) Ahmed, R.M., Rashid, T.A., Fatah, P., Alsadoon, A., Mirjalili, S.: An extensive dataset of handwritten central Kurdish isolated characters. Data Brief. 39 , 107479 (2021). https://doi.org/10.1016/j.dib.2021.107479 Alsaqi, I.M., Fattah, P.: Kurdish Handwritten Word Recognition withCRNN Using Multi-Backbone Evaluation: Kurdish Handwritten Word Recognition. Acad. J. Int. Univ. Erbil. 2 , 439–449 (2025). https://doi.org/10.63841/iue24539 Maalej, R., Kherallah, M.: Convolutional Neural Network and BLSTM for Offline Arabic Handwriting Recognition. In: 2018 International Arab Conference on Information Technology (ACIT). pp. 1–6 (2018) Mutawa, A.M., Allaho, M.Y., Al-Hajeri, M.: Machine Learning Approach for Arabic Handwritten Recognition. Appl. Sci. 14 , 9020 (2024). https://doi.org/10.3390/app14199020 Ahmed, R.M., Rashid, T.A., Fattah, P., Alsadoon, A., Bacanin, N., Mirjalili, S., Vimal, S., Chhabra, A.: Kurdish Handwritten character recognition using deep learning techniques. Gene Expr. Patterns. 46 , 119278 (2022). https://doi.org/10.1016/j.gep.2022.119278 Mostafa, A., Mohamed, O., Ashraf, A., Elbehery, A., Jamal, S., Khoriba, G., Ghoneim, A.S.: OCFormer: A Transformer-Based Model For Arabic Handwritten Text Recognition. In: 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC). pp. 182–186 (2021) Momeni, S., BabaAli, B.: A transformer-based approach for Arabic offline handwritten text recognition. SIViP. 18 , 3053–3062 (2024). https://doi.org/10.1007/s11760-023-02970-9 Kass, D., Vats, E.: AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks. In: Uchida, S., Barney, E., Eglin, V. (eds.) Document Analysis Systems, pp. 507–522. Springer International Publishing, Cham (2022) Al-Ma’adeed, S., Elliman, D., Higgins, C.A.: A data base for Arabic handwritten text recognition research. In: Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition. pp. 485–489 (2002) Mezghani, A., Kanoun, S., Khemakhem, M., Abed, H.E.: A Database for Arabic Handwritten Text Image Recognition and Writer Identification. In: 2012 International Conference on Frontiers in Handwriting Recognition. pp. 399–402 (2012) Goh, T.Y., Basah, S.N., Yazid, H., Aziz Safar, M.J., Ahmad Saad, F.S.: Performance analysis of image thresholding: Otsu technique. Measurement. 114 , 298–307 (2018). https://doi.org/10.1016/j.measurement.2017.09.052 Li, S., Shen, Q., Sun, J.: Skew detection using wavelet decomposition and projection profile analysis. Pattern Recognit. Lett. 28 , 555–562 (2007). https://doi.org/10.1016/j.patrec.2006.10.002 He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. Presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Presented at the International Conference on Learning Representations October 2 (2020) Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., Liu, T.: On Layer Normalization in the Transformer Architecture. In: Proceedings of the 37th International Conference on Machine Learning. pp. 10524–10533. PMLR (2020) He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked Autoencoders Are Scalable Vision Learners. Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022) Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., Jégou, H.: Going Deeper With Image Transformers. Presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision (2021) Huang, G., Sun, Y., Liu, Z., Sedra, D., Weinberger, K.Q.: Deep Networks with Stochastic Depth. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016, pp. 646–661. Springer International Publishing, Cham (2016) Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-Aware Minimization for Efficiently Improving Generalization, (2021). http://arxiv.org/abs/2010.01412 Jemni, S.K., Kessentini, Y., Kanoun, S., Ogier, J.-M.: Offline Arabic Handwriting Recognition Using BLSTMs Combination. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). pp. 31–36 (2018) Ahmad, R., Naz, S., Afzal, M., Rashid, S., Liwicki, M., Dengel, A.: A Deep Learning based Arabic Script Recognition System: Benchmark on KHAT. IAJIT. 17 , 299–305 (2020). https://doi.org/10.34028/iajit/17/3/3 Lamtougui, H., Moubtahij, E., Fouadi, H., Satori, H.: An Efficient Hybrid Model for Arabic Text Recognition. Computers Mater. Continua. 74 , 2871–2888 (2023). https://doi.org/10.32604/cmc.2023.032550 Ahmad, R., Afzal, M.Z., Rashid, S.F., Liwicki, M., Breuel, T., Dengel, A.: KPTI: Katib’s Pashto Text Imagebase and Deep Learning Benchmark. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 453–458 (2016) Footnotes https://data.mendeley.com/datasets/xdj9f55rkm/1 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9225627","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":613066548,"identity":"ab9f2705-e481-4edb-b400-08b28652997f","order_by":0,"name":"Faraedwn M. Salih","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABB0lEQVRIiWNgGAWjYBAC9gYQycYAJxn7QWRCAW4tPAfQtcwEGZJgQIqWDWARfFrYzz788KHscB6D2OFnj25U2MhuPr868cMDAwZ5frED2LXwpBtLzjh3uJhBOs3cOOdMmvG2G283SwAdZjhzdgJWLfYMaQzSvG23ExukE8ykc9sOJ267cXYDSEuCwW3sWnj4nzH//gvWkv4NrGXzjLObf+DVIpHGJs0I1pIDsWUDf+82/LZIPGOz7Dn3P7FNOqdMGuSXGTd4t1kkGEjg9AsPfxrzjR9laYn90unbpHOAIdbff3bzzR8VNvL80ti1wAEbnCUBVimBXzkq4D9AiupRMApGwSgYAQAAf2BgxDgIldEAAAAASUVORK5CYII=","orcid":"","institution":"Koya University","correspondingAuthor":true,"prefix":"","firstName":"Faraedwn","middleName":"M.","lastName":"Salih","suffix":""},{"id":613066549,"identity":"851223fc-5a1c-40e1-8cf3-6a7d2877f7e2","order_by":1,"name":"Abdulbasit K. Al-talabani","email":"","orcid":"","institution":"Koya University","correspondingAuthor":false,"prefix":"","firstName":"Abdulbasit","middleName":"K.","lastName":"Al-talabani","suffix":""}],"badges":[],"createdAt":"2026-03-25 16:38:31","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9225627/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9225627/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106259274,"identity":"d068eadf-899e-4ad1-af7d-8d7b99736fd5","added_by":"auto","created_at":"2026-04-06 20:14:47","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":337761,"visible":true,"origin":"","legend":"\u003cp\u003eCommon challenges of Sorani cursive handwriting.\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/564be8c36bf7482843b270ad.jpeg"},{"id":106259275,"identity":"1491bd1c-187f-4e8a-8ea7-693a2352baf2","added_by":"auto","created_at":"2026-04-06 20:14:47","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":4233918,"visible":true,"origin":"","legend":"\u003cp\u003eA representative sample of the DASNUS handwriting data collection form. Each form consists of four pages: the first records writer demographic information, while the remaining three pages contain Kurdish text prompts that participants were instructed to transcribe by hand using a pen in a their own writing style.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/3b0ba6922aafee49fee7dc02.png"},{"id":106403758,"identity":"611f73cf-b5e2-488b-8765-24abc7fac3a8","added_by":"auto","created_at":"2026-04-08 09:14:55","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1930773,"visible":true,"origin":"","legend":"\u003cp\u003e(a) Skew correction applied to incorrectly skewed paragraphs. (b) Text-line segmentation steps from a handwritten paragraph sample.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/e9ce6ad76c6321937a4e743c.png"},{"id":106403667,"identity":"c869eb6c-3409-4ab7-9564-089f94ddfff9","added_by":"auto","created_at":"2026-04-08 09:14:44","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":237178,"visible":true,"origin":"","legend":"\u003cp\u003eOur modified ViT-based model architecture.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/615236f0d7725e24b4e3ac08.png"},{"id":106259276,"identity":"c62a211d-0ae4-4ea8-8ec8-474f3ae8cbcb","added_by":"auto","created_at":"2026-04-06 20:14:47","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":182564,"visible":true,"origin":"","legend":"\u003cp\u003eOur ResNet-inspired CNN feature extractor architecture.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/8b0597ff4bcd30f375ed1873.png"},{"id":106403236,"identity":"539ca186-3291-45e8-bfe9-3736597d4776","added_by":"auto","created_at":"2026-04-08 09:13:56","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":2219333,"visible":true,"origin":"","legend":"\u003cp\u003e(a) Learning rate schedule. (b) Training and validation loss functions. (c) Performance of the model in CER and WER during training and validation progress.\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/8acf62e454628a27d3ce771f.png"},{"id":106259278,"identity":"42333b1d-5c54-4cce-baf9-978c5c4a76fb","added_by":"auto","created_at":"2026-04-06 20:14:47","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":447420,"visible":true,"origin":"","legend":"\u003cp\u003eSome recognition examples from the DASNUS test set, showing the handwritten image, ground-truth transcription (green), and model prediction (red) for three samples. (a) (CER = 5.56%, WER = 40.00%): inter-word space omission merges \"ئامانجە سەرەکییەکە\" into a single token. (b) (CER = 12.07%, WER = 70.00%): multiple diacritic substitutions and a spurious character insertion produce the highest error rate among the examples.\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/ff77860026fc1e3ea9295cd4.png"},{"id":106259280,"identity":"30e63053-efd0-4f99-82cf-dde16e1d518d","added_by":"auto","created_at":"2026-04-06 20:14:47","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":68818,"visible":true,"origin":"","legend":"\u003cp\u003eMost misrecognized characters.\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/f990091159cbd5be43eecdcf.png"},{"id":108490565,"identity":"d9cfc9d4-2d21-4a4e-925a-8869611b7236","added_by":"auto","created_at":"2026-05-05 09:44:28","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":10377278,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9225627/v1/ce601aee-b346-475e-8dda-af643c105560.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Modified Vision Transformer for Kurdish Cursive RTL Handwritten Text Recognition","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eIn the fields of computer vision and pattern recognition, handwriting recognition remains one of the most technically challenging tasks [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Its applications vary from digitization of historical manuscripts to assistive technologies for people with disabilities [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Despite significant advances for well-resourced languages like English, Arabic, Chinese, and German, there is still a large gap in research and tools for a significant number of written languages. Central Kurdish (Sorani), also known as CKB, is one of them.\u003c/p\u003e \u003cp\u003eSorani is used by 12\u0026ndash;14\u0026nbsp;million people in Iraq and Iran [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. It is written with a modified Arabic alphabet and has the same cursive writing direction and characters connected to each other from right to left like Arabic and Persian. However, it also contains unique characters for distinct sounds of the Kurdish language: ڤ, ڵ, پ, چ, ژ, ڕ, ۆ, and گ [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. As a result, the Sorani alphabet contains 34 characters compared to 28 for Arabic and 32 for Persian. This makes handwriting recognition for Sorani significantly more difficult than for Latin and roughly Arabic scripts. Some of these difficulties include context-dependent characters with up to four different forms for some of them [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], long horizontal ligatures where two or more characters are connected to each other to form a single unit, and diacritical marks, which tend to be faint and misplaced in handwriting. Additionally, there is a lot of variation between writers in terms of pressure and stroke width and spacing (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eSignificant advancements in handwriting recognition have been made on languages with large annotated datasets. The advancements have evolved from Convolutional Neural Network \u0026ndash; Recurrent Neural Network (CNN-RNN) architectures with Connectionist Temporal Classification (CTC) loss functions [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] to Vision Transformer (ViT) architectures with self-attention mechanisms for connected cursive scripts [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. However, as far as we know, no publicly available annotated dataset for Central Kurdish handwriting at the line level was available before this research, and no previous research utilized a ViT-based architecture for handwriting recognition on the Sorani script at the line level.\u003c/p\u003e \u003cp\u003eThis paper contributes to filling these research gaps in two aspects: a dataset and a model. First, we introduce a large-scale dataset for Central Kurdish handwriting. Second, we propose a modified ViT-based model for handwriting recognition on the Sorani script and also works well for other cursive scripts, achieving a Character Error Rate (CER) of 3.47% and a Word Error Rate (WER) of 17.37% on our proposed dataset, which are comparable with or even better than the state-of-the-art results for Arabic, Persian, and English handwriting recognition.\u003c/p\u003e \u003cp\u003eThe rest of the paper is structured as follows: Section 2 reviews related works on handwriting recognition and transformer-based architectures for handwriting recognition. In Section 3, we introduce the proposed dataset for Central Kurdish handwriting recognition, named DASNUS. In Section 4, we explain the proposed model for handwriting recognition on the Sorani script, and in Section 5, we report the experimental results on the proposed dataset and some fair comparisons. In Section 6, we conclude this paper.\u003c/p\u003e"},{"header":"2. Related Work","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1. Datasets for cursive script handwriting recognition\u003c/h2\u003e \u003cp\u003eThe development of handwriting recognition systems has largely depended on the availability of standardized datasets. For Arabic, the IFN/ENIT database [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] is one of the most widely used, consisting of 26,459 handwritten Tunisian town and village names from 411 writers. This dataset has become a benchmark for Arabic handwriting recognition, enabling CNN\u0026ndash;BLSTM\u0026ndash;CTC systems to achieve recognition rates above 92%. Another critical resource is the KHATT dataset [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e], which contains 165,890 words, 589,924 characters, and 9,326 lines of text contributed by 1,000 different writers, with annotations at both the word and line level. The training of deep models that are resistant to handwriting variations has been facilitated by the availability of such datasets.\u003c/p\u003e \u003cp\u003eCurated resources have also been advantageous for Persian handwriting recognition. The dataset introduced by the authors of [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] allowed CNN\u0026ndash;BLSTM\u0026ndash;CTC models to obtain a 99.35% accuracy in character recognition. In the same vein, Urdu recognition has progressed with datasets of handwritten text lines, such as the 6,000-line dataset utilized in the research paper of [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], which facilitated the development of segmentation-free sequence modeling approaches.\u003c/p\u003e \u003cp\u003eConversely, Kurdish continues to be underfunded. The KRDOH dataset [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] comprises 4,304 scanned pages, which amount to 93,612 words and 17,466 text lines from 1,076 authors. Although this is a significant contribution, its potential for end-to-end recognition is restricted by the absence of word-level annotation. Other Kurdish resources concentrate on isolated characters or numerals. More than 315,000 images of digits and isolated characters are included in the K-ZHMARA and K-PIT datasets [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e], while the dataset in [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] contains 40,940 images of individual Kurdish letters that are evenly distributed across the alphabet. These datasets are indispensable for character-level recognition; however, they fail to address the obstacle of continuous, cursive handwriting recognition. At the word level, the KHWD dataset [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] provided a notable improvement at the word level by obtaining approximately 400,000 handwritten word images from about 8,000 writers containing 10,000 Sorani Kurdish vocabulary words; this new approach provides evidence supporting the significantly greater challenge in cursive recognition of handwritten words versus isolated characters\u0026mdash;the KHWD produced a best WER of 45.68% using an enhanced version of the MobileNetV2 model.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2. CNN\u0026ndash;RNN architectures\u003c/h2\u003e \u003cp\u003eIn Arabic, Persian, and Urdu, the use of hybrid architectures of CNN, BLSTM, and CTC has been highly significant in the field of Handwritten Text Recognition (HTR). Hassan et al. [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] proposed a CNN-BLSTM architecture with CTC loss for the case of Urdu, which attained 83.69% accuracy in recognizing characters without explicit segmentation. However, the system faced problems due to the high similarity between classes and the complex placement of diacritic marks, which also applies in the case of Kurdish text.\u003c/p\u003e \u003cp\u003eMaalej and Kherallah [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] also attained high accuracy in recognizing characters by using a combination of CNNs for feature extraction and BLSTM for sequence modeling on the IFN/ENIT dataset for Arabic. In a subsequent study, Mutawa et al. [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] improved the accuracy of this approach by using a combination of ResNet for feature extraction and BLSTM with CTC for sequence modeling, which attained a CER of 13.2% and a WER of 27.31% on the KHATT dataset. This also proves the effectiveness of complex architectures of CNN-RNN, though with high dependence on large datasets.\u003c/p\u003e \u003cp\u003eIn the paper [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], the authors were able to prove that a CNN-BLSTM-CTC architecture can attain a high accuracy of 99.35% in the case of Persian, thereby eliminating the need for explicit segmentation in the case of Persian text. In Kurdish, however, attempts have been limited. Ahmed et al. [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] proposed a CNN model trained on 40,940 isolated characters, achieving 83% test accuracy, but the model did not extend to word- or line-level recognition. Similarly, the authors in reference [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] focused only on character-level modeling, leaving the sequential dependencies of connected text largely unaddressed.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3. Transformer-based approaches\u003c/h2\u003e \u003cp\u003eTransformer-based models have also been considered in HTR in light of learning long-range dependencies in connection with recurrent connections. In the case of Arabic, the researchers in reference [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] proposed a CNN\u0026ndash;Transformer architecture hybrid (OCFormer) based on convolutional embeddings and self-attention. Performance was high, but it was dependent on a humongous set of synthetic data of over 30\u0026nbsp;million images, which posed applicability issues in hard-resource regimes.\u003c/p\u003e \u003cp\u003eMore recent research has investigated applying ViTs to handwriting text recognition (HTR). In [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] span feature masking was added to the boosting of contextual learning, and the architecture proposed was able to surpass CNN\u0026ndash;BLSTM baselines on English and German data, illustrating the promise of ViTs in handwriting recognition. In another research paper [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e], it was suggested substituting CNN\u0026ndash;RNN-based architecture with full Transformer-based architecture and achieving the best results on the KHATT dataset. This, however, needed to have been done with much pretrained knowledge and computational power.\u003c/p\u003e \u003cp\u003eAttention-based sequence-to-sequence models have also been tried. In [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] the authors showed that they perform well on handwriting recognition but showed substantial drops in performance on dense cursive scripts where only small amounts of training data could be had. All these works point to the conclusion that while Transformers give state-of-the-art performance, they require high amounts of labeled or synthesized data \u0026mdash; which happens to be a big concern in Kurdish, where such data still do not exist.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e2.4. Comparative analysis: Arabic, Persian, Urdu vs. Kurdish\u003c/h2\u003e \u003cp\u003eComparison between related scripts finds both similarities and exigent challenges. In Urdu, authors of paper [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] exemplified that segmentation-free CNN\u0026ndash;BLSTM model-based systems could process at the stroke level while being tripped up by dot placement and letter likeness, both of which also appear in Kurdish. In Arabic scripts, IFN/ENIT [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] and KHATT [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] datasets allowed CNN\u0026ndash;BLSTM\u0026ndash;CTC-based systems to break the 92%-threshold in terms of accuracy. The authors in [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e] presented the AHDB dataset in highlighting the value of standardized corpora in the development of the field in Arabic HTR. Mezghani et al. [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] similarly presented the multi-writer corpus in the form of AHTID/MW to accentuate its value in real-world writer-independent identification. Persian-based research in reference [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] presented that with deeply learned CNN\u0026ndash;BLSTM\u0026ndash;CTC systems, it was possible to get close to perfect accuracy upon being trained on finely curated datasets.\u003c/p\u003e \u003cp\u003eKurdish, on the other hand, poses unique challenges. While in comparison to Arabic or Persian, the script of Kurdish excludes some characters such as ڕ, ڵ, ۆ, and ژ that exist in others related scripts. More seriously, no big-scale, word-level, or line-level dataset comparable to IFN/ENIT or KHATT is available. Although KRDOH [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] represents a progression, it has no word annotation, and data such as K-ZHMARA [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] and the one in reference [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] contain only isolated symbol-based data. Therefore, Kurdish significantly trails Arabic, Persian and even Urdu in available dataset richness and benchmark material.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e2.5. Motivation and research gap\u003c/h2\u003e \u003cp\u003eFrom the above review, several conclusions emerge:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eCNN\u0026ndash;BLSTM\u0026ndash;CTC architectures [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] have proven highly effective for cursive handwriting recognition, but their performance depends on large, well-curated datasets.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eTransformer-based approaches outperform CNN-RNN architectures for long sequence handling but need enormous amounts of training data.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eArabic, Persian, and Urdu have the advantage of standardized datasets like IFN/ENIT [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e], KHATT [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e], and AHDB [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], while Kurdish only has character- or word-based datasets.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eThere are script-specific challenges for Central Kurdish handwriting recognition: more characters, diacritic marks, and positional variations.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eThis establishes a distinct research gap. Existing CNN\u0026ndash;RNN or Transformer approaches cannot be directly applied to Kurdish due to its low-resource status and script-specific complexity. Our work addresses this by developing a ViT-based tailored for Kurdish text-line recognition, and contributing toward dataset expansion, thereby bridging a critical gap in low-resource handwriting recognition research.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. The DASNUS Dataset","content":"\u003cp\u003eThe DASNUS dataset was collected as a part of this study and to the best of our knowledge, DASNUS (دەستنووس \u0026mdash; Central Kurdish for handwriting manuscript) is the largest publicly available dataset for recognizing handwritten Central Kurdish (Sorani) text. We made the dataset open to the public\u003csup\u003e1\u003c/sup\u003e.\u003c/p\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.1. Data collection\u003c/h2\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eA total of 867 writers from the four main governorates of the Kurdistan Region of Iraq volunteered to write, and there were no restrictions on the type of pen or style of writing. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e show a summary of the demographic information. Each writer filled out a structured four-page form (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). This included a demographic page, a fixed paragraph that make sure all the letters of the Sorani alphabet, digits, and some common symbols were covered, and two randomly assigned paragraphs from a pool of 2,500 distinct Kurdish writings in page 3, and a free-writing portion in page 4. Out of 1,250 forms sent out, 867 were filled out completely and sent back, which made 3,468 pages.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDemographic distribution of DASNUS writers (N\u0026thinsp;=\u0026thinsp;867).\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCategory\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSubgroup\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCount\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePercentage (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eAge\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eBelow 15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e11.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e16\u0026ndash;25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e591\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e68.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e26\u0026ndash;50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e156\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e18.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMore than 50\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eAddress\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eErbil\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e305\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e35.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSulaymaniyah\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e507\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e58.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eKirkuk\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDuhok\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eEducation Level\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrimary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSecondary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e113\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e13.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eHigh-School\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e163\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e18.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUniversity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e558\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e64.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eGender\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e538\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e62.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e329\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e37.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eHandedness\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRight-handed\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e765\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e88.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLeft-handed\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e102\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e11.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e3.2. Scanning and preprocessing\u003c/h2\u003e \u003cp\u003eTo keep the delicate nuances of the script, forms were scanned at 600 DPI. Using Otsu's thresholding [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e], morphological closing, contour detection, and skew correction through projection profile analysis [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e], an OpenCV-based pipeline subsequently binarized the data and created 11,475 annotated line pictures. The process is visually presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e3.3. Annotation and quality control\u003c/h2\u003e \u003cp\u003eNative speakers of the Sorani language manually transcribed each line image verbatim into UTF-8. Inter-annotator agreement was 99.1% (character-level) and 97.6% (word-level) in the double-annotation of 500 randomly sampled lines. All disagreements were resolved by an expert annotator, resulting in an overall transcription accuracy that surpassed 98%.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e3.4. Dataset splits\u003c/h2\u003e \u003cp\u003eA writer-disjoint strategy is employed to divide DASNUS into training, validation, and test sets (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The agglutinative morphology of Sorani Kurdish is reflected in the out-of-vocabulary (OOV) rate of approximately 47% for both evaluation splits, which explains the greater WER disparity that was observed in our experiments.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDataset splits and vocabulary statistics.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSplit\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e#Writers\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e#Lines\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e#Unique Words\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eVocabulary Size\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eOOV (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTrain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e606\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8042\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e10702\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e10702\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e131\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1700\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3298\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e12258\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e47.18\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTest\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e130\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1733\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3532\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e13817\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e47.28\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"4. Methodology","content":"\u003cp\u003eWe have modified a handwriting recognition model that is based on ViT and was expressly designed for the recognition of Central Kurdish (Sorani) cursive text lines. This model can also be used to train on other cursive scripts. An architecture composed of three components: a CNN feature extractor inspired by ResNet, a transformer encoder, and a CTC decoder. The complete architecture is depicted in Fig. \u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n \u003ch2\u003e4.1. CNN feature extractor\u003c/h2\u003e\n \u003cp\u003eWe utilize a custom CNN front-end that is based on the residual learning framework of He et al. [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], as opposed to the conventional patch embedding of the original ViT [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Two factors motivate this decision: (1) pure ViTs lack the spatial inductive bias necessary for stable convergence on small datasets, as demonstrated by Li et al. [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], who demonstrated that the removal of the CNN front-end from their HTR-VT model resulted in a validation CER increase from 3.3% to 26.6% on IAM. Additionally, the Sorani script contains small diacritical marks that are essential for character disambiguation and are easily lost through aggressive downsampling.\u003c/p\u003e\n \u003cp\u003eOur feature extractor generates 128 tokens with a dimension of 768 from a 64\u0026times;512 grayscale image. A comprehensive description of the architecture is provided in Fig. \u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. It is distinguished from both ResNet-18 [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] and the HTR-VT variant [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] in five significant ways:\u003c/p\u003e\n \u003cp\u003eThere is no initial downsampling. We replace the 7\u0026times;7 strided convolution and max-pool of ResNet-18 with a 5\u0026times;5 convolution at stride (1,1) and omit the initial pooling layer entirely, thereby preserving the full input resolution and retaining fine diacritical details.\u003c/p\u003e\n \u003cp\u003eGradual downsampling that is asymmetric. Using stride (2,1), layers 1 and 2 reduce the height by half while maintaining the breadth. Layer 2 is followed by an intermediate max-pool with stride (2,1), and stride (2,2) is only applied by layer 3. The sequential character information along the width dimension, which is essential for CTC alignment, is preserved by this height-first strategy.\u003c/p\u003e\n \u003cp\u003eBlock configuration that is deeper. We employ a [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] block configuration (12 total blocks), which is more profound than both ResNet-18 [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] and HTR-VT [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. This configuration enables us to acquire the intricate positional variants and ligature patterns of cursive Arabic-script writing.\u003c/p\u003e\n \u003cp\u003eConvolutions that are dilated during the ultimate stage. We retain the fourth stage with stride (1,1) and apply dilation rate 2, rather than removing it as in HTR-VT. This expands the receptive field to capture inter-character stroke dependencies without further spatial resolution loss.\u003c/p\u003e\n \u003cp\u003eRegularization of dropouts. A CNN-level regularization is provided in our low-resource context by the application of a dropout layer (p\u0026thinsp;=\u0026thinsp;0.3) following the final residual stage, which is absent from both He et al. [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] and HTR-VT [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eThe expansion is delayed by the channel progression (192\u0026rarr;192\u0026rarr;384\u0026rarr;768), which concentrates early layers on low-level stroke and diacritical features. The final 768-channel dimension is directly proportional to the transformer embedding size, thereby eradicating any projection layer between the two components. The final 128\u0026times;768 token sequence is generated through a two-stage spatial collapse, which involves average pooling and adaptive average pooling. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e provides a comprehensive comparison between our architecture and the two discussed architectures.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eComparison of CNN feature extractor architectures.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eFeature\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eResNet-18\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eHTR-VT\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eOurs\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eInput channels\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e3 (RGB)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e1 (grayscale)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e1 (grayscale)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eInitial conv kernel\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e7\u0026times;7, stride (2,2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e3\u0026times;3, stride (2,1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e5\u0026times;5, stride (1,1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eInitial pooling\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eMaxPool, stride 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eMaxPool, stride (2,1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eNone\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eResidual stages\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eBlocks per stage\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eTotal residual blocks\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eChannel progression\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e64\u0026rarr;128\u0026rarr;256\u0026rarr;512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e192\u0026rarr;384\u0026rarr;768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e192\u0026rarr;192\u0026rarr;384\u0026rarr;768\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eDilated convolutions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes (layer4, d\u0026thinsp;=\u0026thinsp;2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eDropout in CNN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes (p\u0026thinsp;=\u0026thinsp;0.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eIntermediate pooling\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes (after layer2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eFinal spatial collapse\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eGlobal AvgPool\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eMaxPool (2,1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eAvgPool\u0026thinsp;+\u0026thinsp;AdaptiveAvgPool\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eOutput dimension\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e768\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\n \u003ch2\u003e4.2. Transformer encoder\u003c/h2\u003e\n \u003cp\u003eThe transformer encoder simulates long-range contextual dependencies throughout the token sequence. In comparison to ViT-Base [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] and HTR-VT [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] in Table \u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, our configuration employs 8 encoder layers and 8 attention centers with a 768-dimensional embedding. We adhere to the Pre-LayerNorm formulation [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e]:\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003cbr\u003e\n \u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\varvec{y}}^{\\varvec{n}}={\\varvec{x}}^{\\left(\\varvec{n}-1\\right)}+\\varvec{D}\\varvec{r}\\varvec{o}\\varvec{p}\\varvec{P}\\varvec{a}\\varvec{t}\\varvec{h}\\left(\\varvec{L}\\varvec{a}\\varvec{y}\\varvec{e}\\varvec{r}\\varvec{S}\\varvec{c}\\varvec{a}\\varvec{l}\\varvec{e}\\left(\\varvec{M}\\varvec{S}\\varvec{A}\\left(\\varvec{L}\\varvec{N}\\left({\\varvec{x}}^{\\left(\\varvec{n}-1\\right)}\\right)\\right)\\right)\\right)\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e(1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\u003cbr\u003e\u003c/div\u003e\n \u003c/caption\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd colname=\"c1\"\u003e\n \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\varvec{x}}^{\\varvec{n}}={\\varvec{y}}^{\\varvec{n}}+\\text{DropPath}\\left(\\text{LayerScale}\\left(\\text{FFN}\\left(\\text{LN}\\left({\\varvec{y}}^{\\varvec{n}}\\right)\\right)\\right)\\right)\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e(2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003cbr\u003e\u003c/div\u003e\n \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003eTable 4. Transformer encoder configuration comparison.\u003c/div\u003e\u0026nbsp;\u003ctable float=\"No\" id=\"Tabb\" border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eParameter\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eViT-Base\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eHTR-VT\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eOurs\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eEncoder layers\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eAttention heads\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eEmbedding dimension\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e768\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e768\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eFFN hidden dimension\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e3,072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e3,072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e3,072\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eActivation function\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eGELU\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eGELU\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eGELU\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003ePositional embeddings\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eLearned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003e2D sinusoidal (fixed)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003e2D sinusoidal (fixed)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eClass token\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eLayerScale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:1\\times\\:{10}^{-5}\\)\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eStochastic depth\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes (0\u0026rarr;0.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eQKV bias\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003ePatch embedding\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eLinear projection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eCNN (ResNet-18)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c4\"\u003e\n \u003cp\u003eCNN (ResNet-inspired)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eAs follows are the primary configuration decisions. The depth for the Sorani script model is doubled from the 4 layers in HTR-VT to 8, as it necessitates the resolution of multiple concurrent dependencies, including diacritic-to-base character association, four positional character forms, and flowing ligature continuity. These dependencies necessitate more successive attention passes than those in Latin-script HTR. We increase the number of attention heads from 6 to 8 to facilitate parallel specialization across these distinct relationship categories. For the sake of data efficiency, we implement fixed 2D sinusoidal positional embeddings [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] to circumvent the expense of learning positional parameters on a limited dataset. To stabilize the training of the deeper encoder, we introduce LayerScale [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] with an initialization of 1\u0026times;10⁻⁵, which enables each block to progressively learn its contributing factor. We employ stochastic depth [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e] with a linearly increasing drop rate of 0\u0026rarr;0.1 to regulate the subsequent encoder blocks against overfitting. Therefore, no class token is employed, as the assignment necessitates that each spatial position contribute to the CTC output sequence.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\n \u003ch2\u003e4.3. CTC decoder\u003c/h2\u003e\n \u003cp\u003eA linear projection is used to map each of the 128 output tokens from the 768-dimensional embedding space to K\u0026thinsp;=\u0026thinsp;56 classes, which includes the CTC blank token and 55 Sorani characters and vocabulary in DASNUS, following the transformer encoder. The CTC loss [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] is employed for both inference and training, marginalizing over all valid alignment paths between the predicted and ground-truth sequences without necessitating explicit character segmentation. This property is particularly well-suited to connected cursive script. A clear evaluation of the model\u0026apos;s intrinsic recognition capability is achieved by employing greedy decoding during inference in the absence of an external language model.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\n \u003ch2\u003e4.4. Training strategy\u003c/h2\u003e\n \u003cp\u003eA summary of all hyperparameters is provided in Table \u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. AdamW, which is augmented with SAM [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e], is employed to optimize for flat loss landscape minima and to significantly enhance generalization on limited datasets [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Over 1,000 iterations, the learning rate increases linearly from 1\u0026times;10⁻⁷ to 1\u0026times;10⁻\u0026sup3;, and then decreases to zero over 100,000 total iterations, following a warm-up cosine annealing schedule. An EMA with a decay of 0.9999 is maintained for the purposes of evaluation and inference.\u003c/p\u003e\n \u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eTraining hyperparameters.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eHyperparameter\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eValue\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eOptimiser\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eAdamW\u0026thinsp;+\u0026thinsp;SAM\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eBase LR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e1 \u0026times; 10⁻⁷ (warm-up start)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eMax LR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e1 \u0026times; 10⁻\u0026sup3;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eLR schedule\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eWarm-up cosine\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eWarm-up iterations\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e1,000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eTotal iterations\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e100,000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eWeight decay\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e0.5\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eBatch size\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eAdamW betas\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e(0.9, 0.99)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eEMA decay\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e0.9999\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eHardware\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e1 \u0026times; RTX 4090 (24 GB)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003eA span feature masking strategy [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] is implemented on the input token sequence prior to positional embeddings during training. This strategy replaces contiguous spans with a shared learnable mask token. We employ a mask ratio of \u0026tau;\u0026thinsp;=\u0026thinsp;0.3 and a maximum span length of 4 to prevent the masking of entire words in the more densely packed Sorani script, which is smaller than the values of HTR-VT (\u0026tau;\u0026thinsp;=\u0026thinsp;0.4, span\u0026thinsp;=\u0026thinsp;8).\u003c/p\u003e\n \u003cp\u003eIn Table\u0026nbsp;\u003cspan refid=\"Tab6\" class=\"InternalRef\"\u003e6\u003c/span\u003e, the stochastic data augmentation pipeline (p\u0026thinsp;=\u0026thinsp;0.5 per augmentation) is described in detail. The input images are resized to 64\u0026times;512 grayscale.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable float=\"Yes\" id=\"Tab6\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 6\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eData augmentation pipeline with parameter ranges.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eAugmentation Type\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eParameter Range\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eElastic Distortion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003e\u0026alpha; \u0026isin; [0.5, 1.0], \u0026sigma; \u0026isin; [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], kernel\u0026thinsp;\u0026le;\u0026thinsp;3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003ePerspective Warping\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003edistortion \u0026isin; [0.0, 0.4]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eMorphological (Dilation/Erosion)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eiterations\u0026thinsp;=\u0026thinsp;1, kernel\u0026thinsp;\u0026le;\u0026thinsp;3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eColor \u0026amp; Contrast Jitter\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003ebrightness\u0026thinsp;\u0026plusmn;\u0026thinsp;0.4, contrast\u0026thinsp;\u0026plusmn;\u0026thinsp;0.4, hue\u0026thinsp;\u0026plusmn;\u0026thinsp;0.2, saturation\u0026thinsp;\u0026plusmn;\u0026thinsp;0.4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eGaussian Blur\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003ekernel \u0026isin; [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e], \u0026sigma; \u0026isin; [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eSharpening\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003estrength \u0026isin; [0, 1], alpha \u0026isin; [0, 1]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eZoom (H/W scaling)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eH \u0026isin; [0.8, 1.0], W \u0026isin; [0.99, 1.0]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eSpan Masking\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003emask ratio\u0026thinsp;=\u0026thinsp;0.3, max span length\u0026thinsp;=\u0026thinsp;4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e\n \u003ch2\u003e4.5. Regularization strategy\u003c/h2\u003e\n \u003cp\u003eOur model addresses the fundamental challenge of training a deep model on a limited cursive script dataset by employing a coordinated multi-level regularization strategy, as summarized in Table \u003cspan refid=\"Tab7\" class=\"InternalRef\"\u003e7\u003c/span\u003e. Each of them operates at a separate stage of the pipeline, offering complementary rather than redundant protection against overfitting.\u003c/p\u003e\n \u003ctable float=\"Yes\" id=\"Tab7\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 7\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eMulti-level regularisation strategy.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eTechnique\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eComponent\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003ePurpose\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eDropout (p\u0026thinsp;=\u0026thinsp;0.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eCNN feature extractor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003ePrevents co-adaptation of CNN features; reduces overfitting in the feature extraction stage\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eSpan masking (\u0026tau;\u0026thinsp;=\u0026thinsp;0.3, s\u0026thinsp;=\u0026thinsp;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eTransformer input tokens\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eForces contextual learning; prevents tokens from relying solely on local information\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eLayerScale (init\u0026thinsp;=\u0026thinsp;1\\times{10}^{-5})\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eEach transformer block\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eStabilises training of the 8-layer encoder; enables gradual contribution learning\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eStochastic depth (0\u0026rarr;0.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eEach transformer block\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eRegularises deeper blocks; prevents overfitting in later layers\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eWeight decay (0.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eAll parameters\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eL2 regularisation on model weights; prevents parameter magnitude growth\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eEMA (\u0026alpha;\u0026thinsp;=\u0026thinsp;0.9999)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eModel-level\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eTemporal ensembling; smooths parameter trajectory for better generalisation\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eData augmentation (p\u0026thinsp;=\u0026thinsp;0.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eInput images\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eSimulates real-world variation\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colname=\"c1\"\u003e\n \u003cp\u003eSAM optimiser\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c2\"\u003e\n \u003cp\u003eOptimisation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" colname=\"c3\"\u003e\n \u003cp\u003eSeeks flat minima in loss landscape; improves out-of-distribution (OOD) generalisation\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"5. Results and Experiments","content":"\u003cp\u003eTo verify the performance of our HTR‑VT adapted model, we conducted a comprehensive series of ablations on the DASNUS dataset. The ablations were aimed not only at measuring the training convergence and error rate of the model, but also its relative performance with respect to state-of-the-art HTR systems.\u003c/p\u003e \u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003ch2\u003e5.1. Progressive ablation study\u003c/h2\u003e \u003cp\u003eIn order to study the relative importance of each part in our model, we have done a progressive ablation analysis. Starting with a baseline ViT architecture with normal patch embeddings and a CTC decoder, we added architectural and learning refinements incrementally. At each step, CER and WER on the validation and test set were computed. Results have been given in Table\u0026nbsp;\u003cspan refid=\"Tab8\" class=\"InternalRef\"\u003e8\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab8\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 8\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eMain ablation (progressive build-up).\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eVariant\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eTest\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline (ViT patch embedding only)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e5.30\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e27.10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e5.10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e26.20\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ CNN feature extractor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e21.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e4.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e20.92\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Span masking\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e20.82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e4.07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e19.94\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Data augmentation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19.24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.75\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.28\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Deeper transformer (6 layers)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.32\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Deeper transformer (8 layers)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.79\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.91\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.38\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ DropPath (0.1) + LayerScale (1e\u0026ndash;5), 4 layers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.67\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.17\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ DropPath (0.1) + LayerScale (1e\u0026ndash;5), 8 layers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.01\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e17.53\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe base model achieved decent results but was not suitable for recognizing continuous strokes and ligatures common in the Sorani script, as it relied on patch embeddings. Replacing patch embeddings with our proposed feature extractor resulted in the highest improvement, as convolutional features are more suitable for preserving stroke characteristics and continuity. The model's performance was also improved using span-based masking and data augmentation. The former helped improve character reconstruction from incomplete or faded parts, while data augmentation, including distortions and contrast, improved model generalization for different styles of writing. The model's depth was increased, but it did not improve performance on test set and even resulted in slight overfitting due to the small size of the dataset. However, when using regularization techniques like DropPath and LayerScale, increasing depth was advantageous. An 8-layer model achieved the best results, demonstrating that the combination of all techniques, rather than any single technique, improved performance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003e5.2. Isolated ablation study\u003c/h2\u003e \u003cp\u003eWhile the cumulative improvements in section 5.1 demonstrate how successive modifications contribute to the final architecture, they do not reveal the independent effect of each factor. To address this, we conducted an isolated ablation study, starting from the best-performing progressive model in section 5.1 and altering one component at a time. This allows us to disentangle the role of training stabilizers, positional encodings, masking strategies, and architectural variations in the ResNet-inspired encoder. Results are presented in Table\u0026nbsp;\u003cspan refid=\"Tab9\" class=\"InternalRef\"\u003e9\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab9\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 9\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eIsolated ablation study (one factor at a time).\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eVariant\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eTest\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNo EMA\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.63\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e17.54\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOptimizer\u0026thinsp;=\u0026thinsp;AdamW\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.83\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.80\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.40\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLearnable positional enc.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.75\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e17.84\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMask ratio\u0026thinsp;=\u0026thinsp;0.15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.72\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.53\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMask ratio\u0026thinsp;=\u0026thinsp;0.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.74\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.83\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.21\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMax span length\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.87\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19.34\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.80\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.67\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMax span length\u0026thinsp;=\u0026thinsp;8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.22\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.00\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNo dilation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.83\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e19.26\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e18.45\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNo pool after L2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.72\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.63\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e17.98\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStride = (1,1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e3.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e17.37\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe performance was slightly impaired by disabling EMA, indicating its importance in stabilizing the training and improving performance through generalization. The performance was impaired by a larger margin when the SAM optimizer was replaced by AdamW. The SAM optimizer performed better because it focuses on finding flatter minima of the loss function, making the model more robust to variations of handwritten text by different writers. The performance was impaired when deterministic sinusoidal positional encodings were replaced by learnable embeddings. The sinusoidal encodings performed better because they maintain deterministic positional consistency, which is important for the ligature-rich structure of Sorani handwritten text. Experiments conducted to find the best masking approach indicated that a moderate amount of masking was necessary. Less masking was not effective because it failed to induce contextual learning. Excessive masking, however, eliminated important information. The best performance was obtained when the span lengths were moderate because they are representative of the irregularities of handwritten text. The importance of the different components of the ResNet-inspired encoder was also identified. The dilation and max pooling components of the ResNet-inspired encoder were important. The performance was impaired when either of the two was disabled. The best performance was obtained when the initial stride was (1,1) instead of (2,1). The horizontal information was important in the early stages of the network, which was necessary to recognize subtle differences such as diacritics and spacing. The best-performing configuration was a combination of a well-tuned of our CNN feature extractor, an 8-layer Transformer network, regularization, sinusoidal positional encodings, balanced masking, the SAM optimizer, and dropout. The ablation study indicates that the different components of the Sorani HTR model, both major and minor, contributed to achieving stable and accurate performance.\u003c/p\u003e \u003cp\u003eThis final setup was trained with 100,000 iterations and a scheduled learning rate. The learning rate was initially set at 0.0001 and stepped up to 0.001 during the initial 1,000 iterations so that early fast convergence was allowed (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ea). During training, both losses on the training and validation sets had smooth and continuous decreasing trends, with loss on the training set falling from 190.69 to 7.61 and loss on the validation set from 37.81 to 9.81 (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003eb). This narrow and stable difference between these two curves also points to good generalization and little overfitting, supporting the stability in model configuration deduced in the ablation studies.\u003c/p\u003e \u003cp\u003eFigure\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec presents CER and WER, which also decreased with the passage of time. In iteration 1000, the CER was 19.76% and the WER was 67.74%, which decreased with the improvement of training. By iteration 50,000, the CER went below 5%, and the WER was in the mid-20%. Finally, the model had the best validation CER of 3.58% and the WER of 18.18%.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003e5.3. Comparison with state-of-the-art\u003c/h2\u003e \u003cp\u003eThis section compares the proposed HTR model with extant state-of-the-art systems across multiple datasets. Initially, we conducted a comparison between the proposed architecture and the HTR-VT model [CITE], which served as the foundation for our methodology. HTR-VT was re-implemented and trained on the DASNUS dataset under identical conditions to guarantee a fair comparison. The baseline achieved a validation CER of 4.16% and a test CER of 4.09%, with validation and test WERs of 20.46% and 19.80%, respectively. Conversely, our modified model demonstrated superior outcomes, attaining a validation CER of 3.58%, a test CER of 3.47%, a validation WER of 18.18%, and a test WER of 17.37%. These enhancements illustrate that the modifications introduced, including the ResNet-inspired encoder with stride adjustments, span masking, data augmentation, and optimized training strategies, considerably enhance the recognition of Sorani handwriting.\u003c/p\u003e \u003cp\u003eIn order to evaluate the performance of the models for the task of generalization, the models were also tested on an additional set of 100 handwritten text lines from writers outside the DASNUS dataset. In this case, the performance of the re-implemented baseline model resulted in a CER of 7.57%, while the proposed model resulted in a CER of 6.59%. Similarly, the proposed model resulted in a WER of 31.76%, while the baseline model resulted in a WER of 35.89%. This shows that the proposed model has been revised to be more robust for the task. The results have been summarized in Table\u0026nbsp;\u003cspan refid=\"Tab10\" class=\"InternalRef\"\u003e10\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab10\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 10\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of our adapted model with the re-implemented HTR-VT [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] on DASNUS and on an external set of 100 real-world Sorani text-lines. Our model consistently outperforms the baseline across both benchmarks.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eVariant\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eDataset\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c4\" namest=\"c3\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003eTest\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eWER (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHTR-VT [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] (baseline)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDASNUS\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4.16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e20.46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e4.09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e19.80\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs (adapted model)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDASNUS\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18.18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e3.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e17.37\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHTR-VT [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] (baseline)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eExternal (100 lines)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e7.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e35.89\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs (adapted model)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eExternal (100 lines)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e6.59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e31.76\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAdditionally, we conducted a comparison between our model and a variety of Arabic handwriting recognition methods on the KHATT dataset (Table\u0026nbsp;\u003cspan refid=\"Tab11\" class=\"InternalRef\"\u003e11\u003c/span\u003e). The proposed system obtained a validation CER of 7.97% and a test CER of 5.05% without the use of any external language model or lexicon. Our proposed model attained more robust character-level accuracy while functioning as a single visual model, despite the fact that an ensemble BLSTM\u0026ndash;CTC system [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] reported a lower WER (13.52%) as a result of language modeling and output voting. In comparison to other systems, including MDLSTM-CTC [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e], CNN\u0026ndash;BLSTM\u0026ndash;CTC [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e], and Transformer-based models [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e], our proposed architecture obtained a significantly lower CER. This was primarily due to the CNN-Transformer design, which captures both local stroke features and long-range contextual dependencies. Improved computational efficiency, reduced dependence on sequential recurrence, and avoidance of handcrafted features are all advantages of this design.\u003c/p\u003e \u003cp\u003eFurther evaluation was conducted on Katib\u0026rsquo;s Pashto Text Imagebase (KPTI) [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e], which showed strong cross-script generalization (Table\u0026nbsp;\u003cspan refid=\"Tab11\" class=\"InternalRef\"\u003e11\u003c/span\u003e). Our proposed model, based on the CNN, Transformer, and CTC, was found to have a validation CER of 3.04% and a test CER of 3.10%, which is a reduction of about 66% compared to the MDLSTM baseline [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. Overall, the proposed model is shown to not only perform better than a strong baseline on the Sorani handwriting dataset but also perform state-of-the-art on a range of Arabic-script datasets, all without the need for any external linguistic resources.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab11\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 11\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance comparison on the KHATT and KPTI datasets.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDataset\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eReference\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMethods\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eCER (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"5\" rowspan=\"6\"\u003e \u003cp\u003eKHATT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBLSTM\u0026thinsp;+\u0026thinsp;CTC\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e7.85%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMDLSTM\u0026thinsp;+\u0026thinsp;CTC\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e19.98%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCNN\u0026thinsp;+\u0026thinsp;BLSTM\u0026thinsp;+\u0026thinsp;CTC\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e19.85%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eResNet\u0026ndash;BiLSTM\u0026ndash;CTC hybrid model\u0026thinsp;+\u0026thinsp;3-gram LM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e13.2%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTransformer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e18.45%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOurs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eViT-based model\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5.05%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eKPTI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBLSTM and MDLSTM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e9.22%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOurs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eViT-based model\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3.10%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec23\" class=\"Section2\"\u003e \u003ch2\u003e5.4. Misrecognized character analysis\u003c/h2\u003e \u003cp\u003eThe character-level error analysis reveals numerous systematic defects in the model that are not apparent through aggregate CER and WER metrics alone. The model's propensity to omit characters, particularly spaces and faint letters, is one of the most significant issues. This results in merged words and structural distortions in the transcription. This behavior is plainly demonstrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003ea, where the model fails to detect the inter-word space in \"ئامانجە سەرەکییەکە,\" resulting in the entire phrase being merged into a single token. The DASNUS test set frequently contains such omissions due to the highly inconsistent spacing practices of handwritten Central Kurdish, which range from wide gaps to tightly connected letters that visually resemble ligatures.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eDot-based confusions between visually similar characters are another significant source of error. Letters such as ت, ن, ب, ی, and ز are primarily distinguished by the number or location of their dots, which are frequently overlooked in negligent or artistic handwriting styles. The most misrecognized characters presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e, as the confusion pairs ت, ن, and ب, ی are among the most prevalent. This type of error is frequently the result of the dots being either indistinguishable or absent in the original handwriting, which renders the letters nearly identical during the handwriting process. A practical example of this type of error is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003eb, where the indistinguishable form of ز leads to the confusion of تەرکیز with تەرکیت.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAnother observation is that the model overpredicts; therefore, the model introduces characters that do not exist. This is attributed to the model's inability to differentiate between character strokes and noise. The inability of the model to recognize punctuation marks, especially dots, is an indication of the model's low sensitivity to small details. It is also evident from the observations that the model can easily miss a character or add an extra character even if the model is doing extremely well.\u003c/p\u003e \u003cp\u003eThe aforementioned observations broadly classify the model's misrecognition of characters into three categories: overpredicted characters, dot-based confusions, and missing characters. Based on the aforementioned observations, it is evident that the model's misrecognition of characters is attributed to the complexity of Kurdish cursive handwriting and the model's inability. The model's accuracy can be improved by implementing the proposed solutions, which include a better context sequence model to leverage the characteristics of Kurdish words, an enhanced attention model to increase sensitivity to small details, and the incorporation of dot recognition.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec24\" class=\"Section2\"\u003e \u003ch2\u003e5.5. Discussion of results and findings\u003c/h2\u003e \u003cp\u003eThe performance obtained by our adapted HTR-VT model on the DASNUS dataset demonstrates a major improvement in the field of CKB handwriting recognition. The model not only generalizes very well on unseen text, with its CER of 3.47% and WER of 17.37% on the last test set, but also surpasses many of the current methods formulated for other cursive and semitic languages. Through these experiments, it can be inferred that certain architectural decisions along with the training process we used played a valuable role in enhancing the recognition performance, especially in reducing the number of character-level misclassifications.\u003c/p\u003e \u003cp\u003eOne of the most important contributing elements to the low CER was the our HTR-VT architecture's design, which makes use of a vision-transformer backbone configured for the specific visual and spatial characteristics of Sorani handwriting. This is in contrast to standard CNN-based models that can be prone to the minute differences in curvature, shape, and ligature position typical of cursive scripts. The transformer architecture is well-suited to handle long-range relationships as well as fine-grained differences along sequences of visual tokens. This enabled the model to attempt to better discriminate among characters that are very similar in shape but varying in position, context, or small diacritics.\u003c/p\u003e \u003cp\u003eAnother major reason for this performance is the quality and size of the DASNUS dataset. The DASNUS dataset, being a large dataset of Sorani handwritten text-line images, offered a wide range of ligatures of characters, handwriting styles, and inter-word space variability. The model was thus subjected to a wide range of writing styles. Furthermore, since the model was trained on the entire text-line images and not just the cropped characters, this approach helped to reduce confusion between morphologically ambiguous characters.\u003c/p\u003e \u003cp\u003eThe strategy of training itself was also pivotal. Early on, aggressive learning rate (up to 0.001) helped the model quickly acquire global features of the handwriting. Later on, a gradual decay of the learning rate enabled the model to refine its weights subtlety, attending to fine details without deranging previously acquired patterns. This resulted in a smooth, stable decline of both training loss and validation loss, with very little overfitting, as indicated by the close correspondence of validation and test performance.\u003c/p\u003e \u003cp\u003eOur emphasis on CER in particular was intentional, since CER is a smaller-grained measure of recognition quality compared to WER, particularly in richly inflected languages such as CKB. In most real-world applications such as manuscript digitization or Optical Character Recognition (OCR)-based search, partial word recognition is useful even if most of the characters are correctly recognized. Thus, low CER is a good sign of usefulness in subsequent tasks such as indexing, archiving, or helping preserve languages.\u003c/p\u003e \u003cp\u003eCompared to models we studied in 5.4 Comparison with State-of-the-Art, we can see that our architecture fits better with the structure and linguistic requirements of Sorani handwriting. Other models with good performance on Latin or Arabic were found wanting as soon as they are applied to scripts with complicated joining behavior or varied stroke representations. Our HTR-VT's use of attention mechanisms and global receptive fields addresses this by allowing the model to acquire context-sensitive character embeddings. Lastly, we highlight that transformer-based models, in conjunction with sufficient data variance, customized preprocessing, and curriculum-based methods, can redefine the benchmark for HTR in lesser-represented languages such as Kurdish. These findings not only provide evidence of our HTR-VT model's viability but also are a strong justification for future work on deep-learn-based script-specific models as well as multilingual HTR.\u003c/p\u003e \u003cp\u003eWhile our writer-disjoint evaluation provides a fair measure of generalization to unseen handwriting styles, a limitation of the present study is the absence of an external OOD test set collected under different acquisition conditions (e.g., new writers, varied scanning devices, alternative prompts). Such an OOD benchmark would allow us to measure robustness under truly novel scenarios. We plan to address this in future work by expanding DASNUS with additional forms collected from different environments and contributors.\u003c/p\u003e \u003cp\u003eWhile our evaluation demonstrates strong performance under writer-disjoint splits of DASNUS, broader generalization analyses remain an important direction. In particular, zero-shot testing on unseen Kurdish handwriting (e.g., new page formats, layouts, or writing prompts), one-shot and few-shot writer adaptation using Parameter-Efficient Fine-Tuning (PEFT) techniques such as adapters or LoRA, and cross-script transfer (e.g., Arabic or Persian pretraining followed by Sorani adaptation) would provide stronger evidence of robustness and adaptability. These evaluations were not included in the present work due to resource and dataset limitations, but they represent natural next steps in future research. We anticipate that such studies would reveal how much transferable structure exists across cursive scripts and whether parameter-efficient fine-tuning can further improve writer adaptation in low-resource contexts.\u003c/p\u003e \u003c/div\u003e"},{"header":"6. Conclusion and Future Work","content":"\u003cp\u003eThis paper proposed a novel deep learning method for Central Kurdish Language (CKB) handwriting recognition with a ViT-based model structure modified for the task. Addressing the long-standing challenge of low-resource script recognition, particularly for a linguistically dense and visually complex script like CKB, we developed the DASNUS dataset\u0026mdash;a high-quality, diverse, and annotated collection of over 11,000 handwritten text-line images from 867 writers. The dataset is a groundbreaking achievement, as it is the largest public dataset that is specifically designed for Sorani handwriting recognition.\u003c/p\u003e \u003cp\u003eOur modified model, which features a convolutional backbone inspired by ResNet and a span-masked transformer encoder, exhibited a robust ability to learn the global contextual relationships and local stroke information of CKB handwriting. We were able to effectively address ligature, position-sensitive glyph, and handwriting variation issues by structuring the model with careful consideration of horizontal continuity and span-based regularization at training time.\u003c/p\u003e \u003cp\u003eFavorable results were obtained in these experiments, as the adapted model attained a CER of 3.47% and a WER of 17.37% on the held-out test set. These rates are almost identical to the validation results, which shows that the model generalizes the training data much more than it overfits. It is important to take these results into consideration. Though it doesn't make use of synthetic inputs nor of pretraining on enormous corpora, its accuracy still stands comparably to the state-of-the-art systems trained with other languages. That in itself testifies to the stability of the architecture\u0026mdash;particularly in the context of low-resource settings\u0026mdash;and belies the promise of ViTs in dealing with the idiosyncrasies of cursive script.\u003c/p\u003e \u003cp\u003eLooking ahead, A number of avenues suggest themselves:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eMultilinguality: Future releases may provide support to other Kurdish dialects\u0026mdash;or even totally related scripts such as Pashto or Uyghur\u0026mdash;by transferring knowledge across related languages.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eSynthetic Data Creation: There also exists future promise in creating artificial Central Kurdish handwriting samples, potentially with GANs or with style-transfer methods, to augment the model to generalize better across diversified handwriting systems.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eWriter Identification and Adaptation: In the light of DASNUS's population richness, the next step shall comprise writer-adaptive identification systems or a fusion of writer identification and handwriting recognition.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eMultimodal Fusion: Handwriting with either oral modality or auditory data could potentially allow for more complete transcription systems to preserve the Kurdish language.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eIn this research, we have proposed a ViT-based model of CKB handwriting recognition and have presented the DASNUS dataset. The new system produces rivaling outcomes with no dependency on linguistic priors or auxiliary lexicons. In future research, we intend to incorporate a lightweight language model or lexicon-based decoder to minimize even more the WER and enhance linguistic normality. In addition, we intend to extend comparative analyses by incorporating newer transformer-based models in an effort to put our approach in more direct competition with newer state-of-the-art models. We also intend to investigate multilingual transfer learning and cross-pretraining scripts approaches to improve recognition robustness in other cursive scripts with limited resources.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003eBriefly, this work lays a strong foundation for Kurdish handwriting recognition with state-of-the-art deep learning approaches and offers seminal resources and approaches that can be extended by the research community. It is a significant step toward balanced coverage of low-resource languages for AI-driven document processing and pattern detection systems.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eParticipant consent statement: All participants who contributed handwriting samples to the DASNUS dataset were informed about the purpose of the study and voluntarily agreed to participate. Written informed consent was obtained from the participants prior to data collection. The collected forms contained only demographic information and handwriting samples, and no personally identifying information is included in the published dataset or the study. The study procedures followed institutional research and ethical guidelines.\u003c/p\u003e\u003cp\u003e \u003ch2\u003e Competing interests\u003c/b\u003e:\u003c/h2\u003e \u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eFunding:\u003c/h2\u003e \u003cp\u003ethis research did not receive any funding.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eFARAEDWN M. SALIH led the study's design, data collection, methodology, deep learning implementation, experiments, software, and manuscript drafting. ABDULBASIT K. AL-TALABANI contributed to data collection, validated and analyzed results, supervised the research, and reviewed the manuscript. Both authors approved the final version.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe dataset used in this study is publicly available at: https://data.mendeley.com/datasets/xdj9f55rkm/1\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAfkari-Fahandari, A., Shabaninia, E., Asadi-Zeydabadi, F., Nezamabadi-Pour, H.: A Comprehensive Survey of Transformers in Text Recognition: Techniques, Challenges, and Future Directions. ACM Comput. Surv. \u003cb\u003e58\u003c/b\u003e(1\u0026ndash;111), 42 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1145/3771273\u003c/span\u003e\u003cspan address=\"10.1145/3771273\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRomein, C.A., Rabus, A., Leifert, G., Str\u0026ouml;bel, P.B.: Assessing advanced handwritten text recognition engines for digitizing historical documents. Int. J. Digit. Humanit. \u003cb\u003e7\u003c/b\u003e, 115\u0026ndash;134 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s42803-025-00100-0\u003c/span\u003e\u003cspan address=\"10.1007/s42803-025-00100-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBamoki, M., Wady, S.H., Badawi, S.: Data Brief. \u003cb\u003e60\u003c/b\u003e, 111533 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.dib.2025.111533\u003c/span\u003e\u003cspan address=\"10.1016/j.dib.2025.111533\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e Holy Quran Kurdish Sorani translation dataset for language modelling\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbdalla, P.A., Shakor, M.Y., Ameen, A.K., Hassan, D.A.: Critical review of the model description in \u0026lsquo;Kurdish handwritten character recognition using deep learning techniques\u0026rsquo;. Gene Expr. Patterns. \u003cb\u003e56\u003c/b\u003e, 119399 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.gep.2025.119399\u003c/span\u003e\u003cspan address=\"10.1016/j.gep.2025.119399\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbdalla, P.A., Qadir, A.M., Shakor, M.Y., Saeed, A.M., Jabar, A.T., Salam, A.A., Amin, H.H.H.: A vast dataset for Kurdish handwritten digits and isolated characters recognition. Data Brief. \u003cb\u003e47\u003c/b\u003e, 109014 (2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.dib.2023.109014\u003c/span\u003e\u003cspan address=\"10.1016/j.dib.2023.109014\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQader, A., Rashid, I.: T.A.: A Comprehensive Dataset of Complete Kurdish Handwritten Characters and Digits. 1, (2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.17632/shny5k9f4b.1\u003c/span\u003e\u003cspan address=\"10.17632/shny5k9f4b.1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGraves, A., Fern\u0026aacute;ndez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369\u0026ndash;376. Association for Computing Machinery, New York, NY, USA (2006)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKizilirmak, F., Yanikoglu, B.: CNN-BiLSTM model for English Handwriting Recognition: Comprehensive Evaluation on the IAM Dataset, (2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2307.00664\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2307.00664\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChan, A., Mijar, A., Saeed, M., Wong, C.-W., Khater, A.: HATFormer: Historic Handwritten Arabic Text Recognition with Transformers, (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2410.02179\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2410.02179\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, Y., Chen, D., Tang, T., Shen, X.: HTR-VT: Handwritten text recognition with vision transformer. Pattern Recogn. \u003cb\u003e158\u003c/b\u003e, 110967 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.patcog.2024.110967\u003c/span\u003e\u003cspan address=\"10.1016/j.patcog.2024.110967\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePechwitz, M., Maddouri, S.S., M\u0026auml;rgner, V., Ellouze, N., Amiri, H.: IFN/ENIT - DATABASE OF HANDWRITTEN ARABIC WORDS. In Proc. of CIFED. 127\u0026ndash;136 (2002)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMahmoud, S.A., Ahmad, I., Al-Khatib, W.G., Alshayeb, M., Tanvir Parvez, M., M\u0026auml;rgner, V., Fink, G.A.: KHATT: An open Arabic offline handwritten text database. Pattern Recogn. \u003cb\u003e47\u003c/b\u003e, 1096\u0026ndash;1112 (2014). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.patcog.2013.08.009\u003c/span\u003e\u003cspan address=\"10.1016/j.patcog.2013.08.009\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSafarzadeh, V.M., Jafarzadeh, P.: Offline Persian Handwriting Recognition with CNN and RNN-CTC. In: 2020 25th International Computer Conference, Computer Society of Iran (CSICC). pp. 1\u0026ndash;10 (2020)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHassan, S., Irfan, A., Mirza, A., Siddiqi, I.: Cursive Handwritten Text Recognition using Bi-Directional LSTMs: A Case Study on Urdu Handwriting. In: 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications (Deep-ML). pp. 67\u0026ndash;72 (2019)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMohammed, T.L., Ahmed, A.A., Al-Sanjary, O.I.: KRDOH: Kurdish Offline Handwritten Text Database. In: 2019 IEEE 7th Conference on Systems, Process and Control (ICSPC). pp. 86\u0026ndash;89 (2019)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmed, R.M., Rashid, T.A., Fatah, P., Alsadoon, A., Mirjalili, S.: An extensive dataset of handwritten central Kurdish isolated characters. Data Brief. \u003cb\u003e39\u003c/b\u003e, 107479 (2021). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.dib.2021.107479\u003c/span\u003e\u003cspan address=\"10.1016/j.dib.2021.107479\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlsaqi, I.M., Fattah, P.: Kurdish Handwritten Word Recognition withCRNN Using Multi-Backbone Evaluation: Kurdish Handwritten Word Recognition. Acad. J. Int. Univ. Erbil. \u003cb\u003e2\u003c/b\u003e, 439\u0026ndash;449 (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.63841/iue24539\u003c/span\u003e\u003cspan address=\"10.63841/iue24539\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMaalej, R., Kherallah, M.: Convolutional Neural Network and BLSTM for Offline Arabic Handwriting Recognition. In: 2018 International Arab Conference on Information Technology (ACIT). pp. 1\u0026ndash;6 (2018)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMutawa, A.M., Allaho, M.Y., Al-Hajeri, M.: Machine Learning Approach for Arabic Handwritten Recognition. Appl. Sci. \u003cb\u003e14\u003c/b\u003e, 9020 (2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/app14199020\u003c/span\u003e\u003cspan address=\"10.3390/app14199020\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmed, R.M., Rashid, T.A., Fattah, P., Alsadoon, A., Bacanin, N., Mirjalili, S., Vimal, S., Chhabra, A.: Kurdish Handwritten character recognition using deep learning techniques. Gene Expr. Patterns. \u003cb\u003e46\u003c/b\u003e, 119278 (2022). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.gep.2022.119278\u003c/span\u003e\u003cspan address=\"10.1016/j.gep.2022.119278\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMostafa, A., Mohamed, O., Ashraf, A., Elbehery, A., Jamal, S., Khoriba, G., Ghoneim, A.S.: OCFormer: A Transformer-Based Model For Arabic Handwritten Text Recognition. In: 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC). pp. 182\u0026ndash;186 (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMomeni, S., BabaAli, B.: A transformer-based approach for Arabic offline handwritten text recognition. SIViP. \u003cb\u003e18\u003c/b\u003e, 3053\u0026ndash;3062 (2024). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11760-023-02970-9\u003c/span\u003e\u003cspan address=\"10.1007/s11760-023-02970-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKass, D., Vats, E.: AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks. In: Uchida, S., Barney, E., Eglin, V. (eds.) Document Analysis Systems, pp. 507\u0026ndash;522. Springer International Publishing, Cham (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAl-Ma\u0026rsquo;adeed, S., Elliman, D., Higgins, C.A.: A data base for Arabic handwritten text recognition research. In: Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition. pp. 485\u0026ndash;489 (2002)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMezghani, A., Kanoun, S., Khemakhem, M., Abed, H.E.: A Database for Arabic Handwritten Text Image Recognition and Writer Identification. In: 2012 International Conference on Frontiers in Handwriting Recognition. pp. 399\u0026ndash;402 (2012)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoh, T.Y., Basah, S.N., Yazid, H., Aziz Safar, M.J., Ahmad Saad, F.S.: Performance analysis of image thresholding: Otsu technique. Measurement. \u003cb\u003e114\u003c/b\u003e, 298\u0026ndash;307 (2018). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.measurement.2017.09.052\u003c/span\u003e\u003cspan address=\"10.1016/j.measurement.2017.09.052\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, S., Shen, Q., Sun, J.: Skew detection using wavelet decomposition and projection profile analysis. Pattern Recognit. Lett. \u003cb\u003e28\u003c/b\u003e, 555\u0026ndash;562 (2007). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.patrec.2006.10.002\u003c/span\u003e\u003cspan address=\"10.1016/j.patrec.2006.10.002\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. Presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Presented at the International Conference on Learning Representations October 2 (2020)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., Liu, T.: On Layer Normalization in the Transformer Architecture. In: Proceedings of the 37th International Conference on Machine Learning. pp. 10524\u0026ndash;10533. PMLR (2020)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe, K., Chen, X., Xie, S., Li, Y., Doll\u0026aacute;r, P., Girshick, R.: Masked Autoencoders Are Scalable Vision Learners. Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTouvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., J\u0026eacute;gou, H.: Going Deeper With Image Transformers. Presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang, G., Sun, Y., Liu, Z., Sedra, D., Weinberger, K.Q.: Deep Networks with Stochastic Depth. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision \u0026ndash; ECCV 2016, pp. 646\u0026ndash;661. Springer International Publishing, Cham (2016)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eForet, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-Aware Minimization for Efficiently Improving Generalization, (2021). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2010.01412\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2010.01412\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJemni, S.K., Kessentini, Y., Kanoun, S., Ogier, J.-M.: Offline Arabic Handwriting Recognition Using BLSTMs Combination. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). pp. 31\u0026ndash;36 (2018)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmad, R., Naz, S., Afzal, M., Rashid, S., Liwicki, M., Dengel, A.: A Deep Learning based Arabic Script Recognition System: Benchmark on KHAT. IAJIT. \u003cb\u003e17\u003c/b\u003e, 299\u0026ndash;305 (2020). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.34028/iajit/17/3/3\u003c/span\u003e\u003cspan address=\"10.34028/iajit/17/3/3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLamtougui, H., Moubtahij, E., Fouadi, H., Satori, H.: An Efficient Hybrid Model for Arabic Text Recognition. Computers Mater. Continua. \u003cb\u003e74\u003c/b\u003e, 2871\u0026ndash;2888 (2023). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.32604/cmc.2023.032550\u003c/span\u003e\u003cspan address=\"10.32604/cmc.2023.032550\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmad, R., Afzal, M.Z., Rashid, S.F., Liwicki, M., Breuel, T., Dengel, A.: KPTI: Katib\u0026rsquo;s Pashto Text Imagebase and Deep Learning Benchmark. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 453\u0026ndash;458 (2016)\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Footnotes","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://data.mendeley.com/datasets/xdj9f55rkm/1\u003c/span\u003e\u003cspan address=\"https://data.mendeley.com/datasets/xdj9f55rkm/1\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Deep Learning, Computer Vision, Pattern Recognition, Handwriting, Vision Transformers","lastPublishedDoi":"10.21203/rs.3.rs-9225627/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9225627/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe problem of handwritten text recognition (HTR) of low-resource cursive scripts is a major problem in document image analysis. This research aims to bridge the gap of unavailable annotated data and effective text recognition systems for Central Kurdish, also known as Sorani, which is a complex script consisting of 34 letters, rich ligatures, and context-dependent diacritical marks, and it is the first time this script is studied in HTR research like our approach. In this research, we present DASNUS, a new large-scale dataset of 11,475 annotated text lines from 867 writers across the Kurdistan region of Iraq. We also propose a deep learning framework for HTR, which combines ResNet-inspired convolutional encoders and a Vision Transformer model. The model is particularly suited for addressing variability, dependencies, and ligatures, which are inherent in Sorani script. The model was trained using several techniques, including span-based masking, geometric and photometric transformations, and depth regularization of the transformer using DropPath and LayerScale. The proposed model achieved a CER of 3.47%, and a WER of 17.37%, comparable to or even better than other state-of-the-art models for Arabic, Persian, and English HTR. In this research, we have established the first benchmark for Kurdish cursive handwriting. We have also demonstrated that well-regularized transformer-based models are capable of effectively recognizing complex, low-resource cursive scripts. The results of this research are expected to pave the way for future research in multilingual OCR, writer adaptation, synthetic data, and inclusive AI.\u003c/p\u003e","manuscriptTitle":"A Modified Vision Transformer for Kurdish Cursive RTL Handwritten Text Recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-06 20:14:40","doi":"10.21203/rs.3.rs-9225627/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"456a1cd5-6968-47ba-811f-8642c3ccf0b6","owner":[],"postedDate":"April 6th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-04-06T20:14:42+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-06 20:14:40","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9225627","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9225627","identity":"rs-9225627","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}