Refocus attention span networks for handwriting line recognition

Abstract

Recurrent neural networks have achieved outstanding recognition performance for handwriting identification despite the enormous variety observed across diverse handwriting structures, and poor-quality scanned documents. Initially, we proposed a BiLSTM baseline model with a sequential architecture well-suited for modeling text lines due to its capacity to learn probability distributions over character or word sequences. However, employing such recurrent paradigms prevents parallelization and suffers from vanishing gradients for long sequences during training. To alleviate these limitations, we propose four significant contributions to this work. First, we devised an end-to-end model composed of a splitattention CNN-backbone that serves as a feature extraction method and a self-attention Transformer encoder-decoder that serves as a transcriber method to recognize handwriting manuscripts. The multihead self-attention layers in an encoder-decoder transformer-based enhance the model's ability to tackle handwriting recognition and learn the linguistic dependencies of character sequences. Second, we conduct various studies on transfer learning (TL) from large datasets to a small database, determining which model layers require fine-tuning. Third, we attained an efficient paradigm by combining different strategies of TL with data augmentation (DA). Finally, since the robustness of the proposed model is lexicon-free and can recognize sentences not presented in the training phase, the model is only trained on a few labeled examples with no extra cost of generating and training on synthetic datasets. We recorded comparable and outperformed Character and Word Error Rates CER/WER on four benchmark datasets to the most recent models (SOTA). The code and models will be uploaded on Github.

Handwritten Recognition Techniques: A Comprehensive Review

“…Sampath and Gomathi [24] used MNIST to train neural networks for English handwritten recognition. RIMES, IAM, and READ were used in [25][26][27] to generate variable-length symbol sequences from the English handwritten text. More related work is shown in Table 1, which includes the datasets' name, size, and the procedure used.…”

Recognition of Hot-Cast Billet Numbers Based on ICRNN

“…The purpose of sequence modeling is to capture the contextual information of characters in image text for the next prediction, which is more suitable than handling each character separately. Bidirectional Long Short Term Memory (BiLSTM) can better capture long-term dependencies than traditional RNN structures in the sequence modeling stage, and many studies have begun to use BiLSTM in character recognition tasks 18) . However, although BiLSTM can efficiently capture contextual information, its structure itself determines that it is very time-consuming during both training and inference.…”