Large-Scale Printed Chinese Character Recognition for ID Cards Using Deep Learning and Few Samples Transfer Learning

Abstract

In the field of computer vision, large-scale image classification tasks are both important and highly challenging. With the ongoing advances in deep learning and optical character recognition (OCR) technologies, neural networks designed to perform large-scale classification play an essential role in facilitating OCR systems. In this study, we developed an automatic OCR system designed to identify up to 13,070 large-scale printed Chinese characters by using deep learning neural networks and fine-tuning techniques. The proposed framework comprises four components, including training dataset synthesis and background simulation, image preprocessing and data augmentation, the process of training the model, and transfer learning. The training data synthesis procedure is composed of a character font generation step and a background simulation process. Three background models are proposed to simulate the factors of the background noise patterns on ID cards. To expand the diversity of the synthesized training dataset, rotation and zooming data augmentation are applied. A massive dataset comprising more than 19.6 million images was thus created to accommodate the variations in the input images and improve the learning capacity of the CNN model. Subsequently, we modified the GoogLeNet neural architecture by replacing the fully connected layer with a global average pooling layer to avoid overfitting caused by a massive amount of training data. Consequently, the number of model parameters was reduced. Finally, we employed the transfer learning technique to further refine the CNN model using a small number of real data samples. Experimental results show that the overall recognition performance of the proposed approach is significantly better than that of prior methods and thus demonstrate the effectiveness of proposed framework, which exhibited a recognition accuracy as high as 99.39% on the constructed real ID card dataset.

A custom-built deep learning approach for text extraction from identity card images

“…However, using pixel accuracy alone may not be the most appreciated solution since in certain cases with high score values, some pixels may be wrongly classified. In (5) represents pixel accuracy (PA), with k + 1 classes, pij being the number of pixels of class i inferred to be of class j and pii are true positives. Binary cross entropy (BCE) is a loss function that is used to evaluate the performance of binary classification algorithms [20].…”

“…Automating this process using the image of an identity card by traditional computing approaches is virtually impossible, as it requires an exact set of requirements to be respected regarding lighting conditions, orientation or background among others. With the increasing interest in deep learning and especially computer vision, few research works are applying deep learning techniques to retrieve text information from images including identity cards [2]- [5]. In computer vision, the field of study related to extracting and recognising text information from images is termed optical character recognition (OCR) [6].…”

Construction of Alphabetic Character Recognition Systems: A Review

“…In literature, lots of research can be found on handwritten CR in these scripts, for instance [11]- [13] work on Chinese, Japanese (Kanji), and Korean (Hanja), respectively. The accuracy rates for the scripts based on the aforementioned references are 99.39%, 99.64%, and 86.9%, respectively.…”

“…[48] English → online-handwritten → line level → public dataset → FC 5. [13] Chinese → offline-printed → character level → self-constructed dataset → PFC…”

Telugu letters dataset and parallel deep convolutional neural network with a SGD optimizer model for TCR

“…In the case of handwritten documents, the content is created by a human; the uniformity of the space usage and the smoothness of the strokes influence the document's quality [6]. Another problem with handwritten papers is that some people simplify complex glyphs in the language script while writing them, which reduces inter-class diversity and increases intra-class variability [7]. Document aging and document digitization for processing may create many types of distortion in addition to problems in the paper used and defects that occurred during printing or writing.…”