Smart IDReader: Document Recognition in Video Stream

2020

IEEE Access

Self Cite

In this paper, we introduce an ''on the device'' text line recognition framework that is designed for mobile or embedded systems. We consider per-character segmentation as a language-independent problem and individual character recognition as a language-dependent one. Thus, the proposed solution is based on two separate artificial neural networks (ANN) and dynamic programming instead of employing image processing methods for the segmentation step or end-to-end ANN. To satisfy the tight constraints on memory size imposed by embedded systems and to avoid overfitting, we employ ANNs with a small number of trainable parameters. The primary purpose of our framework is the recognition of low-quality images of identity documents with complex backgrounds and a variety of languages and fonts. We demonstrate that our solution shows high recognition accuracy on natural datasets even being trained on purely synthetic data. We use MIDV-500 and Census 1961 Project datasets for text line recognition. The proposed method considerably surpasses the algorithmic method implemented in Tesseract 3.05, the LSTM method (Tesseract 4.00), and unpublished method used in the ABBYY FineReader 15 system. Also, our framework is faster than other compared solutions. We show the language-independence of our segmenter with the experiment with Cyrillic, Armenian, and Chinese text lines.

Section: Introductionmentioning

confidence: 99%

Two-Step CNN Framework for Text Line Recognition in Camera-Captured Images

Chernyshova¹,

Sheshkus²,

2020

IEEE Access

Self Cite

“…As a rule, regular smartphones are used for document recognition, due to relatively low cost, sufficient computational power for performing recognition tasks, and ability of capturing video (or sequence of images). The ability to capture video is one of the most important advantages over traditional scanners, as in such case more information could be retrieved in comparison with a single image, and each newly acquired document image may be used to improve the recognition result [10]. Figure 1 illustrates an example of per-frame recognition results combination in a video stream.…”

Section: Introductionmentioning

confidence: 99%

Next integrated result modelling for stopping the text field recognition process in a video using a result model with per-character alternatives

Bulatov

Savelyev

Twelfth International Conference on Machine Vision (ICMV 2019)

2020

Self Cite

In the field of document analysis and recognition using mobile devices for capturing, and the field of object recognition in a video stream, an important problem is determining the time when the capturing process should be stopped. Efficient stopping influences not only the total time spent for performing recognition and data entry, but the expected accuracy of the result as well. This paper is directed on extending the stopping method based on next integrated recognition result modelling, in order for it to be used within a string result recognition model with per-character alternatives. The stopping method and notes on its extension are described, and experimental evaluation is performed on an open dataset MIDV-500. The method was compares with previously published methods based on input observations clustering. The obtained results indicate that the stopping method based on the next integrated result modelling allows to achieve higher accuracy, even when compared with the best achievable configuration of the competing methods.

“…Mobile document analysis systems have become the focus of a wide range of research in the recent years [5,7,12,22]. Modern mobile devices are equipped with high-quality cameras and decent computing power which allows them to be used for camera-based document analysis tasks.…”

Section: Introductionmentioning

confidence: 99%

“…While object recognition in a video stream presents some advantages and helps to solve problems related to camerabased image analysis, it creates two novel problems which have to be addressed: the problem of optimal integration of multiple results (or selection of the best result among the candidates obtained from different image frames), and the problem of stopping the video stream recognition process. The stopping problem is particularly important in relation to real-time computer vision systems working on a mobile device [5,17,24], where the time required to obtain the result is often as important as the accuracy of the result itself. Although there are some published works mentioning multiple OCR results integration [6] and the video stream OCR stopping problem [2,5], the corpus of tested and evaluated methods related to this task is hardly sufficient.…”

Section: Introductionmentioning

confidence: 99%

“…The stopping problem is particularly important in relation to real-time computer vision systems working on a mobile device [5,17,24], where the time required to obtain the result is often as important as the accuracy of the result itself. Although there are some published works mentioning multiple OCR results integration [6] and the video stream OCR stopping problem [2,5], the corpus of tested and evaluated methods related to this task is hardly sufficient. Examples of text field frames as appeared in the a video stream.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On optimal stopping strategies for text recognition in a video stream as an application of a monotone sequential decision model

Bulatov

Razumnyi

2019

IJDAR

Self Cite

The paper describes the problem of stopping the text field recognition process in a video stream, which is a novel problem, particularly relevant to real-time mobile document recognition systems. A decision-theoretic framework for this problem is provided, and similarities with existing stopping rule problems are explored. Following the theoretical works on monotone stopping rule problems, a strategy is proposed based on thresholding the estimation of the expected difference between consequent recognition results. The efficiency of this strategy is evaluated on an openly accessible dataset. The results show that this method outperforms the previously published methods based on identical results cluster size thresholding. Notes on future work include incorporation of recognition result confidence estimations in the proposed model and more precise evaluation of the observation cost. Keywords Recognition in video stream • Mobile OCR • Stopping rules • Decision making • Mobile document recognition • Anytime algorithms This work was partially financially supported by Russian Foundation for Basic Research, Projects 17-29-03170 and 17-29-03370.