A Human-Inspired Recognition System for Pre-Modern Japanese Historical Documents

Abstract

Recognition of historical documents is a challenging problem due to the noised, damaged characters, and background. However, in Japanese historical documents, not only contains the mentioned problems, pre-modern Japanese characters were written in cursive and are connected. Therefore, character segmentation-based methods do not work well. This leads to the idea of creating a new recognition system. In this paper, we propose a human-inspired document reading system to recognize multiple lines of premodern Japanese historical documents. During the reading, people employ eyes movement to determine the start of a text line. Then, they move the eyes from the current character/word to the next character/word. They can also determine the end of a line or skip a figure to move to the next line. The eyes movement integrates with visual processing to operate the reading process in the brain. We employ attention-based encoder-decoder to implement this recognition system. First, the recognition system detects were to start a text line. Second, the system scans and recognize character by character until the text line is completed. Then, the system continues to detect the start of the next text line. This process is repeated until reading the whole document. As results, the system is successful to recognize multiple lines, connected and cursive characters without performing character/line segmentation. Besides, we also employ a coverage model which stores the history of eyes movement to predict the next movement more precisely. We tested our human-inspired recognition system on the pre-modern Japanese historical document provided by the PRMU Kuzushiji competition. The results of the experiments demonstrate the superiority and effectiveness of our proposed system by achieving Sequence Error Rate of 9.87% and 53.81% on level 2 and level 3 of the dataset, respectively. These results outperform to any other systems participated in the PRMU Kuzushiji competition. INDEX TERMS A human reading-inspired recognition system, recognition of pre-modern Japanese historical document, attention-based encoder-decoder, Kuzushiji.

Predicting the Ordering of Characters in Japanese Historical Documents

“…Predicting Ordering and Layout in Historical Japanese Documents: The closest work that tried to determine Kuzushiji reading order was done by [7]. Human reading behavior was defined as people determining the start character of a paragraph/line, with the idea that they move the eyes from the current character/word to the next character/word.…”

Automated Transcription for Pre-modern Japanese Kuzushiji Documents by Random Lines Erasure and Curriculum Training

“…It contains two modules: a DenseNets for feature extraction and an LSTM Decoder with an attention model for generating the target characters. We employed a similar setting for the system as our previous works [4]. The advantage of our model is that it requires images and corresponding transcriptions without bounding boxes of characters.…”

“…Figure 7 shows the process of generating multiple lines dataset. In the previous work [4], we were able to train the recognition on multiple lines. Recognition of multiple lines is easier than that of full-page documents.…”

“…Nguyen et al won the competition by developing recognition systems based on convolutional neural network and Bidirectional Long Short-Term Memory [3]. In our previous work, we proposed a human-inspired reading system to recognize Kuzushi characters on PRMU Algorithm Contest [4]. The recognition system based on an attention-based encoder-decoder approach to simulate human reading behavior.…”

Vietnamese character recognition based on CNN model with reduced character classes

“…Text recognition is the process of converting the text in one or more images into documents such as those written on a computer [1][2][3]. Currently, the word recognition problem has been studied and widely used in the automation of office operations in many languages like English [4,5], Chinese [6,7], Japanese [8]. In Vietnam, there are some identification systems such as VietOCR software [9] and VNDOCR [10].…”