Offline Features for Classifying Handwritten Math Symbols with Recurrent Neural Networks

Abstract

Abstract-In mathematical expression recognition, symbol classification is a crucial step. Numerous approaches for recognizing handwritten math symbols have been published, but most of them are either an online approach or a hybrid approach. There is an absence of a study focused on offline features for handwritten math symbol recognition. Furthermore, many papers provide results difficult to compare. In this paper we assess the performance of several well-known offline features for this task. We also test a novel set of features based on polar histograms and the vertical repositioning method for feature extraction. Finally, we report and analyze the results of several experiments using recurrent neural networks on a large public database of online handwritten math expressions. The combination of online and offline features significantly improved the recognition rate.

Recognition of Online Handwritten Math Symbols Using Deep Neural Networks

“…From these points of view, the effectiveness of com- bining online and offline classifiers is supported. Moreover, our system is compared with others published in [22], [23]. As shown in Table 11, our system outperforms slightly the system I which is currently holding the-state-of-the-art on TestCRHOME 2013 and 2014.…”

Crowdsourcing Online Handwriting Acquisition to Develop and Deploy a Unicode Character Classifier

“…Many approaches have been presented in the literature for character recognition [8], such as hidden Markov Models [9], support vector machines [10] or neural networks [11]. Converting online patterns into offline patterns makes possible to apply offline techniques for recognizing online characters [12]. Moreover, many previous works have opted for combining both modalities, by fusing them [13] or by using a representation like path iterated-integral signature [14] that produces an image where each pixel also encodes information regarding the input trajectory.…”

“…Generally, online systems achieve better performance than offline systems [15], because they can use more information from the trajectory of the handwriting. However, offline methods are less dependent on input order or writing styles such that in some cases they can produce better results [12]. Current state-of-the-art approaches to offline recognition make essential use of Convolutional Neural Networks [16,17] (CNNs).…”

An integrated grammar-based approach for mathematical expression recognition

“…It should be noted that no resampling is required prior to the feature extraction process because first derivatives implicitly perform writing speed normalization [37]. Furthermore, the combination of online and offline information has been proven to improve recognition accuracy [32,30,38]. For this reason, we also rendered the image representing the symbol hypothesis b i and extracted offline features to train another BLSTM-RNN classifier.…”

“…Following [38], for a segmentation hypothesis b i , we generated the image representation as follows. We set the image height to H pixels and kept the aspect ratio (up to 5H, in order to prevent creating too wide images).…”