Online Handwritten Mathematical Symbol Segmentation and Recognition with Bidirectional Context

Nguyen, Cuong Tuan; Truong, Thanh-Nghia; Ung, Huy Quang; Nakagawa, Masaki

doi:10.1109/icfhr2020.2020.00071

Cited by 9 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We re-implemented the online bag-of-features proposed by Ung et al [6] (denoted as M1) to compare with our proposed method. M1 used an OnHME recognizer outputting a symbol relation tree instead of a LaTeX sequence, so that we used the OnHME recognizer proposed in [23] combined with an n-gram language model to extract the online bag-of-features. This recognizer achieves the expression rate of 51.70% on the CROHME 2014 testing set, which is 2.82 percentage points better than MTAP.…”

Section: Discussionmentioning

confidence: 99%

GSSF: A Generative Sequence Similarity Function Based on a Seq2Seq Model for Clustering Online Handwritten Mathematical Answers

Ung

Nguyen

et al. 2021

Document Analysis and Recognition – ICDAR 2021

Self Cite

View full text Add to dashboard Cite

Toward a computer-assisted marking for descriptive math questions, this paper presents clustering of online handwritten mathematical expressions (OnHMEs) to help human markers to mark them efficiently and reliably. We propose a generative sequence similarity function for computing a similarity score of two OnHMEs based on a sequence-to-sequence OnHME recognizer. Each OnHME is represented by a similarity-based representation (SbR) vector. The SbR matrix is inputted to the k-means algorithm for clustering OnHMEs. Experiments are conducted on an answer dataset (Dset_Mix) of 200 OnHMEs mixed of real patterns and synthesized patterns for each of 10 questions and a real online handwritten mathematical answer dataset of 122 student answers at most for each of 15 questions (NIER_CBT). The best clustering results achieved around 0.916 and 0.915 for purity, and around 0.556 and 0.702 for the marking cost on Dset_Mix and NIER_CBT, respectively. Our method currently outperforms the previous methods for clustering HMEs.

show abstract

Section: Discussionmentioning

confidence: 99%

GSSF: A Generative Sequence Similarity Function Based on a Seq2Seq Model for Clustering Online Handwritten Mathematical Answers

Ung

Nguyen

et al. 2021

Document Analysis and Recognition – ICDAR 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nguyen et al applied a temporal classification method for recognizing mathematical symbols [7]. The method uses a BLSTM model to take advantage of the bi-directional context for classifying symbols.…”

Section: Symbol Recognition With Bidirectional Contextmentioning

confidence: 99%

“…The deep BLSTM model is trained using a combination of the constraint loss and CTC loss as shown in Eq. (7). 𝑙𝑜𝑠𝑠 = 𝑙𝑜𝑠𝑠 𝐶𝑇𝐶 + 𝜆𝑙𝑜𝑠𝑠 𝐶𝐸 (7) where 𝜆 is a weighted parameter that is determined experimentally.…”

Section: Constraint For Output At Precise Time Stepsmentioning

confidence: 99%

“…(7). 𝑙𝑜𝑠𝑠 = 𝑙𝑜𝑠𝑠 𝐶𝑇𝐶 + 𝜆𝑙𝑜𝑠𝑠 𝐶𝐸 (7) where 𝜆 is a weighted parameter that is determined experimentally.…”

Section: Constraint For Output At Precise Time Stepsmentioning

confidence: 99%

“…In handwritten text recognition, temporal recognition of characters benefits from the bidirectional context of preceding strokes and succeeding strokes by Recurrent Neural Networks (RNNs) [6]. Nguyen et al applied a bidirectional RNN model for improving symbol segmentation and recognition of online HMEs [7]. Nonetheless, classification of spatial relations (relation classification) between symbols was separated from other subtasks, which does not benefit from the global context.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Global Context for Improving Recognition of Online Handwritten Mathematical Expressions

Nguyen

Truong

Nguyen

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This paper presents a temporal classification method for all three subtasks of symbol segmentation, symbol recognition and relation classification in online handwritten mathematical expressions (HMEs). The classification model is trained by multiple paths of symbols and spatial relations derived from the Symbol Relation Tree (SRT) representation of HMEs. The method benefits from global context of a deep bidirectional Long Short-term Memory network, which learns the temporal classification directly from online handwriting by the Connectionist Temporal Classification loss. To recognize an online HME, a symbollevel parse tree with Context-Free Grammar is constructed, where symbols and spatial relations are obtained from the temporal classification results. We show the effectiveness of the proposed method on the two latest CROHME datasets.

show abstract

Learning Symbol Relation Tree for Online Handwritten Mathematical Expression Recognition

Truong

Hùng

Nguyen

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Online Handwritten Mathematical Symbol Segmentation and Recognition with Bidirectional Context

Cited by 9 publications

References 17 publications

GSSF: A Generative Sequence Similarity Function Based on a Seq2Seq Model for Clustering Online Handwritten Mathematical Answers

GSSF: A Generative Sequence Similarity Function Based on a Seq2Seq Model for Clustering Online Handwritten Mathematical Answers

Global Context for Improving Recognition of Online Handwritten Mathematical Expressions

Learning Symbol Relation Tree for Online Handwritten Mathematical Expression Recognition

Contact Info

Product

Resources

About