ICDAR 2019 CROHME + TFD: Competition on Recognition of Handwritten Mathematical Expressions and Typeset Formula Detection

Abstract

We summarize the tasks, protocol, and outcome for the 6th Competition on Recognition of Handwritten Mathematical Expressions (CROHME), which includes a new formula detection in document images task (+ TFD). For CROHME + TFD 2019, participants chose between two tasks for recognizing handwritten formulas from 1) online stroke data, or 2) images generated from the handwritten strokes. To compare L A T E X strings and the labeled directed trees over strokes (label graphs) used in previous CROHMEs, we convert L A T E X and stroke-based label graphs to label graphs defined over symbols (symbol-level label graphs, or symLG). More than thirty (33) participants registered for the competition, with nineteen (19) teams submitting results. The strongest formula recognition results were produced by the USTC-iFLYTEK research team, for both stroke-based (81%) and image-based (77%) input. For the new typeset formula detection task, the Samsung R&D Institute Ukraine (Team 2) obtained a very strong F-score (93%). System performance has improved since the last CROHME-still, the competition results suggest that recognition of handwritten formulae remains a difficult structural pattern recognition task.

A Hybrid Method for Mathematical Expression Detection in Scientific Document Images

“…As our method relies only on the appearance of mathematical expressions in the document images, it is general and can be applied for different datasets. In comparison with the similar method, the proposed method shows better performance than that of the Michiking sytem [48] because the employment of CNNs extracts features more efficiently than that of traditional rule-based and machine learning techniques.…”

“…The number of isolated and inline expressions in each page is described in Figure 9 The second one is GTDB public dataset [27]. It has recently been used for performance evaluation of researches [48]. The dataset consists of diverse font and mathematical symbol styles.…”

“…The performance comparison between the proposed and state of the art methods on the GTDB dataset is shown in Table 8. For the GTDB dataset, the method of Samsung R&D based on graph theory [48] has shown the highest performance. However, it is worth noting that the method in [48] exploits character-level information it is provided in GTDB dataset for the detection of mathematical expression.…”

“…For the GTDB dataset, the method of Samsung R&D based on graph theory [48] has shown the highest performance. However, it is worth noting that the method in [48] exploits character-level information it is provided in GTDB dataset for the detection of mathematical expression. This information is not available in others datasets such as Marmot dataset.…”

MathSeer: A Math-Aware Search Interface with Intuitive Formula Editing, Reuse, and Lookup

“…Handwritten formula entry is convenient for small expressions, but for large expressions such as Eq. 1 handwriting is slow [12], and accurate recognition is challenging [5]. Users may also want to avoid re-entering formulas, and to share formulas with others [12].…”