A Transformer-Based Math Language Model for Handwritten Math Expression Recognition

A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN models

“…A standard encoder-decoder model generates a LaTeX representation without considering this ambiguity, and so a decoder may generate an ungrammatical sequence such as x^{2 and x^{2}} for the expression x^{2}. To overcome this problem, many systems use linguistic context from mathematical language models (LMs) to support the decoding process [11,58,87,88]. While these LMs tend to be much simpler than full expression grammars, they serve the same purpose by introducing syntactic constraints in the space of possible expressions.…”

“…Language Models (LMs) have often been used to resolve ambiguities in HME recognition [20,11,19,58,87,88], although they are generally not as effective as for natural languages where redundancy is much higher, and so misrecognized characters can often be replaced reliably by correct characters. However, mathematical expressions follow a more formal language with less redundancy (e.g., a specific superscript in a formula appears just once).…”

“…All methods used a large number of LaTeX strings to train the LMs. formulas [88]. Based on the self-attention mechanism of Transformer Networks, the high-level representation of an input token in a sequence of tokens is computed by how it is related to the previous tokens.…”

Language Model is Suitable for Correction of Handwritten Mathematical Expressions Recognition

“…Tree decoders pay less attention to context when predicting triples, often unable to distinguish differences such as '2' and 'z'. Ung et al (2021) try to employ a language model (LM) for post-correction, but Gupta et al (2021) underline the inherent risk of wholly depending on a LM for the correction of low-redundancy information, such as numbers, which is particularly susceptible to biases introduced by probabilistic skewing.…”