An Encoder-Decoder Approach to Offline Handwritten Mathematical Expression Recognition with Residual Attention

Lin, Qiqiang; Wang, Chunyi; Bi, Ning; Suen, Ching Y.; Tan, Jun

doi:10.1007/978-3-031-09037-0_28

Cited by 5 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Encoder-decoder models have gained widespread adoption in the field of online handwritten text recognition, as evidenced by recent studies [13][14][15]. These models, equipped with attention mechanisms, have proven effective in converting handwritten trajectories into textual outputs.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Online Mongolian Handwriting Recognition Based on Encoder–Decoder Structure with Language Model

Fan,

Sun,

Wang

et al. 2023

Electronics

View full text Add to dashboard Cite

Mongolian online handwriting recognition is a complex task due to the script’s intricate characters and extensive vocabulary. This study proposes a novel approach by integrating a pre-trained language model into the sequence-to-sequence(Seq2Seq) + attention mechanisms(AM) model to enhance recognition accuracy. Three fusion models, including former, latter, and complete fusion, are introduced, showing substantial improvements over the baseline model. The complete fusion model, combined with synchronized language model parameters, achieved the best results, significantly reducing character and word error rates. This research presents a promising solution for accurate Mongolian online handwriting recognition, offering practical applications in preserving and utilizing the Mongolian script.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The higher the sentence probability, the lower the perplexity, which means that the model is better trained. The perplexity is calculated by Equation (15):…”

Section: Evaluation Indexmentioning

confidence: 99%

Online Mongolian Handwriting Recognition Based on Encoder–Decoder Structure with Language Model

Fan,

Sun,

Wang

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…At the inference stage, beam search is applied to find the output with the maximum probability, and the beam size is set to 4. In order to verify the effectiveness of the ClipMath we propose, we will compare it with other state-of-the-art methods, including DenseWAP [14], PAL-v2 [30], WS-WAP [31], DenseWAP-MSA [14], DenseWAP-TD [15], BTTR [11], and DATWAP [32]. We take DenseWAP as the baseline, and do not use any data enhancement in the experiment.…”

Section: Experimental Configurationmentioning

confidence: 99%

Offline Handwritten Mathematical Expression Recognition based on CLIP enhancement

2022

JRSE

View full text Add to dashboard Cite

Handwritten Mathematical Expression Recognition (HMER) aims to convert complex mathematical expression images into LaTeX expressions, which is of great significance in electronic documents and electronic education. The attention-based encoder-decoder architecture is very popular in HMER tasks. However, in the absence of large and medium-sized handwritten mathematical expression datasets, it is impossible to establish a pre-training task, and it is difficult to train a strong encoder from scratch. In addition, usually the decoder directly uses the high-level semantic features extracted by the encoder for decoding, and the low-level details are filtered out in the pooling layer, so the decoder pays little attention to the low-level information. In order to alleviate these two problems, this paper proposes a new ClipMath model. Its encoder is extended to a formula-text matching network based on CLIP-enhancement, including a formula-head and a text-head, which are used to cross-modal alignment of the image-state formula and the textual LaTeX label. The decoder uses the proposed Cascaded Multi-scale Decoder, which can incorporate features of different scales in the decoding process and improve the attention to detail information. Experiments show that the ExpRate of the proposed method on CROHME 2014, CROHME2016 and CROHME 2019 is 1.22%, 0.64% and 2.01% higher than the state-of-the-art model respectively.

show abstract

Relative Position Embedding Asymmetric Siamese Network for Offline Handwritten Mathematical Expression recognition

Wang,

Luo,

Rao

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

An Encoder-Decoder Approach to Offline Handwritten Mathematical Expression Recognition with Residual Attention

Cited by 5 publications

References 15 publications

Online Mongolian Handwriting Recognition Based on Encoder–Decoder Structure with Language Model

Online Mongolian Handwriting Recognition Based on Encoder–Decoder Structure with Language Model

Offline Handwritten Mathematical Expression Recognition based on CLIP enhancement

Relative Position Embedding Asymmetric Siamese Network for Offline Handwritten Mathematical Expression recognition

Contact Info

Product

Resources

About