DAN: A Segmentation-Free Document Attention Network for Handwritten Document Recognition

Coquenet, Denis; Chatelain, Clément; Paquet, Thierry

doi:10.1109/tpami.2023.3235826

“…Many open-source architectures have been proposed for handwritten text recognition (HTR). This task is generally performed at linelevel, either relying on a CNN-HMM architecture [4], a CNN-RNN architecture [35], or a transformer-based architecture [13]. Recently, an increasing number of articles have been dedicated to learning from paragraphs or full pages [5,45,13].…”

Section: Handwritten Text Recognitionmentioning

confidence: 99%

“…Most of these approaches have been proposed as part of the 2017 Information Extraction in Historical Handwritten Records (IEHHR) competition 2 [17]. Following the same principle, attention-based neural networks such as DAN [13] could also be used for information extraction from full pages, by taking advantage of special tokens for both semantic labelling and structure description.…”

Section: Handwritten Text Recognitionmentioning

confidence: 99%

Large-scale genealogical information extraction from handwritten Quebec parish records

Tarride

¹

,

Maarand

²

,

Boillet

³

et al. 2023

IJDAR

8

0

View full text Add to dashboard Cite

This paper presents a complete workflow designed for extracting information from Quebec handwritten parish registers. The acts in these documents contain individual and family information highly valuable for genetic, demographic and social studies of the Quebec population. From an image of parish records, our workflow is able to identify the acts and extract personal information. The workflow is divided into successive steps: page classification, text line detection, handwritten text recognition, named entity recognition and act detection and classification. For all these steps, different machine learning models are compared. Once the information is extracted, validation rules designed by experts are then applied to standardize the extracted information and ensure its consistency with the type of act (birth, marriage, and death). This validation step is able to reject records that are considered invalid or merged. The full workflow has been used to process over two million pages of Quebec parish registers from the 19-20th centuries. On a sample comprising 65% of registers, 3.2 million acts were recognized. Verification of the birth and death acts from this sample shows that 74% of them are considered complete and valid. These records will be integrated into the BALSAC database and linked together to recreate family and genealogical relations at large scale.

show abstract

“…Many open-source architectures have been proposed for handwritten text recognition (HTR). This task is generally performed at linelevel, either relying on a CNN-HMM architecture [4], a CNN-RNN architecture [35], or a transformer-based architecture [13]. Recently, an increasing number of articles have been dedicated to learning from paragraphs or full pages [5,45,13].…”

Section: Handwritten Text Recognitionmentioning

confidence: 99%

Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records

Tarride¹,

Maarand²,

Boillet³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents a complete workflow designed for extracting information from Quebec handwritten parish registers. The acts in these documents contain individual and family information highly valuable for genetic, demographic and social studies of the Quebec population. From an image of parish records, our workflow is able to identify the acts and extract personal information. The workflow is divided into successive steps: page classification, text line detection, handwritten text recognition, named entity recognition and act detection and classification. For all these steps, different machine learning models are compared. Once the information is extracted, validation rules designed by experts are then applied to standardize the extracted information and ensure its consistency with the type of act (birth, marriage, and death). This validation step is able to reject records that are considered invalid or merged. The full workflow has been used to process over two million pages of Quebec parish registers from the 19-20th centuries. On a sample comprising 65% of registers, 3.2 million acts were recognized. Verification of the birth and death acts from this sample shows that 74% of them are considered complete and valid. These records will be integrated into the BALSAC database and linked together to recreate family and genealogical relations at large scale.

show abstract

“…SMT We propose an image-to-sequence approach with three different feature extractors. The common aspect of all the SMT tested in this paper is the implementation of a decoder, and we specifically follow the implementation of [12]. The SMT contains a transformer decoder with eight layers, four attention heads and an embedding size of 256 features in both the attention and feed-forward modules.…”

Section: Neural Network Configurationmentioning

confidence: 99%

“…There is no solution that currently goes beyond monophonic transcription, but rather adaptations that reduce the problem in order to make it close to a monophonic scenario and solve it using state-of-the-art methods. According to the recent works of HTR [33,13] and Document Understanding (DU) [12,22], OMR should seek to break this monophonic-dependent barrier. In this paper, we propose the Sheet Music Transformer (SMT), an image-to-sequence approach-based on autoregressive Transformers-that is able to transcribe music input images beyond monophony, without adaptations or specific preprocessing steps.…”

Section: Introductionmentioning

confidence: 99%

On the Use of Transformers for End-to-End Optical Music Recognition

Ríos-Vila

¹

,

Iñesta

²

,

Calvo-Zaragoza

³

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

State-of-the-art end-to-end Optical Music Recognition (OMR) has, to date, primarily been carried out using monophonic transcription techniques to handle complex score layouts, such as polyphony, often by resorting to simplifications or specific adaptations. Despite their efficacy, these approaches imply challenges related to scalability and limitations. This paper presents the Sheet Music Transformer (SMT), the first end-to-end OMR model designed to transcribe complex musical scores without relying solely on monophonic strategies. Our model employs a Transformer-based image-to-sequence framework that predicts score transcriptions in a standard digital music encoding format from input images. Our model has been tested on two polyphonic music datasets and has proven capable of handling these intricate music structures effectively. The experimental outcomes not only indicate the competence of the model, but also show that it is better than the state-of-the-art methods, thus contributing to advancements in end-to-end OMR transcription.

show abstract

DAN: A Segmentation-Free Document Attention Network for Handwritten Document Recognition

Cited by 43 publications

References 45 publications

Large-scale genealogical information extraction from handwritten Quebec parish records

Large-scale genealogical information extraction from handwritten Quebec parish records

Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records

On the Use of Transformers for End-to-End Optical Music Recognition

Contact Info

Product

Resources

About