Handling Heavily Abbreviated Manuscripts: HTR Engines vs Text Normalisation Approaches

Abstract

Although abbreviations are fairly common in handwritten sources, particularly in medieval and modern Western manuscripts, previous research dealing with computational approaches to their expansion is scarce. Yet abbreviations present particular challenges to computational approaches such as handwritten text recognition and natural language processing tasks. Often, pre-processing ultimately aims to lead from a digitised image of the source to a normalised text, which includes expansion of the abbreviations. We explore different setups to obtain such a normalised text, either directly, by training HTR engines on normalised (i.e., expanded, disabbreviated) text, or by decomposing the process into discrete steps, each making use of specialist models for recognition, word segmentation and normalisation. The case studies considered here are drawn from the medieval Latin tradition.

The Learnable Typewriter: A Generative Approach to Text Line Analysis

“…• a deep generative approach to text line analysis, inspired by deep unsupervised multi-object segmentation approaches and adapted to work in both a weakly supervised and unsupervised setting, • a demonstration of the potential of our approach in challenging applications, particularly ciphered text analysis and paleographic analysis, • an extended evaluation on three very different datasets: a printed volume of the Google1000 dataset [19,46], the Copiale cipher [2,27] and historical handwritten charters from the 12th and early 13th century [6,44].…”

“…We experiment with three datasets with different characteristics: Google1000 [46], the Copiale cipher [27] and Fontenay manuscripts [6,44]. We check that our method leads to transcription results on par with related baselines using the Character Error Rate metric.…”

“…Fontenay manuscripts. The Fontenay dataset contains digitized charters that originate from the Cistercian abbey of Fontenay in Burgundy (France) [6,44] and were created during the 12th and early 13th century. Each document has been digitized and each line has been manually segmented and transcribed.…”

“…Second, we demonstrate the potential of our method for new applications, more specifically in the field of paleography, which studies the history and variations of handwriting, and for cipher analysis. We evaluate our approach on three very different datasets: a printed volume of the Google1000 dataset [19,46], the Copiale cipher [2, 27] and historical handwritten charters from the 12th and early 13th century [6,44].…”

RASAM – A Dataset for the Recognition and Analysis of Scripts in Arabic Maghrebi

“…We have notably realized transcriptions that restore the spaces in Arabic, even when there are visually no discernable spaces in the manuscript. In view of the great variety of character morphologies in the Maghrebi manuscripts scripts, we favored the word-based approach instead of the character-based approach where the word separation is managed in post-processing [5]. where the copist should have written ṭā' marbūṭa in final position…”