Dealing with orthographic variation in a tagger-lemmatizer for fourteenth century Dutch charters

Halteren, H. van; Rem, M.

doi:10.1007/s10579-013-9236-1

Cited by 8 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Distance measures are used to compare historical variants to modern lexicon entries [20,87,139]. Normalisation systems often combine several of these three approaches [1,12,139,180]. The fourth approach is statistical.…”

Section: Nlp Challengesmentioning

confidence: 99%

LL(O)D and NLP perspectives on semantic change for humanities research

Armaselu

Apostol

Khan

et al. 2022

View full text Add to dashboard Cite

This paper presents an overview of the LL(O)D and NLP methods, tools and data for detecting and representing semantic change, with its main application in humanities research. The paper’s aim is to provide the starting point for the construction of a workflow and set of multilingual diachronic ontologies within the humanities use case of the COST Action Nexus Linguarum, European network for Web-centred linguistic data science, CA18209. The survey focuses on the essential aspects needed to understand the current trends and to build applications in this area of study.

show abstract

Section: Nlp Challengesmentioning

confidence: 99%

LL(O)D and NLP perspectives on semantic change for humanities research

Armaselu

Apostol

Khan

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Both cga and cgl contain medieval Dutch material from the Gysseling corpus curated by the Institute for Dutch Lexicology 5 cga is a charter collection (administrative documents), whereas cgl concerns a variety of literary texts that greatly vary in length. crm is another Middle Dutch charter collection from the 14th century with wide geographic coverage (Van Reenen and Mulder, 1993;van Halteren and Rem, 2013). cgr, finally, is a smaller collection of samples from Middle Dutch religious writings that include later medieval texts (Kestemont et al, 2016).…”

Section: Datasetsmentioning

confidence: 99%

Improving Lemmatization of Non-Standard Languages with Joint Learning

Manjavacas¹,

Kádár²,

Kestemont³

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Lemmatization of standard languages is concerned with (i) abstracting over morphological differences and (ii) resolving token-lemma ambiguities of inflected words in order to map them to a dictionary headword. In the present paper we aim to improve lemmatization performance on a set of non-standard historical languages in which the difficulty is increased by an additional aspect (iii): spelling variation due to lacking orthographic standards. We approach lemmatization as a stringtransduction task with an encoder-decoder architecture which we enrich with sentence context information using a hierarchical sentence encoder. We show significant improvements over the state-of-the-art when training the sentence encoder jointly for lemmatization and language modeling. Crucially, our architecture does not require POS or morphological annotations, which are not always available for historical corpora. Additionally, we also test the proposed model on a set of typologically diverse standard languages showing results on par or better than a model without enhanced sentence representations and previous state-of-the-art systems. Finally, to encourage future work on processing of non-standard varieties, we release the dataset of non-standard languages underlying the present study, based on openly accessible sources.

show abstract

“…1300 AD. Secondly, we use the CRM-ADELHEID collection, a comparable collection of fourteenth century Middle Dutch charters, which has been the subject of a study comparable to ours (Van Halteren and Rem, 2013). As literary materials, we first of all use the literary counterpart of CG-ADMIN in the Corpus-Gysseling, a collection of Middle Dutch literary texts that all survive in manuscript copies predating ca.…”

Section: Data Setsmentioning

confidence: 99%

Lemmatization for variation-rich languages using deep learning

Kestemont

Pauw

Nie

et al. 2016

Digital Scholarship Humanities

View full text Add to dashboard Cite

In this article, we describe a novel approach to sequence tagging for languages that are rich in (e.g. orthographic) surface variation. We focus on lemmatization, a basic step in many processing pipelines in the Digital Humanities. While this task has long been considered solved for modern languages such as English, there exist many (e.g. historic) languages for which the problem is harder to solve, due to a lack of resources and unstable orthography. Our approach is based on recent advances in the field of 'deep' representation learning, where neural networks have led to a dramatic increase in performance across several domains. The proposed system combines two approaches: on the one hand, we apply temporal convolutions to model the orthography of input words at the character level; secondly, we use distributional word embeddings to represent the lexical context surrounding the input words. We demonstrate how this system reaches state-ofthe-art performance on a number of representative Middle Dutch data sets, even without corpus-specific parameter tuning.

show abstract

Dealing with orthographic variation in a tagger-lemmatizer for fourteenth century Dutch charters

Cited by 8 publications

References 12 publications

LL(O)D and NLP perspectives on semantic change for humanities research

LL(O)D and NLP perspectives on semantic change for humanities research

Improving Lemmatization of Non-Standard Languages with Joint Learning

Lemmatization for variation-rich languages using deep learning

Contact Info

Product

Resources

About