Lemmatization for Ancient Languages: Rules or Neural Networks?

Dereza, Oksana

doi:10.1007/978-3-030-01204-5_4

Cited by 7 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Initial evaluation of the POS tagging of the 1882-1926 segment of the corpus pointed to F-scores 33 ranging from 91-96% (Uí Dhonnchadha et al 2014). Dereza (2018), who discusses lemmatisation approaches for ancient and morphologically complex languages, reports that neither rule-based approaches 32 Code available at: https://github.com/kscanne/caighdean/ [accessed 10 March 2020]. 33 A measure of a test's accuracy that incorporates "precision" (e.g.…”

Section: Natural Language Processing Methodsmentioning

confidence: 99%

“…Fransen carefully weighs the advantages of different approaches in order to ensure the applicability of his analyser. He also envisions a fully functioning POS-tagger suitable for both Old and Middle Irish by making some suggestions for allowing interoperability of resources, especially between his morphological analyser and Dereza's (2018) Old Irish lemmatiser.…”

Section: Description Of Partmentioning

confidence: 99%

“…In the current implementation, the surface or lower level adheres, as closely as possible, to classical Old Irish grammar and orthography. Orthographical variation in Early Irish texts is expected to be successfully handled by one of Dereza's (2018) lemmatiser implementations (see 5.4) used as a standardiser (see 7.4).…”

Section: Lexical and Surface-level Descriptionmentioning

confidence: 99%

See 2 more Smart Citations

Morphosyntactic Variation in Medieval Celtic Languages

2020

View full text Add to dashboard Cite

Marieke Meelen 2 Annotating Middle Welsh: POS tagging and chunk-parsing a corpus of native prose 27 Theodorus Fransen 3 Automatic morphological analysis and interlinking of historical Irish cognate verb forms 49 Christopher Guy Yocum 4 Text clustering and methods in the Book of Leinster 85 Part 2: Morphosyntactic variation and change in medieval Celtic languages Liam Breatnach 5 The demonstrative pronouns in Old and Middle Irish 115 Carlos García-Castillero 6 Paradigmatic split and merger: The descriptive and diachronic problem of Old Irish Class B infixed pronouns 143 Elisa Roma 7 Nasalisation after inflected nominals in the Old Irish glosses: Evidence for variation and change 179 Jürgen Uhlich 8 On the obligatory use of a nasalising relative clause after an adjectival antecedent in the Old Irish glosses 195 Aaron Griffith 9 The "Cowgill particle", preverbal ceta 'first', and prepositional cleft sentences in the Old Irish glosses 239 Britta Irslinger 10 The functions and semantics of Middle Welsh X hun(an): A quantitative study 269

show abstract

Section: Natural Language Processing Methodsmentioning

confidence: 99%

Section: Description Of Partmentioning

confidence: 99%

Section: Lexical and Surface-level Descriptionmentioning

confidence: 99%

See 1 more Smart Citation

Morphosyntactic Variation in Medieval Celtic Languages

2020

View full text Add to dashboard Cite

show abstract

“…Examples of variants are: igreja matris/igreja matriz, parochia/parrochia. This pre-processing task is very challenging (Baron & Rayson, 2008;Dereza, 2018). There are some tools for classic languages, like CLTK: the classical language toolkit, 14 but concerning the classic period of the Portuguese language, existing tools 15 still need to be trained for this period.…”

Section: (42) Applicationsmentioning

confidence: 99%

Enriching the 1758 Portuguese Parish Memories (Alentejo) with Named Entities

Vieira

Olival

Cameron

et al. 2021

Journal of Open Humanities Data

View full text Add to dashboard Cite

This work presents an enriched version of the Parish Memories (1758-1761), an essential Portuguese historical source manually transcribed. It is enriched with annotations of named entities of the types PERSON, LOCATION, and ORGANIZATION. The annotation was done automatically for the whole collection where two researchers annotated a portion of it manually for evaluation purposes. In this dataset, we provide the tagged texts, the lists of extracted entities, and frequency counts. The corpus is useful for historians, allowing, for instance, comparative analyses between parishes and regions or to calculate the area of influence of a locality. The paper describes the creation and evaluation of the corpus, discusses its applications and limitations. This first release may be improved by other researchers interested in the historical source itself or in the technology employed in its annotation.

show abstract

“…For instance, Eger et al (2016) evaluates different pre-existing models on a dataset of German and Medieval Latin, and Dereza (2018) focuses on Early Irish. The most similar to the present paper in this area is work by Kestemont et al (2016), which tackled lemmatization of Middle Dutch with a neural encoder that extracts character and word-level features from a fixed-length token window and predicts the target lemma from a closed-set of true lemmas.…”

Section: Related Workmentioning

confidence: 99%

Improving Lemmatization of Non-Standard Languages with Joint Learning

Manjavacas¹,

Kádár²,

Kestemont³

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Lemmatization of standard languages is concerned with (i) abstracting over morphological differences and (ii) resolving token-lemma ambiguities of inflected words in order to map them to a dictionary headword. In the present paper we aim to improve lemmatization performance on a set of non-standard historical languages in which the difficulty is increased by an additional aspect (iii): spelling variation due to lacking orthographic standards. We approach lemmatization as a stringtransduction task with an encoder-decoder architecture which we enrich with sentence context information using a hierarchical sentence encoder. We show significant improvements over the state-of-the-art when training the sentence encoder jointly for lemmatization and language modeling. Crucially, our architecture does not require POS or morphological annotations, which are not always available for historical corpora. Additionally, we also test the proposed model on a set of typologically diverse standard languages showing results on par or better than a model without enhanced sentence representations and previous state-of-the-art systems. Finally, to encourage future work on processing of non-standard varieties, we release the dataset of non-standard languages underlying the present study, based on openly accessible sources.

show abstract

Lemmatization for Ancient Languages: Rules or Neural Networks?

Cited by 7 publications

References 18 publications

Morphosyntactic Variation in Medieval Celtic Languages

Morphosyntactic Variation in Medieval Celtic Languages

Enriching the 1758 Portuguese Parish Memories (Alentejo) with Named Entities

Improving Lemmatization of Non-Standard Languages with Joint Learning

Contact Info

Product

Resources

About