Augmenting a BiLSTM Tagger with a Morphological Lexicon and a Lexical Category Identification Step

Abstract

Previous work on using BiLSTM models for PoS tagging has primarily focused on small tagsets. We evaluate BiLSTM models for tagging Icelandic, a morphologically rich language, using a relatively large tagset. Our baseline BiLSTM model achieves higher accuracy than any previously published tagger not taking advantage of a morphological lexicon. When we extend the model by incorporating such data, we outperform previous state-of-theart results by a significant margin. We also report on work in progress that attempts to address the problem of data sparsity inherent in morphologically detailed, finegrained tagsets. We experiment with training a separate model on only the lexical category and using the coarse-grained output tag as an input for the main model. This method further increases the accuracy and reduces the tagging errors by 21.3% compared to previous state-of-the-art results. Finally, we train and test our tagger on a new gold standard for Icelandic.

A Warm Start and a Clean Crawled Corpus -- A Recipe for Good Language Models

“…incorrect spelling) and e (foreign) labels. In comparison, (Steingrímsson et al, 2019) achieve 94.04% accuracy.…”

“…A good deal of work has been done on NLP for Icelandic that concerns these benchmarks. PoS tagging is implemented using a rule-based approach in the IceNLP toolkit (Loftsson and Rögnvaldsson, 2007a), and using a Bi-LSTM model in the ABLTagger (Steingrímsson et al, 2019). Constituency parsing has been implemented using a hand-crafted context-free grammar in the Greynir package (Þorsteinsson et al, 2019), using finite-state transducers in IceParser (Loftsson and Rögnvaldsson, 2007), and using an mBERT model in (Arnardóttir and Ingason, 2020).…”

Help Yourself from the Buffet: National Language Technology Infrastructure Initiative on CLARIN-IS

“…• PoS tagger: Before the LTPI started, the best performing PoS tagger for Icelandic was ABLTagger 0.9, a BiLSTM model implemented in DyNet, achieving an accuracy of 94.47% when evaluated on the MIM-GOLD corpus with the original tagset (Steingrímsson et al, 2019). During the LTPI, this tagger has been gradually improved.…”

Language Technology Programme for Icelandic 2019-2023

“…Nefnir ) is a lemmatiser which uses suffix substitution rules, derived from the Database of Icelandic Morphology (Bjarnadóttir et al, 2019), giving results that outperform IceNLP. ABLTagger (Steingrímsson et al, 2019) is a PoS tagger that outperforms other taggers that have been trained for tagging Icelandic texts. Some of these tools give good results, but can be improved upon.…”

“…A number of PoS-taggers have been developed for Icelandic, with the best results achieved by a recent bidirectional LSTM tagging model (Steingrímsson et al, 2019). While developing PoS taggers for Icelandic further using state-of-the-art methods, we will also study and try to estimate how much accuracy can theoretically be reached in tagging a variety of Icelandic text styles, using the tag set chosen for the LT programme (see Section 5.1).…”