An Extensive Empirical Evaluation of Character-Based Morphological Tagging for 14 Languages

Abstract

This paper investigates neural characterbased morphological tagging for languages with complex morphology and large tag sets. Character-based approaches are attractive as they can handle rarelyand unseen words gracefully. We evaluate on 14 languages and observe consistent gains over a state-of-the-art morphological tagger across all languages except for English and French, where we match the state-of-the-art. We compare two architectures for computing characterbased word vectors using recurrent (RNN) and convolutional (CNN) nets. We show that the CNN based approach performs slightly worse and less consistently than the RNN based approach. Small but systematic gains are observed when combining the two architectures by ensembling.

Modeling Composite Labels for Neural Morphological Tagging

“…In addition, we compare our SEQ model with a neural tagger presented by Dozat et al (2017), which is similar to our MC model, but employs a more sophisticated encoder. We train this model on UDv2.1 on the same set of languages used by Heigold et al (2017). Table 8 reports evaluation results for the three models.…”

“…The morphologically annotated Universal Dependencies (UD) corpora (Nivre et al, 2017) offer a great opportunity for experimenting on many languages. Some previous work have reported results on several UD languages (Yu et al, 2017;Heigold et al, 2017). Morphological tagging results on many UD languages have been also reported for parsing systems that predict POS and morphological tags as preprocessing (Andor et al, 2016;Straka et al, 2016;Straka and Straková, 2017).…”

“…Neural Multiclass classifier (MC) As the second baseline, we employ the standard multiclass classifier used by both Heigold et al (2017) and Yu et al (2017). The proposed model consists of an LSTM-based encoder, identical to the one described above in section 3.3, and a softmax classifier over the full tagset.…”

A Simple Joint Model for Improved Contextual Neural Lemmatization

“…We employ a simple LSTM-based tagger to recover the morphology of a sentence (Heigold et al, 2017;Cotterell and Heigold, 2017). We also experimented with the neural conditional random field of Malaviya et al (2018), but Heigold et al (2017) gave slightly better tagging scores on average and is faster to train. Given a sequence of n words w = w 1 , .…”

“…The word-level biLSTM tagger predicts a tag from Y. A full description of the model is found in Heigold et al (2017). We use standard cross-entropy loss for training this model and decode greedily while predicting the tags during test-time.…”