Semi-Supervised Neural System for Tagging, Parsing and Lematization

Abstract

This paper describes the ICS PAS system which took part in CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies. The system consists of jointly trained tagger, lemmatizer, and dependency parser which are based on features extracted by a biL-STM network. The system uses both fully connected and dilated convolutional neural architectures. The novelty of our approach is the use of an additional loss function, which reduces the number of cycles in the predicted dependency graphs, and the use of self-training to increase the system performance. The proposed system, i.e. ICS PAS (Warszawa), ranked 3th/4th in the official evaluation 1 obtaining the following overall results: 73.02 (LAS), 60.25 (MLAS) and 64.44 (BLEX).

Empirical Linguistic Study of Sentence Embeddings

“…The sentences are tokenised with UDPipe 10 (Straka and Straková, 2017) and POS-tagged and dependency parsed with COMBO 11 (Rybak and Wróblewska, 2018). The UDPipe and COMBO models are trained on the UD English-EWT treebank 12 (Silveira et al, 2014) with 16k trees (254k tokens) and on the Polish PDB-UD treebank 13 (Wróblewska, 2018) with 22k trees (351k tokens).…”

Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders

“…For discriminative models, DT is the direct transfer baseline and S-T is the self-training baseline, both of which use the biaffine parser (Dozat and Manning, 2017). S-T follows Rybak and Wróblewska (2018) who use the source model to predict parse trees on the target data and then perform supervised training of the target model. The last eight methods are our methods.…”

Deep Contextualized Self-training for Low Resource Dependency Parsing

“…While several authors failed to demonstrate the efficacy of self-training for dependency parsing (e.g. (Rush et al, 2012)), recently it was found useful for neural dependency parsing in fully supervised multilingual settings (Rybak and Wróblewska, 2018).…”

“…For constituency parsing, selftraining has shown to improve linear parsers both when considerable training data are available (McClosky et al, 2006a,b), and in the lightly supervised and the cross-domain setups (Reichart and Rappoport, 2007). Although several authors failed to demonstrate the efficacy of self-training for dependency parsing (e.g., Rush et al, 2012), recently it was found useful for neural dependency parsing in fully supervised multilingual settings (Rybak and Wróblewska, 2018).…”