The Galactic Dependencies Treebanks: Getting More Data by                     Synthesizing New Languages

Wang, Dingquan; Eisner, Jason

doi:10.1162/tacl_a_00113

Cited by 51 publications

(65 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The world presumably does not offer enough natural languages-particularly with machine-readable corpora-to train a good classifier to detect, say, Object-Verb-Subject (OVS) languages, especially given the class imbalance problem that OVS languages are empirically rare, and the non-IID problem that the available OVS languages may be evolutionarily related. 1 We mitigate this issue by training on the Galactic Dependencies treebanks (Wang and Eisner, 2016), a collection of more than 50,000 human-like synthetic languages. The treebank of each synthetic language is generated by stochastically permuting the subtrees in a given real treebank to match the word order of other real languages.…”

Section: Approachmentioning

confidence: 99%

“…GD: Galactic Dependencies version 1.0 (Wang and Eisner, 2016) A collection of projective dependency treebanks for 53,428 synthetic languages, using the same format as UD. The treebank of each synthetic language is generated from the UD treebank of some real language by stochastically permuting the dependents of all nouns and/or verbs to match the dependent orders of other real UD languages.…”

Section: Datamentioning

confidence: 99%

“…With scoring functions that use only hand features, we adjust the feature weights by stochastic gradient descent (SGD). With scoring functions that include neural features, we use RMSProp (Tieleman and Hinton, 2012 Table 2: Data split of the 37 real languages, adapted from Wang and Eisner (2016). (Our "Train," on which we do 5-fold crossvalidation, contains both their "Train" and "Dev" languages.…”

Section: Training Proceduresmentioning

confidence: 99%

See 2 more Smart Citations

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

Wang

Eisner

2017

TACL

Self Cite

View full text Add to dashboard Cite

We show how to predict the basic word-order facts of a novel language given only a corpus of part-of-speech (POS) sequences. We predict how often direct objects follow their verbs, how often adjectives follow their nouns, and in general the directionalities of all dependency relations. Such typological properties could be helpful in grammar induction. While such a problem is usually regarded as unsupervised learning, our innovation is to treat it as supervised learning, using a large collection of realistic synthetic languages as training data. The supervised learner must identify surface features of a language's POS sequence (hand-engineered or neural features) that correlate with the language's deeper structure (latent trees). In the experiment, we show: 1) Given a small set of real languages, it helps to add many synthetic languages to the training data. 2) Our system is robust even when the POS sequences include noise. 3) Our system on this task outperforms a grammar induction baseline by a large margin.

show abstract

Section: Approachmentioning

confidence: 99%

Section: Datamentioning

confidence: 99%

Section: Training Proceduresmentioning

confidence: 99%

See 1 more Smart Citation

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

Wang

Eisner

2017

TACL

Self Cite

View full text Add to dashboard Cite

show abstract

“…Alternatively, the lack of target annotated data can be alleviated by synthesizing new examples, thus boosting the variety and amount of the source data. For instance, the Galactic Dependency Treebanks stem from real trees whose nodes have been permuted probabilistically according to the word order rules for nouns and verbs in other languages (Wang and Eisner 2016). Synthetic trees improve the performance of model transfer for parsing when the source is chosen in a supervised way (performance on target development data) and in an unsupervised way (coverage of target PoS sequences).…”

Section: Data Selection Synthesis and Preprocessingmentioning

confidence: 99%

Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing

Ponti

O'Horan

Berzak

et al. 2019

Computational Linguistics

View full text Add to dashboard Cite

Linguistic typology aims to capture structural and semantic variation across the world's languages. A large-scale typology could provide excellent guidance for multilingual Natural Language Processing (NLP), particularly for languages that suffer from the lack of human labeled resources. We present an extensive literature survey on the use of typological information in the development of NLP techniques. Our survey demonstrates that to date, the use of information in existing typological databases has resulted in consistent but modest improvements in system performance. We show that this is due to both intrinsic limitations of databases (in terms of coverage and feature granularity) and under-employment of the typological features included in them. We advocate for a new approach that adapts the broad and discrete nature of typological categories to the contextual and continuous nature of machine learning algorithms used in contemporary NLP. In particular, we suggest that such approach could be facilitated by recent developments in data-driven induction of typological knowledge.

show abstract

“…Crosslingual transfer can eliminate the need for the expensive and time-consuming task of treebank annotation for low-resource languages. Approaches include annotation projection using parallel data sets (Hwa et al, 2005;Ganchev et al, 2009), direct model transfer through learning of a delexicalized model from other treebanks (Zeman and Resnik, 2008;Täckström et al, 2013), treebank translation (Tiedemann et al, 2014), using synthetic treebanks (Tiedemann and Agić, 2016;Wang and Eisner, 2016), using cross-lingual word representations (Täckström et al, 2012;Guo et al, 2016;Rasooli and Collins, 2017) and using cross-lingual dictionaries (Durrett et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Low-Resource Syntactic Transfer with Unsupervised Source Reordering

Rasooli

Collins

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

We describe a cross-lingual transfer method for dependency parsing that takes into account the problem of word order differences between source and target languages. Our model only relies on the Bible, a considerably smaller parallel data than the commonly used parallel data in transfer methods. We use the concatenation of projected trees from the Bible corpus, and the gold-standard treebanks in multiple source languages along with cross-lingual word representations. We demonstrate that reordering the source treebanks before training on them for a target language improves the accuracy of languages outside the European language family. Our experiments on 68 treebanks (38 languages) in the Universal Dependencies corpus achieve a high accuracy for all languages. Among them, our experiments on 16 treebanks of 12 non-European languages achieve an average UAS absolute improvement of 3.3% over a state-of-the-art method.1 Specifically, Table 9 of Rasooli and Collins (2017) shows 13 datasets, and 11 languages, with UAS scores of over 80%; all of these datasets are in European languages.2 Specifically, performance of our method gives an improvement of at least 2.3% absolute scores in UAS on 11 datasets in 9 languages-Coptic, Basque, Chinese, Vietnamese, Turkish, Persian, Arabic, Indonesian Hebrew-with an average improvement of over 4.5% UAS.

show abstract

The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages

Abstract: We release Galactic Dependencies 1.0-a large set of synthetic languages not found on Earth, but annotated in Universal Dependencies format. This new resource aims to provide training and development data for NLP methods that aim to adapt to unfamiliar lan-arXiv:1710.03838v1 [cs.CL]

Cited by 51 publications

References 19 publications

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing

Low-Resource Syntactic Transfer with Unsupervised Source Reordering

Contact Info

Product

Resources

About