Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation

Abstract

Existing approaches to automatic VerbNetstyle verb classification are heavily dependent on feature engineering and therefore limited to languages with mature NLP pipelines. In this work, we propose a novel cross-lingual transfer method for inducing VerbNets for multiple languages. To the best of our knowledge, this is the first study which demonstrates how the architectures for learning word embeddings can be applied to this challenging syntactic-semantic task. Our method uses cross-lingual translation pairs to tie each of the six target languages into a bilingual vector space with English, jointly specialising the representations to encode the relational information from English VerbNet. A standard clustering algorithm is then run on top of the VerbNet-specialised representations, using vector dimensions as features for learning verb classes. Our results show that the proposed cross-lingual transfer approach sets new state-of-the-art verb classification performance across all six target languages explored in this work.

Investigating the cross-lingual translatability of VerbNet-style classification

“…Also, given the cross-lingual potential of VerbNet classification, the next natural step would be to use this type of classification to support multilingual NLP. Recent work on cross-lingual word embeddings has demonstrated that they can support crosslingual projection methods (e.g., Guo et al 2015;Ammar et al 2016;Upadhyay et al 2016;Vulić et al 2017). An avenue worth pursuing would be to investigate how verb classes obtained via our linguistically informed translation method compare to more pragmatically driven Brown-style clusters, as in the work of Täckström et al (2012) and Ammar et al (2016)-such comparative study could shed more light on the usefulness of linguistically motivated approaches to the transfer of linguistic structure across languages.…”

Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?

“…C2 C3 UNSUPERVISED unsupervised all tested, we always report the best one S1-S4 (FULL) ORTHG-SUPER provided -length normalization only (partial S1) ORTHG+SL+SYM provided symmetric: mutual nearest neighbours length normalization only (partial S1) FULL-SUPER provided -S1-S4 (FULL) FULL+SL provided (Artetxe et al, 2018b) with dropout S1-S4 (FULL) FULL+SL+NOD provided (Artetxe et al, 2018b) w/o dropout S1-S4 (FULL) FULL+SL+SYM provided symmetric: mutual nearest neighbours, w/o dropout S1-S4 (FULL) Vulić et al, 2017). PanLex currently spans around 1,300 language varieties with over 12M expressions: it offers some support and supervision also for low-resource language pairs (Adams et al, 2017).…”

Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics

“…AttentiveConvNet over (x, y) resembles the conventional hypernymy classifiers which take two representation vectors (one for x, the other for y) as input and output the label. Note that AttentiveConvNet puts filter weights over (x, y) to learn more abstract representations; this actually is in common with some literature such as (Fu et al, 2014;Vulic and Mrksic, 2017;Glavas and Ponzetto, 2017), which utilize weights to project generic word representations into specified representations towards hypernymy annotations.…”

“…Fu et al (2014) first use the skip-gram model to learn generic term embeddings from a large Chinese encyclopedia corpus, then learn a projection function from the generic space to hypernymy space by annotated hypernymy pairs. Other work trying to specify the generic word embeddings to hypernymy detection task include (Vulic and Mrksic, 2017;Glavas and Ponzetto, 2017).…”

“…(i) Pattern match-ing exploits patterns such as "animals such as cats" to indicate a hypernymy relation from "cat" to "animal" (Hearst, 1992;Snow et al, 2004). However, it requires the co-occurrence of the two terms in the same sentence, which limits the recall of this method; (ii) Term representation learning depends on a vector embedding of each term, where each entry in the vector expresses an explicit context feature (Baroni et al, 2012a;Roller and Erk, 2016;Shwartz et al, 2017) or a latent semantic (Fu et al, 2014;Vulic and Mrksic, 2017;Glavas and Ponzetto, 2017).…”