Word Semantic Representations using Bayesian Probabilistic Tensor Factorization

Zhang, Jingwei; Salwen, Jeremy; Glaß, Michael; Gliozzo, Alfio

doi:10.3115/v1/d14-1161

Cited by 33 publications

(33 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Linguistic Constraints and Dictionaries. We use the same set of monolingual constraints as LEAR : synonymy and antonymy constraints from (Zhang et al, 2014;Ono et al, 2015) are extracted from WordNet and Roget's Thesaurus (Kipfer, 2009). As in other work on LE specialisation (Nguyen et al, 2017;Nickel and Kiela, 2017), asymmetric LE constraints are extracted from WordNet, and we collect both direct and indirect LE pairs (i.e., (beagle, dog), (dog, an-7 The proposed CLEAR method is by design agnostic of input distributional vectors and its main purpose is to support fine-tuning of a wide spectrum of input vectors.…”

Section: Methodsmentioning

confidence: 99%

Multilingual and Cross-Lingual Graded Lexical Entailment

Vulić¹,

Ponzetto²,

Glavaš³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Grounded in cognitive linguistics, graded lexical entailment (GR-LE) is concerned with finegrained assertions regarding the directional hierarchical relationships between concepts on a continuous scale. In this paper, we present the first work on cross-lingual generalisation of GR-LE relation. Starting from Hyper-Lex, the only available GR-LE dataset in English, we construct new monolingual GR-LE datasets for three other languages, and combine those to create a set of six cross-lingual GR-LE datasets termed CL-HYPERLEX. We next present a novel method dubbed CLEAR (Cross-Lingual Lexical Entailment Attract-Repel) for effectively capturing graded (and binary) LE, both monolingually in different languages as well as across languages (i.e., on CL-HYPERLEX). Coupled with a bilingual dictionary, CLEAR leverages taxonomic LE knowledge in a resource-rich language (e.g., English) and propagates it to other languages. Supported by cross-lingual LE transfer, CLEAR sets competitive baseline performance on three new monolingual GR-LE datasets and six cross-lingual GR-LE datasets. In addition, we show that CLEAR outperforms current state-ofthe-art on binary cross-lingual LE detection by a wide margin for diverse language pairs.

show abstract

Section: Methodsmentioning

confidence: 99%

Multilingual and Cross-Lingual Graded Lexical Entailment

Vulić¹,

Ponzetto²,

Glavaš³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Additional experiments with other word vectors, e.g., with CONTEXT2VEC (Melamud et al, 2016a) (which uses bidirectional LSTMs (Hochreiter and Schmidhuber, 1997) for context modeling), and with dependency-word based embeddings (Bansal et al, 2014;Melamud et al, 2016b) lead to similar results and same conclusions. 5 We have experimented with another set of constraints used in prior work (Zhang et al, 2014;Ono et al, 2015), reaching similar conclusions: these were extracted from Word-Net (Fellbaum, 1998) and Roget (Kipfer, 2009), and comprise 1,023,082 synonymy pairs and 380,873 antonymy pairs. omitted only before the final output layer to enable full-range predictions (see Fig.…”

Section: Network Design and Parametersmentioning

confidence: 81%

Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources

Vulić¹,

Glavaš²,

Mrkšić³

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Word vector specialisation (also known as retrofitting) is a portable, light-weight approach to fine-tuning arbitrary distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet. By design, these post-processing methods only update the vectors of words occurring in external lexicons, leaving the representations of all unseen words intact. In this paper, we show that constraint-driven vector space specialisation can be extended to unseen words. We propose a novel post-specialisation method that: a) preserves the useful linguistic knowledge for seen words; while b) propagating this external signal to unseen words in order to improve their vector representations as well. Our post-specialisation approach explicits a non-linear specialisation function in the form of a deep neural network by learning to predict specialised vectors from their original distributional counterparts. The learned function is then used to specialise vectors of unseen words. This approach, applicable to any postprocessing model, yields considerable gains over the initial specialisation models both in intrinsic word similarity tasks, and in two downstream tasks: dialogue state tracking and lexical text simplification. The positive effects persist across three languages, demonstrating the importance of specialising the full vocabulary of distributional word vector spaces.

show abstract

“…We work with Wikipediatrained FASTTEXT embeddings (Bojanowski et al, 2017). We take English constraints from previous work -synonyms and antonyms were created from WordNet and Roget's Thesaurus (Zhang et al, 2014;Ono et al, 2015); LE constraints were collected from WordNet by and contain both direct and transitively obtained LE pairs. We retain the constraints for which both words exist in the trimmed (200K) FASTTEXT vocabulary, resulting in a total of 1,493,686 LE, 521,037 synonym, and 141,311 antonym pairs.…”

Section: Discussionmentioning

confidence: 99%

Generalized Tuning of Distributional Word Vectors for Monolingual and Cross-Lingual Lexical Entailment

Glavaš¹,

Vulić²

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Lexical entailment (LE; also known as hyponymy-hypernymy or is-a relation) is a core asymmetric lexical relation that supports tasks like taxonomy induction and text generation. In this work, we propose a simple and effective method for fine-tuning distributional word vectors for LE. Our Generalized Lexical ENtailment model (GLEN) is decoupled from the word embedding model and applicable to any distributional vector space. Yet-unlike existing retrofitting models-it captures a general specialization function allowing for LE-tuning of the entire distributional space and not only the vectors of words seen in lexical constraints. Coupled with a multilingual embedding space, GLEN seamlessly enables cross-lingual LE detection. We demonstrate the effectiveness of GLEN in graded LE and report large improvements (over 20% in accuracy) over state-ofthe-art in cross-lingual LE detection.

show abstract

Word Semantic Representations using Bayesian Probabilistic Tensor Factorization

Cited by 33 publications

References 11 publications

Multilingual and Cross-Lingual Graded Lexical Entailment

Multilingual and Cross-Lingual Graded Lexical Entailment

Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources

Generalized Tuning of Distributional Word Vectors for Monolingual and Cross-Lingual Lexical Entailment

Contact Info

Product

Resources

About