Incorporating Glosses into Neural Word Sense Disambiguation

Luo, Fuli; Liu, Tianyu; Qin, Xia; Chang, Baobao; Sui, Zhifang

doi:10.18653/v1/p18-1230

Cited by 86 publications

(63 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bi-LSTM +att.+LEX+P OS (Raganato et al, 2017a) is a multi-task learning framework for WSD, POS tagging, and LEX with self-attention mechanism, which converts WSD to a sequence learning task. GAS ext (Luo et al, 2018b) is a variant of GAS which is a gloss-augmented variant of the memory network by extending gloss knowledge. CAN s and HCAN (Luo et al, 2018a) are sentence-level and hierarchical co-attention neural network models which leverage gloss knowledge.…”

Section: Resultsmentioning

confidence: 99%

“…For a fair comparison, we use the benchmark datasets proposed by Raganato et al (2017b) which include five standard all-words fine-grained WSD datasets from the Senseval and SemEval competitions: Senseval-2 (SE2), Senseval-3 (SE3), SemEval-2007 (SE07), SemEval-2013 (SE13) and SemEval-2015 (SE15). Following Luo et al (2018a), Luo et al (2018b) and Raganato et al (2017a), we choose SE07, the smallest among these test sets, as the development set.…”

Section: Evaluation Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

Huang

Sun

Qiu

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

171

170

View full text Add to dashboard Cite

Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. Traditional supervised methods rarely take into consideration the lexical resources like WordNet, which are widely utilized in knowledge-based methods. Recent studies have shown the effectiveness of incorporating gloss (sense definition) into neural networks for WSD. However, compared with traditional word expert supervised methods, they have not achieved much improvement. In this paper, we focus on how to better leverage gloss knowledge in a supervised neural WSD system. We construct context-gloss pairs and propose three BERT-based models for WSD. We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task 1 .

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Evaluation Datasetsmentioning

confidence: 99%

GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

Huang

Sun

Qiu

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

171

170

View full text Add to dashboard Cite

show abstract

“…Addressing the issue of scarce annotations, recent works have proposed methods for using resources from knowledge-based approaches. Luo et al (2018a) and Luo et al (2018b) combine information from glosses present in WordNet, with NLMs based on BiLSTMs, through memory networks and co-attention mechanisms, respectively. Vial et al (2018) follows Raganato et al (2017b)'s BiLSTM method, but leverages the semantic network to strategically reduce the set of senses required for disambiguating words.…”

Section: Wsd State-of-the-artmentioning

confidence: 99%

Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation

Loureiro

Jorge

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

105

125

View full text Add to dashboard Cite

Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work, we show that contextual embeddings can be used to achieve unprecedented gains in Word Sense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level embeddings with full-coverage of WordNet, and without recourse to explicit knowledge of sense distributions or task-specific modelling. As a result, a simple Nearest Neighbors (k-NN) method using our representations is able to consistently surpass the performance of previous systems using powerful neural sequencing models. We also analyse the robustness of our approach when ignoring part-of-speech and lemma features, requiring disambiguation against the full sense inventory, and revealing shortcomings to be improved. Finally, we explore applications of our sense embeddings for concept-level analyses of contextual embeddings and their respective NLMs.

show abstract

“…8. Some methods can also be categorized as hybrid, as they make use of both sense-annotated corpora and knowledge resources, e.g., the gloss-augmented model of Luo et al (2018).…”

Section: Word Sense Disambiguationmentioning

confidence: 99%

From Word To Sense Embeddings: A Survey on Vector Representations of Meaning

Camacho-Collados¹,

Pilehvar²

2018

jair

240

View full text Add to dashboard Cite

Over the past years, distributed semantic representations have proved to be effective and flexible keepers of prior knowledge to be integrated into downstream applications. This survey focuses on the representation of meaning. We start from the theoretical background behind word vector space models and highlight one of their major limitations: the meaning conflation deficiency, which arises from representing a word with all its possible meanings as a single vector. Then, we explain how this deficiency can be addressed through a transition from the word level to the more fine-grained level of word senses (in its broader acceptation) as a method for modelling unambiguous lexical meaning. We present a comprehensive overview of the wide range of techniques in the two main branches of sense representation, i.e., unsupervised and knowledge-based. Finally, this survey covers the main evaluation procedures and applications for this type of representation, and provides an analysis of four of its important aspects: interpretability, sense granularity, adaptability to different domains and compositionality.

show abstract

Incorporating Glosses into Neural Word Sense Disambiguation

Cited by 86 publications

References 20 publications

GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation

From Word To Sense Embeddings: A Survey on Vector Representations of Meaning

Contact Info

Product

Resources

About