CSI: A Coarse Sense Inventory for 85% Word Sense Disambiguation

Lacerra, Caterina; Bevilacqua, Michele; Pasini, Tommaso; Navigli, Roberto

doi:10.1609/aaai.v34i05.6324

Cited by 19 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, while the sense clustering provided by CSI (Lacerra et al 2020) covers all PoS categories, it extends BabelDomains (Camacho-Collados and Navigli 2017), a domain clustering resource that covers mainly nouns. Although out of scope for this article, in the future it would be interesting to investigate verb-specific clustering methods (e.g., Peterson and Palmer 2018).…”

Section: Resultsmentioning

confidence: 99%

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Loureiro

Rezaee

Pilehvar

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in capturing context-sensitive semantic nuances. However, there is still little knowledge about their capabilities and potential limitations in encoding and recovering word senses. In this article, we provide an in-depth quantitative and qualitative analysis of the celebrated BERT model with respect to lexical ambiguity. One of the main conclusions of our analysis is that BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. Our analysis also reveals that in some cases language models come close to solving coarse-grained noun disambiguation under ideal conditions in terms of availability of training data and computing resources. However, this scenario rarely occurs in real-world settings and, hence, many practical challenges remain even in the coarse-grained setting. We also perform an in-depth comparison of the two main language model based WSD strategies, i.e., fine-tuning and feature extraction, finding that the latter approach is more robust with respect to sense bias and it can better exploit limited available training data. In fact, the simple feature extraction strategy of averaging contextualized embeddings proves robust even using only three training sentences per word sense, with minimal improvements obtained by increasing the size of this training data.

show abstract

Section: Resultsmentioning

confidence: 99%

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Loureiro

Rezaee

Pilehvar

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Nevertheless, they suffer from the knowledge acquisition bottleneck, which hampers the creation of large manually-curated corpora (Gale, Church, and Yarowsky 1992), and in turn hinders the ability of these approaches to scale over unseen words and new languages. To overcome the aforementioned shortcomings, coarser sense inventories (Lacerra et al 2020) and automatic data augmentation approaches (Pasini and Navigli 2017;Pasini, Elia, and Navigli 2018;Scarlini, Pasini, and Navigli 2019) have been developed to cover more words, senses and languages. At the same time, dedicated architectures have been built to exploit the definitional information of a knowledge base (Luo et al 2018;Kumar et al 2019).…”

Section: Related Workmentioning

confidence: 99%

SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation

Scarlini

Pasini

Navigli

2020

AAAI

Self Cite

101

View full text Add to dashboard Cite

Contextual representations of words derived by neural language models have proven to effectively encode the subtle distinctions that might occur between different meanings of the same word. However, these representations are not tied to a semantic network, hence they leave the word meanings implicit and thereby neglect the information that can be derived from the knowledge base itself. In this paper, we propose SensEmBERT, a knowledge-based approach that brings together the expressive power of language modelling and the vast amount of knowledge contained in a semantic network to produce high-quality latent semantic representations of word meanings in multiple languages. Our vectors lie in a space comparable with that of contextualized word embeddings, thus allowing a word occurrence to be easily linked to its meaning by applying a simple nearest neighbour approach.We show that, whilst not relying on manual semantic annotations, SensEmBERT is able to either achieve or surpass state-of-the-art results attained by most of the supervised neural approaches on the English Word Sense Disambiguation task. When scaling to other languages, our representations prove to be equally effective as their English counterpart and outperform the existing state of the art on all the Word Sense Disambiguation multilingual datasets. The embeddings are released in five different languages at http://sensembert.org.

show abstract

“…WSD relies on annotated sense labels, which in turn requires determining whether any given pair of word uses belong to the same or distinct senses-i.e., whether to "lump" or "split". There is considerable debate about how granular word sense inventories should be (Hanks, 2000;Brown, 2008a); 3 resources range in granularity from Word-Net (Fellbaum, 1998) to the Coarse Sense Inventory, or CSI (Lacerra et al, 2020). Recent work using coarse-grained sense inventories has achieved success rates of 85% and beyond (Lacerra et al, 2020;Loureiro et al, 2020).…”

Section: Word Sense Disambiguationmentioning

confidence: 99%

“…There is considerable debate about how granular word sense inventories should be (Hanks, 2000;Brown, 2008a); 3 resources range in granularity from Word-Net (Fellbaum, 1998) to the Coarse Sense Inventory, or CSI (Lacerra et al, 2020). Recent work using coarse-grained sense inventories has achieved success rates of 85% and beyond (Lacerra et al, 2020;Loureiro et al, 2020).…”

Section: Word Sense Disambiguationmentioning

confidence: 99%

RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English)

Trott¹,

Bergen²

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Most words are ambiguous-they convey distinct meanings in different contexts-and even the meanings of unambiguous words are context-dependent. Both phenomena present a challenge for NLP. Recently, the advent of contextualized word embeddings has led to success on tasks involving lexical ambiguity, such as Word Sense Disambiguation. However, there are few tasks that directly evaluate how well these embeddings accommodate the continuous, dynamic nature of word meaningparticularly in a way that matches human intuitions. We introduce RAW-C, a dataset of graded, human relatedness judgments for 112 ambiguous words in context (with 672 sentence pairs total), as well as human estimates of sense dominance. The average inter-annotator agreement for the relatedness norms (assessed using a leave-one-annotatorout method) was 0.79. We then show that a measure of cosine distance, computed using contextualized embeddings from BERT and ELMo, correlates with human judgments, but that cosine distance also systematically underestimates how similar humans find uses of the same sense of a word to be, and systematically overestimates how similar humans find uses of different-sense homonyms. Finally, we propose a synthesis between psycholinguistic theories of the mental lexicon and computational models of lexical semantics.

show abstract

CSI: A Coarse Sense Inventory for 85% Word Sense Disambiguation

Cited by 19 publications

References 27 publications

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Analysis and Evaluation of Language Models for Word Sense Disambiguation

SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation

RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English)

Contact Info

Product

Resources

About