Personalized PageRank with Syntagmatic Information for Multilingual Word Sense Disambiguation

Scozzafava, Federico; Maru, Marco; Brignone, Fabrizio; Torrisi, Giovanni Luca; Navigli, Roberto

doi:10.18653/v1/2020.acl-demos.6

Cited by 37 publications

(26 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we report results for knowledge-based systems (KB) that mainly rely on WordNet: Lesk ext +emb (Basile, Caputo, and Semeraro 2014), Babelfy (Moro, Raganato, and Navigli 2014), UKB (Agirre, López de Lacalle, and Soroa 2018), and TM (Chaplot and Salakhutdinov 2018). More recently, SyntagRank (Scozzafava et al 2020) showed best KB results by combining WordNet with the SyntagNet (Maru et al 2019) database of syntagmatic relations. However, as discussed in Section 3.2, we categorize these as knowledge-based because they do not directly incorporate sense-annotated instances as their source of knowledge.…”

Section: Comparisonmentioning

confidence: 99%

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Loureiro

Rezaee

Pilehvar

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in capturing context-sensitive semantic nuances. However, there is still little knowledge about their capabilities and potential limitations in encoding and recovering word senses. In this article, we provide an in-depth quantitative and qualitative analysis of the celebrated BERT model with respect to lexical ambiguity. One of the main conclusions of our analysis is that BERT can accurately capture high-level sense distinctions, even when a limited number of examples is available for each word sense. Our analysis also reveals that in some cases language models come close to solving coarse-grained noun disambiguation under ideal conditions in terms of availability of training data and computing resources. However, this scenario rarely occurs in real-world settings and, hence, many practical challenges remain even in the coarse-grained setting. We also perform an in-depth comparison of the two main language model based WSD strategies, i.e., fine-tuning and feature extraction, finding that the latter approach is more robust with respect to sense bias and it can better exploit limited available training data. In fact, the simple feature extraction strategy of averaging contextualized embeddings proves robust even using only three training sentences per word sense, with minimal improvements obtained by increasing the size of this training data.

show abstract

Section: Comparisonmentioning

confidence: 99%

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Loureiro

Rezaee

Pilehvar

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…SensEmBERT, which can, however, only disambiguate nouns; (ii) the best performing all-PoS system, i.e. SyntagRank (Scozzafava et al, 2020), a knowledge-based system; (iii) the feedforward baseline. We report results on the French, German, Italian and Spanish all-words evaluation datasets from SemEval-2013, which contain only nouns, and the Italian and Spanish datasets from SemEval-2015, which contain all PoS.…”

Section: Cross-lingual Wsdmentioning

confidence: 99%

Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information

Bevilacqua¹,

Navigli²

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

118

View full text Add to dashboard Cite

Neural architectures are the current state of the art in Word Sense Disambiguation (WSD). However, they make limited use of the vast amount of relational information encoded in Lexical Knowledge Bases (LKB). We present Enhanced WSD Integrating Synset Embeddings and Relations (EWISER), a neural supervised architecture that is able to tap into this wealth of knowledge by embedding information from the LKB graph within the neural architecture, and to exploit pretrained synset embeddings, enabling the network to predict synsets that are not in the training set. As a result, we set a new state of the art on almost all the evaluation settings considered, also breaking through, for the first time, the 80% ceiling on the concatenation of all the standard allwords English WSD evaluation benchmarks. On multilingual all-words WSD, we report state-of-the-art results by training on nothing but English.

show abstract

“…The ability to identify the intended sense of a polysemous word in a given context is one of the fundamental problems in lexical semantics. It is usually addressed with two different kinds of approaches relying on either sense-annotated corpora (Bevilacqua and Navigli, 2020;Scarlini et al, 2020;Blevins and Zettlemoyer, 2020) or knowledge bases (Moro et al, 2014;Agirre et al, 2014;Scozzafava et al, 2020). Both are usually evaluated on dedicated benchmarks, including at least five WSD tasks in Senseval and SemEval series, from 2001 (Edmonds and Cotton, 2001) to 2015 (Moro and Navigli, 2015a) that are included in the Raganato et al (2017)'s test suite.…”

Section: Word Sense Disambiguationmentioning

confidence: 99%

XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization

Raganato

Pasini

Camacho-Collados

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

The ability to correctly model distinct meanings of a word is crucial for the effectiveness of semantic representation techniques. However, most existing evaluation benchmarks for assessing this criterion are tied to sense inventories (usually WordNet), restricting their usage to a small subset of knowledge-based representation techniques. The Word-in-Context dataset (WiC) addresses the dependence on sense inventories by reformulating the standard disambiguation task as a binary classification problem; but, it is limited to the English language. We put forward a large multilingual benchmark, XL-WiC, featuring gold standards in 12 new languages from varied language families and with different degrees of resource availability, opening room for evaluation scenarios such as zero-shot cross-lingual transfer. We perform a series of experiments to determine the reliability of the datasets and to set performance baselines for several recent contextualized multilingual models. Experimental results show that even when no tagged instances are available for a target language, models trained solely on the English data can attain competitive performance in the task of distinguishing different meanings of a word, even for distant languages. XL-WiC is available at https://pilehvar.github.io/xlwic/.

show abstract

Personalized PageRank with Syntagmatic Information for Multilingual Word Sense Disambiguation

Cited by 37 publications

References 26 publications

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Analysis and Evaluation of Language Models for Word Sense Disambiguation

Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information

XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization

Contact Info

Product

Resources

About