With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation

Abstract

Contextualized word embeddings have been employed effectively across several tasks in Natural Language Processing, as they have proved to carry useful semantic information. However, it is still hard to link them to structured sources of knowledge. In this paper we present ARES (context-AwaRe Embeddings of Senses), a semi-supervised approach to producing sense embeddings for the lexical meanings within a lexical knowledge base that lie in a space that is comparable to that of contextualized word vectors. ARES representations enable a simple 1-Nearest-Neighbour algorithm to outperform state-of-the-art models, not only in the English Word Sense Disambiguation task, but also in the multilingual one, whilst training on sense-annotated data in English only. We further assess the quality of our embeddings in the Word-in-Context task, where, when used as an external source of knowledge, they consistently improve the performance of a neural model, leading it to compete with other more complex architectures. ARES embeddings for all WordNet concepts and the automatically-extracted contexts used for creating the sense representations are freely available at http://sensembert.org/ares.

Bridging the Gap in Multilingual Semantic Role Labeling: a Language-Agnostic Approach

“…As previously stated, our objective has not been to dismiss the undeniable importance of syntaxbased innovation in SRL, but rather to establish a launch pad from which future syntactic developments can take off. In order to encourage future work on joint syntactic and semantic dependency parsing (Cai and Lapata, 2019b), the use of more powerful or cleverly trained language models (Lewis et al, 2020), the integration of SRL into other cross-lingual semantics-first tasks such as Semantic Parsing (Blloshmi et al, 2020) and Word Sense Disambiguation (Scarlini et al, 2020), and the exploitation and integration of newly available knowledge from recently released resources, such as VerbAtlas (Di Fabio et al, 2019) and Conception , we make available not only the code for our SRL model and experiments, but also our model checkpoints and training/validation logs at https://github.com/ SapienzaNLP/multi-srl.…”

InVeRo: Making Semantic Role Labeling Accessible with Intelligible Verbs and Roles

“…InVeRo is a growing platform: in the future, we plan to enhance our Model API by adding, alongside the already available state-of-the-art spanbased model, the state-of-the-art dependency-based model of Conia and Navigli (2020a), so that users can easily switch between the two approaches and choose the one that best suits their needs. Thanks to BabelNet and recent advances in cross-lingual techniques for tasks where semantics is crucial (Barba et al, 2020;Blloshmi et al, 2020;Conia and Navigli, 2020b;Pasini, 2020;Scarlini et al, 2020), we also plan to provide support for multiple languages to enable SRL integration into multilingual and cross-lingual settings. We believe that the InVeRo platform can make SRL more accessible to the research community, and we look forward to the development of semantics-first approaches in an ever wider range of NLP applications.…”

XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization

“…The ability to identify the intended sense of a polysemous word in a given context is one of the fundamental problems in lexical semantics. It is usually addressed with two different kinds of approaches relying on either sense-annotated corpora (Bevilacqua and Navigli, 2020;Scarlini et al, 2020;Blevins and Zettlemoyer, 2020) or knowledge bases (Moro et al, 2014;Agirre et al, 2014;Scozzafava et al, 2020). Both are usually evaluated on dedicated benchmarks, including at least five WSD tasks in Senseval and SemEval series, from 2001 (Edmonds and Cotton, 2001) to 2015 (Moro and Navigli, 2015a) that are included in the Raganato et al (2017)'s test suite.…”