ShotgunWSD 2.0: An Improved Algorithm for Global Word Sense Disambiguation

Abstract

ShotgunWSD is a recent unsupervised and knowledge-based algorithm for global word sense disambiguation (WSD). The algorithm is inspired by the Shotgun sequencing technique, which is a broadly-used whole genome sequencing approach. ShotgunWSD performs WSD at the document level based on three phases. The first phase consists of applying a brute-force WSD algorithm on short context windows selected from the document, in order to generate a short list of likely sense configurations for each window. The second phase consists of assembling the local sense configurations into longer composite configurations by prefix and suffix matching. In the third phase, the resulting configurations are ranked by their length, and the sense of each word is chosen based on a majority voting scheme that considers only the top configurations in which the respective word appears. In this paper, we present an improved version (2.0) of ShotgunWSD which is based on a different approach for computing the relatedness score between two word senses, a step that stays at the core of building better local sense configurations. For each sense, we collect all the words from the corresponding WordNet synset, gloss and related synsets, into a sense bag. We embed the collected words from all the sense bags in the entire document into a vector space using a common word embedding framework. The word vectors are then clustered using k-means to form clusters of semantically related words. At this stage, we consider that clusters with fewer samples (with respect to a given threshold) represent outliers and we eliminate these clusters altogether. Words from the eliminated clusters are also removed from each and every sense bag. Finally, we compute the median of all the remaining word embeddings in a given sense bag to obtain a sense embedding for the corresponding word sense. We compare the improved ShotgunWSD algorithm (version 2.0) with its previous version (1.0) as well as several state-of-the-art unsupervised WSD algorithms on six benchmarks: SemEval 2007, Senseval-2, Senseval-3, SemEval 2013, SemEval 2015, and overall (unified). We demonstrate that ShotgunWSD 2.0 yields better performance than ShotgunWSD 1.0 and some other recent unsupervised or knowledge-based approaches. We also performed paired McNemar's significance tests, showing that the improvements of ShotgunWSD 2.0 over ShotgunWSD 1.0 are in most cases statistically significant, with a confidence interval of 0.01. INDEX TERMS Word sense disambiguation, shotgun sequencing, word embeddings, outlier removal.

The Unreasonable Effectiveness of Machine Learning in Moldavian versus Romanian Dialect Identification

“…After the preliminary work of Bengio et al (2003) and Schütze (1993), various improvements have been made in the quality of the embedding and the training time (Collobert and Weston 2008;Mikolov et al 2013a,b;Pennington, Socher, and Manning 2014), while some efforts have been directed towards learning multiple representations for polysemous words (Huang et al 2012;Reisinger and Mooney 2010;Tian et al 2014). These improvements, and many others not mentioned here, have been extensively used in various NLP tasks (Butnaru and Ionescu 2019b;Garg et al 2018;Glorot, Bordes, and Bengio 2011;Ionescu and Butnaru 2019;Musto et al 2016;Weston, Bengio, and Usunier 2011;Yang, Macdonald, and Ounis 2018).…”

Hybrid Sense Classification Method for Large-Scale Word Sense Disambiguation

“…Hence, performance is largely dependent on data quality. On the other hand, a knowledge-based approach [3], [5], [19], [28], [32] makes use of an external lexical knowledge base, such as such as 1 WordNet [26] or 2 BabelNet [29], to obtain the sense of an ambiguous word corresponding to its context. The resources generally include sets of words with lexical synonyms grouped into a set, and each set is linked in accordance with the lexical and semantic connections.…”

“…This section introduces a new large data set for WSD that is automatically collected from the publicly accessed 3 Oxford Dictionary. Its primary purpose is to learn a versatile WSD model that can address a wide range of topics, from real-life conversations to machine translation, second language education, medical information retrieval, and more.…”

Adapted Lesk Algorithm based Word Sense Disambiguation using the Context Information

“…The brute force algorithm is applied on short context window in first phase. In second phase, the local sense configurations are assembled by the prefix and suffix matching into the composite configurations where resulting configurations are ranked and sense of each word is detected based on majority voting as on [14]. WSD is a very common problem in the field of natural language processing (NLP).…”

“…The Induction technique has been used in first phase of this system where second phase is Word Sense Disambiguation which is developed by the use of Semantic Similarity Measure. ShotgunWSD [25] is a recent algorithm of The Global word sense disambiguation (WSD) which is unsupervised and knowledge-based algorithm. The algorithm has been developed from the Shotgun sequencing technique.…”