The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD

Agirre, Eneko; Lacalle, Oier López de; Soroa, Aitor

doi:10.18653/v1/w18-2505

Cited by 27 publications

(32 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though Babelfy is based on BabelNet, it does not make direct use of the translation information in BabelNet. Similarly, UKB (Agirre et al, 2014(Agirre et al, , 2018, which is based on personalized PageRank on WordNet, achieves state-of-the-art performance on English all-words WSD. Finally, utilizing contextual embeddings, SENSEMBERT learns knowledge-based multilingual sense embeddings obtained by combining representations learned using BERT with knowledge obtained from BabelNet.…”

Section: Wsd Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Improving Word Sense Disambiguation with Translations

Luan

Hauer

Mou

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

It has been conjectured that multilingual information can help monolingual word sense disambiguation (WSD). However, existing WSD systems rarely consider multilingual information, and no effective method has been proposed for improving WSD by generating translations. In this paper, we present a novel approach that improves the performance of a base WSD system using machine translation. Since our approach is language independent, we perform WSD experiments on several languages. The results demonstrate that our methods can consistently improve the performance of WSD systems, and obtain state-ofthe-art results in both English and multilingual WSD. To facilitate the use of lexical translation information, we also propose BABALIGN, an precise bitext alignment algorithm which is guided by multilingual lexical correspondences from BabelNet.

show abstract

Section: Wsd Systemsmentioning

confidence: 99%

“…UKB uses complete sense frequency distributions, which are referred to as the dictionary weight (dict weight). We use the same parameter settings as Agirre et al (2018). For fair comparison, when applying SOFTCONSTRAINT to a system variant without sense frequency information, we set γ to 0 to turn off the p freq component.…”

Section: Oracle Wsd Experimentsmentioning

confidence: 99%

Improving Word Sense Disambiguation with Translations

Luan

Hauer

Mou

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…As noted in Agirre et al (2018), depending on the particular configuration, it is easy to get a wide range of results using UKB. The configurations used here are based on the recommended configuration given by Agirre et al (2018). For all configurations, the ppr w2w algorithm is used, which runs personalised page rank for each target word.…”

Section: Ukbmentioning

confidence: 99%

A Contrastive Evaluation of Word Sense Disambiguation Systems for Finnish

Robertson¹

2019

Proceedings of the Fifth International Workshop on Computational Linguistics for Uralic Languages

View full text Add to dashboard Cite

Previous work in Word Sense Disambiguation (WSD), like many tasks in natural language processing, has been predominantly focused on English. While there has been some work on other languages, including Uralic languages, up until this point no work has been published providing a contrastive evaluation of WSD for Finnish, despite the requisite lexical resources, most notably FinnWord-Net, having long been in place. This work rectifies the situation. It gives results for systems representing the major approaches to WSD, including some of the systems which have performed best at the task for English. It is hoped these results can act as a baseline for future systems, including both multilingual systems and systems specifically targeting Finnish, as well as point to directions for other Uralic languages. Tiivistelmä Aiempi saneiden alamerkitysten yksiselitteistämistä käsittelevä työ, kuten monet muut luonnollisen kielen käsittelyyn liittyvät tehtävät, on enimmäkseen keskittynyt englannin kieleen. Vaikka hieman työtä on tehty myös muilla kielillä, mukaan lukien uralilaiset kielet, vertailevaa arviointia suomen kielen saneiden alamerkitysten yksiselitteistämisestä ei ole tähän mennessä julkaistu huolimatta siitä, että tarvittavat leksikaaliset resurssit, erityisesti FinnWordNet, ovat jo pitkään olleet saatavilla. Tämä työ pyrkii korjaamaan tilanteen. Se tarjoaa tuloksia merkittävimpiä lähestymistapoja saneiden alamerkitysten yksiselitteistämiseen edustavista ohjelmista, sisältäen joitakin parhaiten englanninkielellä samasta tehtävästä suoriutuvia ohjelmia. Näiden tulosten toivotaan voivan toimia lähtökohtana tuleville, sekä monikielisille että erityisesti suomen kieleen kohdentuville, ohjelmille ja tarjota suuntaviivoja muihin uralilaisiin kieliin keskittyvään työhön.

show abstract

“…We have compared our proposal with UKB [1], which is one of the best WSD that can be found in the literature. Further, we have also evaluated the performance of these WSD by using a gold standard manually developed by a human expert.…”

Section: Sensementioning

confidence: 99%

“…Using the reported target distribution in Table 4, the resulting average ratio is 0.5538, from which we can conclude that the baseline performance is 55.35 %. Second, we have disambiguated BLESS using UKB [1], which is a well-known state-ofthe-art tool in WSD, and compare the results with our gold standard. Totally, UKB correctly disambiguates 180 from 200 targets, thus its performance is 90.00 %.…”

Section: Evaluation Frameworkmentioning

confidence: 99%

Towards Word Sense Disambiguation by Reasoning

Àlvez¹,

González-Dios²,

Rigau³

EPiC Series in Computing

View full text Add to dashboard Cite

In this paper, we describe a practical application of Vampire for Word Sense Disam- biguation (WSD), which is an important research area in the field of Natural Language Processing (NLP). Its objective is choosing the intended sense of a word in a given context. In particular, we propose a method for the automatic disambiguation of the semantic rela- tions in BLESS, which is a dataset designed to evaluate models of distributional semantics, by choosing the WordNet synset it belongs to. For this purpose, we use the knowledge in Adimen-SUMO, which is obtained by means of a suitable transformation of the knowledge in the core of SUMO1 into first-order logic (FOL) and enables its use by FOL automated theorem provers such as Vampire. By exploiting the semantic mapping between WordNet and SUMO, we apply a black-box testing method that enables the automatic creation a set of conjectures for each word pair by considering the semantic relations provided by BLESS. Then, these conjectures are evaluated using Vampire and, according to the outcomes, each word is disambiguated to a single synset. Finally, we compare the results provided by our proposal and different disambiguation systems that can be found in the literature.

show abstract

The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD

Cited by 27 publications

References 10 publications

Improving Word Sense Disambiguation with Translations

Improving Word Sense Disambiguation with Translations

A Contrastive Evaluation of Word Sense Disambiguation Systems for Finnish

Towards Word Sense Disambiguation by Reasoning

Contact Info

Product

Resources

About