Lacking the Embedding of a Word? Look it up into a Traditional Dictionary

Abstract

Word embeddings are powerful dictionaries, which may easily capture language variations. However, these dictionaries fail to give sense to rare words, which are surprisingly often covered by traditional dictionaries. In this paper, we propose to use definitions retrieved in traditional dictionaries to produce word embeddings for rare words. For this purpose, we introduce two methods: Definition Neural Network (DefiNNet) and Define BERT (DefBERT). In our experiments, DefiNNet and DefBERT significantly outperform state-of-the-art as well as baseline methods devised for producing embeddings of unknown words. In fact, DefiNNet significantly outperforms FastText, which implements a method for the same task-based on n-grams, and DefBERT significantly outperforms the BERT method for OOV words. Then, definitions in traditional dictionaries are useful to build word embeddings for rare words.

Family Lexicon: using language models to encode memories of personally familiar and famous people and places in the brain

“…[11,100]) and that, following an appropriate methodology, they can be combined to create vectors for entities that do not appear in their pre-training data [27,30]. The questionnaire was composed of nine questions only, to which participants had to answer to with open-ended natural sentences -which by Natural Language Processing (NLP) standards is extremely small [24].…”

“…However, when it comes to personally familiar people or places, two main challenges arise. First, the extraction of semantic representations for concepts based on to robustly capture their meaning [21][22][23][24][25]. Despite their fundamental importance in our lives, personally familiar people and places (a close friend, one's favourite neighbourhood) never -or extremely rarely -get mentioned in large-scale corpora such as Wikipedia.…”