Enhancing Subword Embeddings with Open N-grams

Abstract

Using subword n-grams for training word embeddings makes it possible to subsequently compute vectors for rare and misspelled words. However, we argue that the subword vector qualities can be degraded for words which have a high orthographic neighbourhood; a property of words that has been extensively studied in the Psycholinguistic literature. Empirical findings about lexical neighbourhood effects constrain models of human word encoding, which must also be consistent with what we know about neurophysiological mechanisms in the visual word recognition system. We suggest that the constraints learned from humans provide novel insights to subword encoding schemes. This paper shows that vectors trained with subword properties informed by psycholinguistic evidence are superior to those trained with ad hoc ngrams. It is argued that physiological mechanisms for reading are key factors in the observed distribution of written word forms, and should therefore inform our choice of word encoding.

Meta-Learner for Amharic Sentiment Classification

“…Character n-grams can also be used in a task, regardless of the language. Character n-grams can be used with new languages, and they can also detect rare words that are out of vocabulary (OOV) or misspelled [24][25][26]. According to the findings, n-grams with n = 3 are the most common.…”