Learning Turkish Hypernymy Using Word Embeddings

Abstract

Recently, Neural Network Language Models have been effectively applied to many types of Natural Language Processing (NLP) tasks. One popular type of tasks is the discovery of semantic and syntactic regularities that support the researchers in building a lexicon. Word embedding representations are notably good at discovering such linguistic regularities. We argue that two supervised learning approaches based on word embeddings can be successfully applied to the hypernym problem, namely, utilizing embedding offsets between word pairs and learning semantic projection to link the words. The offset-based model classifies offsets as hypernym or not. The semantic projection approach trains a semantic transformation matrix that ideally maps a hyponym to its hypernym. A semantic projection model can learn a projection matrix provided that there is a sufficient number of training word pairs. However, we argue that such models tend to learn is-a-particular-hypernym relation rather than to generalize is-a relation. The embeddings are trained by applying both the Continuous Bag-of Words and the Skip-Gram training models using a huge corpus in Turkish text. The main contribution of the study is the development of a novel and efficient architecture that is well-suited to applying word embeddings approaches to the Turkish language domain. We report that both the projection and the offset classification models give promising and novel results for the Turkish Language.

Improving automated Turkish text classification with learning‐based algorithms

“…TTC-3600 and TTC-4900 datasets were used as benchmarking datasets in Turkish text classification. 10,11,16,46,47 Although both datasets are balanced, TTC-3600 has six news categories having 600 news in each category, 3600 news in total, while the TTC-4900 dataset has 700 samples in 7 categories and 4900 news. The categories and the number of samples in each category are given in Table 2.…”

Improving word embeddings projection for Turkish hypernym extraction

“…One can say that, for simplicity, the linear model with the optimization of Adagrad or RMSProp, loss function of cosine proximity, and 15 epoch steps can learn a model for mapping a noun to its hypernym. The Table also indicates that our models get a 90.75 % success rate and outperform the other study [22] that used the same dataset and applied the same problem. When we take SGD as the baseline function, our results shows that the proposed model can get faster and better results in terms of accuracy.…”

“…In the benchmark dataset there are 735 noun hypernym pairs, which are taken from another study for the Turkish language [22]. Pairs such as (burdur → il) and (doktor → meslek) have been randomly selected for a Turkish corpus, which is a natural language resource for Turkish.…”

“…The nonlinear models include a single hidden layer (1HL) and two hidden layers (2HL). These models were tested across four optimization algorithms and their performance are listed in the Table . The Table also includes the performance of another study using the same dataset in Turkish [22] for comparison. [22] 88.9 --In the literature there are three important and widely used activation functions: sigmoid, tangent, and ReLU [10,16].…”

“…These models were tested across four optimization algorithms and their performance are listed in the Table . The Table also includes the performance of another study using the same dataset in Turkish [22] for comparison. [22] 88.9 --In the literature there are three important and widely used activation functions: sigmoid, tangent, and ReLU [10,16]. Our experiments showed that ReLU, max(0, input) , is superior to the other two functions, sigmoid 1 1+e −x and tangent e x −e −x e x +e −x .…”