A Neural Network Model for Efficient Antonymy-Synonymy Classification by Exploiting Co-occurrence Contexts and Word-Structure Patterns

Abstract

Antonymy and synonymy are basic semantic relations between words. Automatically distinguishing between antonymy and synonymy is an important task in natural language processing. This task is hard because antonyms and synonyms tend to occur in highly similar contexts. Recent studies often focus on exploiting densevector representations of words to deal with this problem. In this paper, we present a study on antonymy-synonymy discrimination for the Vietnamese language. We propose a deep neural network model (DVASNet) that can utilize not only embedding representations of words but also co-occurrence contexts and specific patterns of Vietnamese word structure. Our experimental results showed that the proposed method achieved significant improvements in comparison with a number of state-of-the-art methods by 14% to 17% in terms of F1 score for both benchmark datasets.

Indonesian Short Essay Scoring Using Transfer Learning Dependency Tree LSTM

“…The deep learning architecture that mainly used is a Convolutional Neural Network (CNN) [6], Recursive Neural Network (RNN) [7,8] and the combination of both [3][4][5]. The most commonly RNN used is Long Short-Term Memory (LSTM).…”

“…Then, the highest layer classifies the score using fully connected that is obtained from the important local feature. Bui, et al [8] propose a deep neural network approach for the classification of antonyms and synonyms using co-occurrence context and word structure. Both co-occurrence context and word structure, each word in sentences are represented by a vector of word embedding and part of speech (POS) representation.…”

“…The feature on word structure is developed by vector construction function including a dot-product similarity value between two words representation transformed by the sigmoid function, a Lexicographers Mutual Information score, and a word-structure pattern encoded value [8]. The feature concatenation of co-occurrence context and word structure used to classify antonyms and synonyms.…”

“…Distribution representation on RNN, that usually used is sequence distribution in unidirectional [3,7] or bidirectional [4,8]. Other than that, the representation can use dependency distribution.…”

“…Recent Studies which is used deep learning approach the most used sequence distribution and not consider the feature POS [3,4,6,7]. Whereas considering the use of the POS feature [8] are only used as additional features not to determine the distribution representation. We assume that the distribution of sequences is inadequate to capture the semantics of sentences.…”