Regularized and Retrofitted models for Learning Sentence Representation with Context

Abstract

Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we rst characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.

Domain Adaptation with Adversarial Training and Graph Embeddings

“…Finally, for context prediction, instead of performing a random walk, we select nodes based on their similarity in the graph. Similar similarity-based graph has shown impressive results in learning sentence representations (Saha et al, 2017).…”

Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets

“…The edge is defined by the distance measure d(i, j) between tweets t i and t j , where the value of d represents how similar the two tweets are. Similar similarity-based graph has shown impressive results in learning sentence representations (Saha et al 2017). To find the nearest instances efficiently, we used k-d tree data structure (Witten et al 2016).…”