Dependency-based Siamese long short-term memory network for learning sentence representations

Zhu, Wenhao; Yao, Tengjun; Ni, Jianyue; Wei, Baogang; Lu, Zhiguo

doi:10.1371/journal.pone.0193919

Cited by 40 publications

(32 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although Benajiba et al [54] employed a similar network model to learn the similarity of semantic patterns, they utilized a regression function that is the mean squared error from the SQL structure distance to train the network model unlike [53]. The authors in [55] proposed a dependency-based Siamese LSTM network model, where they used the main and supporting components in sentence representation to create a difference for learning sentence representation. The authors in [56] aimed to learn thematic similarity between the sentences.…”

Section: Deep Metric Learning Problemsmentioning

confidence: 99%

Deep Metric Learning: A Survey

2019

View full text Add to dashboard Cite

Metric learning aims to measure the similarity among samples while using an optimal distance metric for learning tasks. Metric learning methods, which generally use a linear projection, are limited in solving real-world problems demonstrating non-linear characteristics. Kernel approaches are utilized in metric learning to address this problem. In recent years, deep metric learning, which provides a better solution for nonlinear data through activation functions, has attracted researchers' attention in many different areas. This article aims to reveal the importance of deep metric learning and the problems dealt with in this field in the light of recent studies. As far as the research conducted in this field are concerned, most existing studies that are inspired by Siamese and Triplet networks are commonly used to correlate among samples while using shared weights in deep metric learning. The success of these networks is based on their capacity to understand the similarity relationship among samples. Moreover, sampling strategy, appropriate distance metric, and the structure of the network are the challenging factors for researchers to improve the performance of the network model. This article is considered to be important, as it is the first comprehensive study in which these factors are systematically analyzed and evaluated as a whole and supported by comparing the quantitative results of the methods.

show abstract

Section: Deep Metric Learning Problemsmentioning

confidence: 99%

Deep Metric Learning: A Survey

2019

View full text Add to dashboard Cite

show abstract

“…Classifying duplicate questions can be a tricky task since the variability of language makes it difficult to know the actual meaning of a sentence with certainty. This task is similar to the paraphrase identification problem, which is a thoroughly researched Natural Language Processing (NLP) task [4]. It uses Natural Language Sentence Matching (NLSM) to decide whether a pair of sentences with same intent is written in different words or not [5].…”

Section: Related Workmentioning

confidence: 99%

“…The reason for choosing Manhattan Distance among other similarity measure is that we are working on large set of word embedding consisting of multiple dimensions. It has been observed by many researchers that Manhattan distance similarity measure not only performs well on very high dimensional data but also takes less time for computation since Manhattan distance finds the similarity between textual features by calculating the absolute distance between two points that lies at axes of right angle [4], [6], [23]. Manhattan equation for two points x and y is shown in Equation 7…”

Section: ) Malstmmentioning

confidence: 99%

Duplicate Questions Pair Detection Using Siamese MaLSTM

et al. 2020

View full text Add to dashboard Cite

Quora is a growing platform comprising a user generated collection of questions and answers. The questions and answers are created, edited, and organized by the users. Enormous number of users on the Quora website makes it unavoidable to have multiple questions from different users with similar intent, which raises the issue of duplicate questions. Effectively detecting duplicate questions would make it easier to find high quality answers and help save time, which in turn would result in an improved user experience for writers and readers on Quora. In this paper, Quora Question Pairs dataset is collected from Kaggle for detection of duplicate questions. First, three types of word embeddings involving Google news vector embedding, FastText crawl embedding with 300 dimensions, and FastText crawl sub words embedding with 300 dimensions are implemented individually to vectorize all the questions and train the model. The final features used for prediction are blend of these three types of word embeddings. Then, Siamese MaLSTM (''Ma'' for Manhattan distance) Neural Network model is applied for prediction of duplicate questions in the dataset. Finally, the model is tested on 100000 pairs of questions. The experiments show that the proposed model achieves 91.14% accuracy which is better than the state-of-the-art models. INDEX TERMS Duplicate question pair detection, text mining, deep learning, MaLSTM, word embedding.

show abstract

“…The process, as shown in Fig Secara tidak langsung model genism keyed vector ini akan membuat kamus yang menghasilkan dari library word2vec dan data teks yang dimiliki. Sehingga akan menghasilkan suatu kamus baru yang sesuai dengan kasus pada penelitian ini [18].…”

Section: A Word2vecunclassified

Paraphrase Detection Using Manhattan's Recurrent Neural Networks and Long Short-Term Memory

Aziz¹,

Djamal²,

Ilyas³

2019

EECSI

View full text Add to dashboard Cite

Natural Language Processing (NLP) is a part of artificial intelligence that can extract sentence structures from natural language. Some discussions about NLP are widely used, such as Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) to summarize papers with many sentences in them. Siamese Similarity is a term that applies repetitive twin network architecture to machine learning for sentence similarity. This architecture is also called Manhattan LSTM, which can be applied to the case of detecting paraphrase sentences. The paraphrase sentence must be recognized by machine learning first. Word2vec is used to convert sentences to vectors so they can be recognized in machine learning. This research has developed paraphrase sentence detection using Siamese Similarity with word2vec embedding. The experimental results showed that the amount of training data is dominant to the new data compared to the number of times and the variation in training data. Obtained data accuracy, 800,000 pairs provide accuracy reaching 99% of training data and 82.4% of new data. These results are better than the accuracy of the new data, with half of the training data only yielding 64%. While the amount of training data did not effect on training data.

show abstract

Dependency-based Siamese long short-term memory network for learning sentence representations

Cited by 40 publications

References 13 publications

Deep Metric Learning: A Survey

Deep Metric Learning: A Survey

Duplicate Questions Pair Detection Using Siamese MaLSTM

Paraphrase Detection Using Manhattan's Recurrent Neural Networks and Long Short-Term Memory

Contact Info

Product

Resources

About