Analysis of word co-occurrence in human literature for supporting semantic correspondence discovery

Martinez‐Gil, Jorge; Pichler, Mario

doi:10.1145/2637748.2638422

Cited by 3 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This dataset is used as benchmark to evaluate technique of sentences similarity 40 . Moreover, there are many other datasets that used in this field such asQAsnt 14…”

Section: Sick Datasetmentioning

confidence: 99%

“…In this type, valuable information is extracted from analyzing a big corpus. Moreover, analyzing words co-occurrence in a big corpus helps to assess similarity between these words accurately 14 .…”

Section: Corpus Based Words Similaritymentioning

confidence: 99%

“…This dataset is used as benchmark to evaluate technique of sentences similarity 40 . Moreover, there are many other datasets that used in this field such asQAsnt 14 , WikiQA 56 which is new public available for question answering and sentences paring in open domain.…”

Section: Sick Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Measuring Sentences Similarity: A Survey

Farouk¹

2019

Indian Journal of Science and Technology

View full text Add to dashboard Cite

Objective/Methods: This study is to review the approaches used for measuring sentences similarity. Measuring similarity between natural language sentences is a crucial task for many Natural Language Processing applications such as text classification, information retrieval, question answering, and plagiarism detection. This survey classifies approaches of calculating sentences similarity based on the adopted methodology into three categories. Word-to-word based, structurebased, and vector-based are the most widely used approaches to find sentences similarity. Findings/Application: Each approach measures relatedness between short texts based on a specific perspective. In addition, datasets that are mostly used as benchmarks for evaluating techniques in this field are introduced to provide a complete view on this issue. The approaches that combine more than one perspective give better results. Moreover, structure based similarity that measures similarity between sentences' structures needs more investigation.

show abstract

“…This dataset is used as benchmark to evaluate technique of sentences similarity 40 . Moreover, there are many other datasets that used in this field such asQAsnt 14…”

Section: Sick Datasetmentioning

confidence: 99%

Section: Corpus Based Words Similaritymentioning

confidence: 99%

See 1 more Smart Citation

Measuring Sentences Similarity: A Survey

Farouk¹

2019

Indian Journal of Science and Technology

View full text Add to dashboard Cite

show abstract

“…It believes that semantically similar words would appear in similar manner; there is not much different in structure, or pattern, which similar words appear in. Word co-occurrence is also another key aspect which is used to indicate word similarity [7]. Two techniques used in analysing this approach are normal statistical analysis and deep learning techniques.…”

Section: Word Similarity Methodsmentioning

confidence: 99%

Y-X-Y encoding for identifying types of sentence similarity

Jinnovart

View full text Add to dashboard Cite

The task of finding semantic similarity of any two arbitrary sentences consists of two main steps, which are encoding sentences to produce feature vectors of equal length and measuring the similarity, respectively. The quality of an encoding technique can determine the degree of success a model can achieve in measuring the similarity. This is because a good representation is subjected to how finely established the spectrum of similarities is. The clearer the definition of similarity is, the better the representations can be constructed. This, in turn, helps distinguish between types of sentences. Generally, all existing methods for measuring similarity were designed for vectorized data in a feature space of fixed dimensions. �Thus, transforming a set of various-length sentences into a set of feature vectors in the same dimension is very essential. The dataset used in this thesis provides both relatedness score and textual entailment. Textual entailment distinguishes sentence pair relations among three classes: namely, neutral, entailment and contradiction. The task indicates the types of entailments, which is interpreted as relatedness in this thesis. Additionally, powerful pretrained encoding models are usually of millions of parameters, or even billions. This is one obstacle in training one's own embedding model due to the need of resources with heavy computing capabilities. In this thesis, we propose a self-encoding scheme to classify among the three classes of textual entailment. The relevancy of all words in a sentence is simultaneously captured by this self-encoding structure. Unlike the other encoding methods based on sequential learning, no interference of memory loss due to the length of sentence occurs in this approach. The framework involves filtering contradiction pairs at an early stage and employing a set of y-x-y encoders, where y is the length after two sentences are concatenated and x is the optimal encoding size for samples of length y, and classifiers to output neutral and entailment probabilities. With over 90% accuracy for all classes, our method has proven that this task is possible to be carried out effectively without the need of large-scale datasets and heavy computational resources.�

show abstract