Identifying Similar Sentences by Using N-Grams of Characters

Sultana, Saima; Biskri, Ismaïl

doi:10.1007/978-3-319-92058-0_80

Cited by 4 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Elhadi (2012) introduces a method to calculate text similarity by comparing the longest common sequence between two texts. Sultana and Biskri (2018) propose another method by utilizing the N-grams of characters. N-grams are a subsequence of characters or words that are contained in a sentence or text.…”

Section: String-based Methodsmentioning

confidence: 99%

Short text similarity measurement methods: a review

2021

View full text Add to dashboard Cite

Short text similarity measurement methods play an important role in many applications within natural language processing. This paper reviews the research literature on short text similarity (STS) measurement method with the aim to (i) classify and give a broad overview of existing techniques; (ii) find out its strengths and weaknesses in terms of the domain the independence, language independence, requirement of semantic knowledge, corpus and training data, ability to identify semantic meaning, word order similarity and polysemy; and (iii) identify semantic knowledge and corpus resource that can be utilized for the STS measurement methods. Furthermore, our study also considers various issues such as the difference between the various text similarity methods and the difference between semantic knowledge sources and corpora for text similarity. Although there are a few review papers in this area, they focus mostly only on one/two existing techniques. Furthermore, existing review papers do not cover recent research. To the best of our knowledge, this is a comprehensive systematic literature review on this topic. The findings of this research can be as follows: It identified four semantic knowledge and eight corpus resources as external resources that can be classified into general-purpose and domain-specific. Furthermore, the existing techniques can be classified into string-based, corpus-based, knowledge-based and hybrid-based. Moreover, expert researchers can utilize this review as a benchmark as well as reference to the limitations of current techniques. The paper also identifies the open issues that can be considered as feasible opportunities for future research directions.Keywords Natural language processing Á Text mining Á Linguistics Á Similarity measures Á Semantics Á Syntax Communicated by V. Loia.

show abstract

Section: String-based Methodsmentioning

confidence: 99%

Short text similarity measurement methods: a review

2021

View full text Add to dashboard Cite

show abstract

“…Полученные числовые векторы сравнивались математическими метриками расстояний: евклидово, косинусное, манхэттенское, расстояние Хэмминга [7]. Часто сходство вычислялось путем выделения самой длинной общей подстроки [8] или с использованием 𝑛-грамм [9].…”

Section: обзор научных исследованийunclassified

Automatic determination of semantic similarity of student answers with the standard one using modern models

Lagutina,

Kopnin

2024

Model. anal. inf. sist.

View full text Add to dashboard Cite

The paper presents the results of a study of modern text models in order to identify, on their basis, the semantic similarity of English-language texts. The task of determining semantic similarity of texts is an important component of many areas of natural language processing: machine translation, information retrieval, question and answer systems, artificial intelligence in education. The authors solved the problem of classifying the proximity of student answers to the teacher’s standard answer. The neural network language models BERT and GPT, previously used to determine the semantic similarity of texts, the new neural network model Mamba, as well as stylometric features of the text were chosen for the study. Experiments were carried out with two text corpora: the Text Similarity corpus from open sources and the custom corpus, collected with the help of philologists. The quality of the problem solution was assessed by precision, recall, and F-measure. All neural network language models showed a similar F-measure quality of about 86% for the larger Text Similarity corpus and 50–56% for the custom corpus. A completely new result was the successful application of the Mamba model. However, the most interesting achievement was the use of vectors of stylometric features of the text, which showed 80% F-measure for the custom corpus and the same quality of problem solving as neural network models for another corpus.

show abstract

“…(3) Greedy String Tilling -Karp Robin Problem (GST-KRP) Michael J. Wise [18] proposed the use of the "Greedy String Tiling and Running Karp-Rabin matching (GST -KRP)" algorithm in the areas of plagiarism/similarity detection and other applications such as DNA Matching, amino acids, etc. GST algorithm is based on the concept of one-to-one string matching and can deal with the substring transposition.…”

Section: Local Similarity Algorithmsmentioning

confidence: 99%

Developing Machine Learning Coding Similarity Indicators for C and C++ Corpuses

Kunjir¹,

Fiaidhi²

2020

AJEMR

View full text Add to dashboard Cite

In the digital era of technology and advanced automation, data or information is vulnerable to copying, altering, and claiming someone else's work as their own. Source code theft or e-plagiarism is challenging to track in hundreds of assignments submitted by students. Despite the year's efforts, the digital plagiarism detection software currently available performs well enough for a naïve programmer to detect literal plagiarism. The available source code similarity detectors provide insufficient results when a student uses complex strategies such as word substitution or reordering programming constructs. To overcome the above-mentioned challenges, this research aims to deliver an assistive forensic engine for the professors and teaching assistants to evaluate the similarities in the student's assignments. This research's primary objective is to help the evaluators get closer to the sophisticated code thieves and abide by the university's academic dishonesty regulations. The proposed forensic similarity detection engine's constructive methodology is specially designed for studies where C and C++ programming languages are majorly used in academic assignments. After selecting the ATM (Attribute counting metrics), the system implementation is divided into two phases, where phase one consists of lexical analysis and tokenizer customization. The second phase mostly consists of rolling out the supervised learning algorithm on the generated data to classify the comparison of two files as a truth value. The similarity elements and observations recorded can be represented to the evaluators in the form of visualizations for ease of understanding and efficient decision making. The paper also relates the proposed system with the previous and existing system and mitigates the past issues noted in the latter half.

show abstract

Identifying Similar Sentences by Using N-Grams of Characters

Cited by 4 publications

References 9 publications

Short text similarity measurement methods: a review

Short text similarity measurement methods: a review

Automatic determination of semantic similarity of student answers with the standard one using modern models

Developing Machine Learning Coding Similarity Indicators for C and C++ Corpuses

Contact Info

Product

Resources

About