Improving Academic Plagiarism Detection for STEM Documents by Analyzing Mathematical Content and Citations

Abstract

Identifying academic plagiarism is a pressing task for educational and research institutions, publishers, and funding agencies. Current plagiarism detection systems reliably find instances of copied and moderately reworded text. However, reliably detecting concealed plagiarism, such as strong paraphrases, translations, and the reuse of nontextual content and ideas is an open research problem. In this paper, we extend our prior research on analyzing mathematical content and academic citations. Both are promising approaches for improving the detection of concealed academic plagiarism primarily in Science, Technology, Engineering and Mathematics (STEM). We make the following contributions: i) We present a two-stage detection process that combines similarity assessments of mathematical content, academic citations, and text. ii) We introduce new similarity measures that consider the order of mathematical features and outperform the measures in our prior research. iii) We compare the effectiveness of the math-based, citation-based, and text-based detection approaches using confirmed cases of academic plagiarism. iv) We demonstrate that the combined analysis of math-based and citation-based content features allows identifying potentially suspicious cases in a collection of 102K STEM documents. Overall, we show that analyzing the similarity of mathematical content and academic citations is a striking supplement for conventional textbased detection approaches for academic literature in the STEM disciplines. The data and code of our study are openly available at https://purl.org/hybridPD

Detecting Machine-Obfuscated Plagiarism

“…Approaches analyzing nontextual content features, such as academic citations [16,28], images [25,11], and mathematical content [26,29], complement the text analysis approaches to improve the detection of concealed plagiarism.…”

Forms of Plagiarism in Digital Mathematical Libraries

“…Nontextual content features in academic documents are a valuable source of semantic information that are largely independent of natural language text. Considering these sources of semantic information for similarity analysis raises the effort plagiarists must invest for obfuscating reused content [16,22].…”

“…However, only a few studies have addressed the detection of plagiarism in digital mathematical libraries [19,21,22] regardless of the detection approach. We briefly describe the main findings of these studies hereafter.…”

“…In a follow-up study, Meuschke et al (2019) introduced similarity measures that consider the order of mathematical identifiers and presented a two-stage retrieval process consisting of a candidate retrieval and a detailed analysis stage that replaced the exclusive use of pairwise document comparisons [22]. They implemented the process in the HyPlag prototype that also offers a user interface to investigate the identified similarities [21].…”

Discovering Mathematical Objects of Interest—A Study of Mathematical Notations

“…Fundamental knowledge on frequency distributions of math formulae is beneficial for numerous applications in MathIR, ranging from educational purposes [3] to math recommendation systems, search engines [22,25], and even automatic plagiarism detection systems [29,39,41]. For example, students can search for the conventions to write certain quantities in formulae; document preparation systems can integrate an auto-completion or auto-correction service for math inputs; search or recommendation engines can adjust their ranking scores with respect to standard notations; and plagiarism detection systems can estimate whether two identical formulae indicate potential plagiarism or are just using the conventional notations in a particular subject area.…”