The False-Positive Rate of Automated Plagiarism Detection for SQL Assessments

Abstract

Automated assessment is becoming increasingly common in Computer Science and with it automated plagiarism detection is also common. However, little attention has been paid to SQL assessment where submissions are much shorter and must be less varied than in imperative languages. This brings the challenge of avoiding high false-positive rates that require manual inspection and undermine the usefulness of automated detection.In this paper we investigate the false-positive rate of various automated plagiarism detection algorithms. We find that there is a significant false-positive rate of between 15% and 64%. These results call into question the usefulness of automated detection for SQL since they imply that a lot of manual inspection will still be needed.However, our results suggest that the false-positive rate may be restricted to shorter queries (e.g. under 200 characters). Further research is needed because our datasets consist mostly of short queries and the results for longer queries are based on a small subset of the data.

Toolunterstützung für den Übungsbetrieb in der Datenbanklehre: Erfahrungen mit der Software Praktomat

“…Allerdings ist die Aussagekraft gerade bei Aufgaben auf Anfängerniveau beschränkt: Mitunter umfasst der Lösungsraum, die Anzahl der unterschiedlichen Formulierungen einer SQL-Anfrage (bis auf Groß-Kleinschreibung) nur wenige Zeichenketten, so dass ein Plagiats-Checker zu viele Falsch-Positive generieren würde. Erste Studien in diese Richtung bestätigen diese Hypothese [35].…”

Choosing Code Segments to Exclude from Code Similarity Detection

“…The last two stages (similarity confirmation and investigation) require human intervention, and their accuracy and completion time are heavily affected by the number of false positives generated by the tool; that is, student submissions that are marked as suspicious when they have not actually been copied. This number can be high if the tool's preprocessing nullifies too much program variation while the assessments are too simple, as in first-year courses [14,20]; when they use a restrictive programming language, such as SQL [27] or Verilog [44]; or when they incorporate common 'starter code' provided for them to use [3].…”