Mind your corpus: systematic errors in authorship attribution

Abstract

In computational stylistics, any influence of unwanted noise-e.g. caused by an untidilyprepared corpus-might lead to biased or false results. Relying on contaminated data is quite similar to using dirty test tubes in a laboratory: it inescapably means falling into systematic error. An important question is what degree of nonchalance is acceptable to obtain sufficiently reliable results. The present study attempts to verify the impact of unwanted noise in a series of experiments conducted on several corpora of English, German, Polish, Ancient Greek and Latin prose texts. In 100 iterations, a given corpus was gradually damaged, and controlled tests for authorship were applied. The first experiment was designed to show the correlation between a dirty corpus and attribution accuracy. The second was aimed to test how disorder in word frequencies-produced by scribal and/or editorial modifications-affects the attribution abilities of particular corpora. The goal of the third experiment was to test how much 'authorial' data a given text needs to have in order to trace authorial fingerprint through a mass of external quotations.

Textual Cultures 12.2

“…9b). This result confirmed the robustness of stylometric methods with (slightly) noisy texts, as already suggested by Eder (2012) and . * Ten texts were excluded from the stylometric analysis: nine because they were too short (under 500 words) and one because it had already been attributed to Musil on the basis of philological proof (cf.…”

A Digital Edition between Stylometry and OCR:

“…9b). This result confirmed the robustness of stylometric methods with (slightly) noisy texts, as already suggested by Eder (2012) and Franzini et al (2018). * Ten texts were excluded from the stylometric analysis: nine because they were too short (under 500 words) and one because it had already been attributed to Musil on the basis of philological proof (cf.…”

“…A closer analysis of the noisiest texts confirmed that these peaks issue primarily from errors in image segmentation: in many cases, the correct reading order was not respected, or text regions from different articles were incorrectly intermixed. Apart from these errors, however, the situation appeared quite promising, with a mean character error rate of 2-3%, which is generally considered as a high standard in OCR quality (Fink, Schulz, and Springmann 2017) and which may not influence significantly a stylometric analysis (Eder 2012). For these reasons, instead of proceeding with a manual transcription of the TSZ articles, I decided simply to re-apply the OCR process, while improving the quality of the process as much as possible.…”

Evaluating the Impact of OCR Errors on Topic Modeling

“…While OCR errors remain part of a wider problem of dealing with "noise" in text mining [23], their impact varies depending on the task performed [24]. NLP tasks such as machine translation, sentence boundary detection, tokenization, and part-of-speech tagging on text among others can all be compromised by OCR errors [25].…”