Towards a better understanding of Burrows's Delta in literary authorship attribution

Evert, Stefan; Proisl, Thomas; Vitt, Thorsten; Schöch, Christof; Jannidis, Fotis; Pielström, Steffen

doi:10.3115/v1/w15-0709

Cited by 12 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…8 Viz. the success story of the metric of Burrows's Delta (Burrows 2002), e. g., applied to German literary history by Jannidis and Lauer (2014), and scrutinized methodologically by Evert et al (2015). 9 See e. g., Underwood 2015, but also earlier work such as Biber and Finegan 1989.…”

Section: Preliminariesmentioning

confidence: 99%

Computational Text Analysis within the Humanities

Kuhn¹

2020

Reflektierte Algorithmische Textanalyse

View full text Add to dashboard Cite

This position paper is based on a keynote presentation at the COLING 2016 Workshop on Language Technology for Digital Humanities (LT4DH) in Osaka, Japan. It departs from observations about working practices in Humanities disciplines following a hermeneutic tradition of text interpretation vs. the methodoriented research strategies in Computational Linguistics (CL). The respective praxeological traditions are quite different. Yet more and more researchers are willing to open up towards truly transdisciplinary collaborations, trying to exploit advanced methods from CL within research that ultimately addresses questions from the traditional humanities disciplines and the social sciences. The article identifies two central workflow-related issues for this type of collaborative project in the Digital Humanities (DH) and Computational Social Science: (i) a scheduling dilemma, which affects the point in the course of the project when specifications of the core analysis task are fixed (as early as possible from the computational perspective, but as late as possible from the Humanities perspective) and (ii) the subjectivity problem, which concerns the degree of intersubjective stability of the target categories of analysis. CL methodology demands high interannotator agreement and theory-independent categories, while the categories in hermeneutic reasoning are often tied to a particular interpretive approach (viz. a theory of literary interpretation) and may bear a non-trivial relation to a reader's pre-understanding. Building a comprehensive methodological framework that helps overcome these issues requires considerable time and patience. The established computational methodology has to be gradually opened up to more hermeneutically oriented research questions; resources and tools for the relevant categories of analysis have to be constructed. This article does not call into question that well-targeted efforts along this path are worthwhile. Yet, it makes the following additional programmatic point regarding directions for future research: It might be fruitful to explore-in parallel-the potential lying in DH-specific variants of the concept of rapid Note: This article is a slightly revised version of: Jonas Kuhn (2019).

show abstract

Section: Preliminariesmentioning

confidence: 99%

Computational Text Analysis within the Humanities

Kuhn¹

2020

Reflektierte Algorithmische Textanalyse

View full text Add to dashboard Cite

show abstract

“…While contemporary research varies in the type of features and algorithms, all have roots in observation that grammatical words are strong predictors of style and author [8] especially those occupying the top of the frequency list (following Zipf's law). Another method that revolutionized the field was Burrows's Delta [11] (later perfected by Evert et al [12,13]), which allowed for calculating differences between profiles of feature frequencies in a more balanced way than the one provided by Euclidean distances.…”

Section: Introductionmentioning

confidence: 99%

Parallel Stylometric Document Embeddings with Deep Learning Based Language Models in Literary Authorship Attribution

et al. 2022

View full text Add to dashboard Cite

This paper explores the effectiveness of parallel stylometric document embeddings in solving the authorship attribution task by testing a novel approach on literary texts in 7 different languages, totaling in 7051 unique 10,000-token chunks from 700 PoS and lemma annotated documents. We used these documents to produce four document embedding models using Stylo R package (word-based, lemma-based, PoS-trigrams-based, and PoS-mask-based) and one document embedding model using mBERT for each of the seven languages. We created further derivations of these embeddings in the form of average, product, minimum, maximum, and l2 norm of these document embedding matrices and tested them both including and excluding the mBERT-based document embeddings for each language. Finally, we trained several perceptrons on the portions of the dataset in order to procure adequate weights for a weighted combination approach. We tested standalone (two baselines) and composite embeddings for classification accuracy, precision, recall, weighted-average, and macro-averaged F1-score, compared them with one another and have found that for each language most of our composition methods outperform the baselines (with a couple of methods outperforming all baselines for all languages), with or without mBERT inputs, which are found to have no significant positive impact on the results of our methods.

show abstract

“…In addition to training classifiers, we also briefly explored using distance measures from stylistics and authorship attribution research such as Burrow's Delta (Burrows, 2002) and other related measures (Evert et al, 2015) using 100-1000 most frequent word, character and POS n-grams in the corpus. We did not find them particularly useful for this task, with highest accuracies of less than 60% on the development set.…”

Section: Mixed Word-pos Representationsmentioning

confidence: 99%

A study of N-gram and Embedding Representations for Native Language Identification

Vajjala¹,

Banerjee²

2017

Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

We report on our experiments with Ngram and embedding based feature representations for Native Language Identification (NLI) as a part of the NLI Shared Task 2017 (team name: NLI-ISU). Our best performing system on the test set for written essays had a macro F1 of 0.8264 and was based on word uni, bi and trigram features. We explored n-grams covering word, character, POS and word-POS mixed representations for this task. For embedding based feature representations, we employed both word and document embeddings. We had a relatively poor performance with all embedding representations compared to n-grams, which could be because of the fact that embeddings capture semantic similarities whereas L1 differences are more stylistic in nature.

show abstract

Towards a better understanding of Burrows's Delta in literary authorship attribution

Cited by 12 publications

References 16 publications

Computational Text Analysis within the Humanities

Computational Text Analysis within the Humanities

Parallel Stylometric Document Embeddings with Deep Learning Based Language Models in Literary Authorship Attribution

A study of N-gram and Embedding Representations for Native Language Identification

Contact Info

Product

Resources

About