Assessing the Stylistic Properties of Neurally Generated Text in
            Authorship Attribution

Manjavacas, Enrique; Gussem, Jeroen De; Daelemans, Walter; Kestemont, Mike

doi:10.18653/v1/w17-4914

Cited by 10 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, previous AA work largely focuses on authorship attribution among humans, while only a few papers (Manjavacas et al, 2017;Uchendu et al, 2020;Munir et al, 2021) study neural generated text. Our work aims to provide the first benchmark for Authorship Attribution in the form of the Turing Test by including humans and neural language models.…”

Section: Related Workmentioning

confidence: 99%

TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

Uchendu¹,

Ma²,

Le³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Recent progress in generative language models has enabled machines to generate astonishingly realistic texts. While there are many legitimate applications of such models, there is also a rising need to distinguish machine-generated texts from human-written ones (e.g., fake news detection). However, to our best knowledge, there is currently no benchmark environment with datasets and tasks to systematically study the so-called "Turing Test" problem for neural text generation methods. In this work, we present the TURINGBENCH benchmark environment, which is comprised of (1) a dataset with 200K human-or machine-generated samples across 20 labels {Human,

show abstract

Section: Related Workmentioning

confidence: 99%

TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

Uchendu¹,

Ma²,

Le³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

show abstract

“…AA can also be applied to predicting author(s) of source code (Simko et al, 2018), chatbot detec-tion , and even detecting authors intentionally trying to mask their writing style (Juola, 2012;Sánchez-Junquera et al, 2020). Finally, our work bears similarity to (Manjavacas et al, 2017), which investigates the stylistic properties of different neural text generation techniques (i.e., Ngram-based and RNN-based).…”

Section: Applications Of Authorship Attributionmentioning

confidence: 99%

Authorship Attribution for Neural Text Generation

Uchendu¹,

Le²,

Shu³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

In recent years, the task of generating realistic short and long texts have made tremendous advancements. In particular, several recently proposed neural network-based language models have demonstrated their astonishing capabilities to generate texts that are challenging to distinguish from human-written texts with the naked eye. Despite many benefits and utilities of such neural methods, in some applications, being able to tell the "author" of a text in question becomes critically important. In this work, in the context of this Turing Test, we investigate the so-called authorship attribution problem in three versions: (1) given two texts T 1 and T 2 , are both generated by the same method or not? (2) is the given text T written by a human or machine? (3) given a text T and k candidate neural methods, can we single out the method (among k alternatives) that generated T ? Against one humanwritten and eight machine-generated texts (i.e., CTRL, GPT, GPT2, GROVER, XLM, XL-NET, PPLM, FAIR), we empirically experiment with the performance of various models in three problems. By and large, we find that most generators still generate texts significantly different from human-written ones, thereby making three problems easier to solve. However, the qualities of texts generated by GPT2, GROVER, and FAIR are better, often confusing machine classifiers in solving three problems. All codes and datasets of our experiments are

show abstract

“…Alsulami et al [19] used Long shortterm memory (LSTM) for source code authorship classification with 200 source files from 10 programmers and gained 85.00% accuracy. Enrique et al [13] assessed the quality of neurally generated English text using an established authorship identification. Koppel et al [20] used a naive similaritybased techniques for authorship classification in English texts.…”

Section: A Non-bengali Language-based Authorship Classificationmentioning

confidence: 99%

“…To build this corpus, we crawled texts from four online sources namely NLTR society for natural language technology research [36], Ebanglalibrary [37], Git repository [38] and Blogs [39]- [41]. The maximum number of texts (13,308) are collected from NLTR source whereas minimum number of texts (240) are crawled from Blogs. A self-built automatic web crawler 3 is used to scrapping the data from four sources.…”

Section: ) Bengali Authorship Classification Corpus (Bacc-18)mentioning

confidence: 99%

“…The statistical methods are used for extracting stylometry features [2]- [5], character level embedding features [6], [7], and n-gram features [8], [9] for authorship attribution. Several ML techniques such as support vector machine (SVM), Naive Bayes (NB) [10]- [12], CNN [6], [13]- [15] are the most commonly used methods for authorship classification. Nevertheless, stylometry, character level, and n-gram feature extractors cannot capture the sentence and document level semantic and syntactic features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Authorship Classification in a Resource Constraint Language Using Convolutional Neural Networks

et al. 2021

View full text Add to dashboard Cite

Authorship classification is a technique of automatically determining the appropriate author of an unknown linguistic text. Although research on authorship classification has significantly progressed in high-resource languages, it is at a primitive stage in the realm of resource-constraint languages like Bengali. This paper presents an authorship classification system made of Convolution Neural Networks (CNN) comprising four modules: embedding model generation, feature representation, classifier training and classifier testing. For this purpose, this work develops a new embedding corpus (named WEC) and a Bengali authorship classification corpus (called BACC-18), which are more robust in terms of authors' classes and unique words. Using three text embedding techniques (Word2Vec, GloVe and FastText) and combinations of different hyperparameters, 90 embedding models are created in this study. All the embedding models are assessed by intrinsic evaluators and selected the best 9 performing models out of the 90 models for the authorship classification. In total 36 classification models, including four classification models (CNN, LSTM, SVM, SGD) and three embedding techniques with 100, 200 and 250 embedding dimensions, are trained with optimized hyperparameters and tested on three benchmark datasets BAAD16 and LD). Among the models, the optimized CNN with GloVe model achieved the highest classification accuracies of 93.45%, 95.02%, and 98.67% for the datasets BACC-18, BAAD16, and LD, respectively.INDEX TERMS Natural language processing, Authorship classification, resource constraint language, semantic feature extraction, deep learning.

show abstract

Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution

Cited by 10 publications

References 20 publications

TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

Authorship Attribution for Neural Text Generation

Authorship Classification in a Resource Constraint Language Using Convolutional Neural Networks

Contact Info

Product

Resources

About