A repetition based measure for verification of text collections and for text categorization

Khmelev, Dmitry V.; Teahan, William J.

doi:10.1145/860435.860456

Cited by 60 publications

(32 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sanderson and Guenter (2006) described the use of several sequence kernels based on character n-grams of variable-length and the best results for short English texts were achieved when examining sequences of up to 4-grams. Moreover, various Markov models of variable order have been proposed for handling character-level information (Khmelev & Teahan, 2003a;Marton, et al, 2005). Finally, Zhang and Lee (2006) constructed a suffix tree representing all possible character n-grams of variable-length and then extracted groups of character n-grams as features.…”

Section: Character Featuresmentioning

confidence: 99%

“…A quite particular case of using character information is the compression-based approaches (Benedetto, Caglioti, & Loreto, 2002;Khmelev & Teahan, 2003a;Marton, et al, 2005). The main idea is to use the compression model acquired from one text to compress another text, usually based on off-the-shelf compression programs.…”

Section: Character Featuresmentioning

confidence: 99%

“…This task is also called authorship (or author) identification usually by researchers with a background in computer science. Several studies focus exclusively on authorship attribution (Stamatatos, Fakotakis, & Kokkinakis, 2001;Keselj, Peng, Cercone, & Thomas, 2003;Zheng, Li, Chen, & Huang, 2006) while others use it as just another testing ground for text categorization methodologies (Khmelev & Teahan, 2003a;Peng, Shuurmans, & Wang, 2004;Marton, et al, 2005;Zhang & Lee, 2006). Beyond this problem, several other authorship analysis tasks can be defined, including the following:…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A survey of modern authorship attribution methods

Stamatatos

2008

J. Am. Soc. Inf. Sci.

1,080

822

View full text Add to dashboard Cite

Authorship attribution supported by statistical or computational methods has a long history starting from 19th century and marked by the seminal study of Mosteller and Wallace (1964) on the authorship of the disputed Federalist Papers. During the last decade, this scientific field has been developed substantially taking advantage of research advances in areas such as machine learning, information retrieval, and natural language processing. The plethora of available electronic texts (e.g., e-mail messages, online forum messages, blogs, source code, etc.) indicates a wide variety of applications of this technology provided it is able to handle short and noisy text from multiple candidate authors. In this paper, a survey of recent advances of the automated approaches to attributing authorship is presented examining their characteristics for both text representation and text classification. The focus of this survey is on computational requirements and settings rather than linguistic or literary issues. We also discuss evaluation methodologies and criteria for authorship attribution studies and list open questions that will attract future work in this area.

show abstract

Section: Character Featuresmentioning

confidence: 99%

Section: Character Featuresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A survey of modern authorship attribution methods

Stamatatos

2008

J. Am. Soc. Inf. Sci.

1,080

822

View full text Add to dashboard Cite

show abstract

“…This may be accounted for by using a probabilistic distance measure such as K-L divergence between Markov model probability distributions of the texts (Juola 1998;Khmelev 2001;Khmelev and Tweedie 2002;Juola & Baayen 2003;Sanderson and Guenter 2006), possibly implicitly in the context of compression methods (Kukushkina et al 2001;Benedetto et al 2002;Khmelev and Teahan 2003;Marton et al 2005).…”

Section: Multivariate Analysis Approachmentioning

confidence: 99%

Computational methods in authorship attribution

Koppel

Schler

Argamon

2008

J. Am. Soc. Inf. Sci.

462

278

View full text Add to dashboard Cite

Statistical authorship attribution has a long history, culminating in the use of modern machine learning classification methods. Nevertheless, most of this work suffers from the limitation of assuming a small closed set of candidate authors and essentially unlimited training text for each. Real-life authorship attribution problems, however, typically fall short of this ideal. Thus, following detailed discussion of previous work, three scenarios are considered here for which solutions to the basic attribution problem are inadequate. In the first variant, the profiling problem, there is no candidate set at all; in this case, the challenge is to provide as much demographic or psychological information as possible about the author. In the second variant, the needle-in-a-haystack problem, there are many thousands of candidates for each of whom we might have a very limited writing sample.In the third variant, the verification problem, there is no closed candidate set but there is one suspect; in this case, the challenge is to determine if the suspect is or is not the author. For each variant, it is shown how machine learning methods can be adapted to handle the special challenges of that variant.

show abstract

“…it offers a large pool of texts of unquestioned authorship that cover a variety of topics and it has already been used by previous studies [24,18,31]. We selected the top 10 authors with respect to the amount of texts belonging to the topic class CCAT (about corporate and industrial news) to minimize the topic factor for distinguishing between texts.…”

Section: Corpus and Settingsmentioning

confidence: 99%

Tensor Space Models for Authorship Identification

Plakias

Stamatatos

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Authorship identification can be viewed as a text categorization task. However, in this task the most frequent features appear to be the most important discriminators, there is usually a shortage of training texts, and the training texts are rarely evenly distributed over the authors. To cope with these problems, we propose tensors of second order for representing the stylistic properties of texts. Our approach requires the calculation of much fewer parameters in comparison to the traditional vector space representation. We examine various methods for building appropriate tensors taking into account that similar features should be placed in the same neighborhood. Based on an existing generalization of SVM able to handle tensors we perform experiments on corpora controlled for genre and topic and show that the proposed approach can effectively handle cases where only limited training texts are available.

show abstract

A repetition based measure for verification of text collections and for text categorization

Cited by 60 publications

References 10 publications

A survey of modern authorship attribution methods

A survey of modern authorship attribution methods

Computational methods in authorship attribution

Tensor Space Models for Authorship Identification

Contact Info

Product

Resources

About