Using Compression-Based Language Models for Text Categorization

Teahan, William J.; Harper, David J.

doi:10.1007/978-94-017-0171-6_7

Cited by 92 publications

(63 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hence, it was considered the ideal testing ground for early authorship attribution studies as well as the first fully-automated approaches (Holmes & Forsyth, 1995;Tweedie, et al, 1996). It is also used in some modern studies (Teahan & Harper, 2003;Marton, et al, 2005). Although appealing, this case has a number of important weaknesses.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A survey of modern authorship attribution methods

Stamatatos

2008

J. Am. Soc. Inf. Sci.

1,080

822

View full text Add to dashboard Cite

Authorship attribution supported by statistical or computational methods has a long history starting from 19th century and marked by the seminal study of Mosteller and Wallace (1964) on the authorship of the disputed Federalist Papers. During the last decade, this scientific field has been developed substantially taking advantage of research advances in areas such as machine learning, information retrieval, and natural language processing. The plethora of available electronic texts (e.g., e-mail messages, online forum messages, blogs, source code, etc.) indicates a wide variety of applications of this technology provided it is able to handle short and noisy text from multiple candidate authors. In this paper, a survey of recent advances of the automated approaches to attributing authorship is presented examining their characteristics for both text representation and text classification. The focus of this survey is on computational requirements and settings rather than linguistic or literary issues. We also discuss evaluation methodologies and criteria for authorship attribution studies and list open questions that will attract future work in this area.

show abstract

Section: Discussionmentioning

confidence: 99%

“…It has to be underlined that the prediction by partial matching (PPM) algorithm (Teahan & Harper, 2003) that is used by RAR to compress text files works practically the same as the method of Peng, et al (2004). However, there is a significant difference with the previously described probabilistic method.…”

Section: Compression Modelsmentioning

confidence: 99%

A survey of modern authorship attribution methods

Stamatatos

2008

J. Am. Soc. Inf. Sci.

1,080

822

View full text Add to dashboard Cite

show abstract

“…Also, the method is very easy to understand and implement, unlike the PPM approach [8], with almost similar detection rates.…”

Section: Introductionmentioning

confidence: 99%

Language Identification Using G-Lda

Saini

Kasliwal

Bhatia

2013

IJRET

View full text Add to dashboard Cite

show abstract

“…Character level language models have been found to be effective in text classification [9] and author attribution [10] tasks. The present paper deals with a relatively small corpus of spoken message transcripts.…”

Section: Introductionmentioning

confidence: 99%

Automatic Categorization of Voicemail Transcripts Using Stochastic Language Models

Koumpis

2004

Text, Speech and Dialogue

View full text Add to dashboard Cite

Abstract. This paper is about the applicability of stochastic language models to the task of categorizing voicemail message transcripts. The target categories are related to priority and content and are thus suitable for mobile messaging applications based on profiles which can be determined by users' physical and social environment. Categorization is performed by comparing the posterior probabilities of test messages under the language models of each target category. Stochastic models were selected over other lexical features because of their ability to incorporate context dependencies while their parameters are determined automatically from data. Despite the relatively small amount of training data used and given the spontaneous nature of voicemail, the models performed fairly accurately. Our experiments examine the effects that factors such as the word error rate, the n-gram order, smoothing and textual representation have on overall categorization accuracy.

show abstract

Using Compression-Based Language Models for Text Categorization

Cited by 92 publications

References 14 publications

A survey of modern authorship attribution methods

A survey of modern authorship attribution methods

Language Identification Using G-Lda

Automatic Categorization of Voicemail Transcripts Using Stochastic Language Models

Contact Info

Product

Resources

About