Feature instability as a criterion for selecting potential style markers

Abstract

We introduce a new measure on linguistic features, called stability, which captures the extent to which a language element, such as a word or a syntactic construct, is replaceable by semantically equivalent elements.This measure may be perceived as quantifying the degree of available "synonymy" for a language item. We show that frequent but unstable features are especially useful as discriminators of an author's writing style. IntroductionOften we wish to find linguistic markers that distinguish the writing style of a particular author or class of authors. We seek features that are typically used variably by different authors but are used consistently by any given author -or, at least, by the author whose writing we wish to distinguish. Obviously, these markers will differ from author to author.In this paper, we wish to identify the pool of potentially useful linguistic features from which markers might fruitfully be chosen. To be precise, we do not seek features that distinguish a particular author but rather author-independent criteria for ranking features which are worth considering when seeking distinguishing characteristics of any given author.Consider some examples. If a particular author were found to use the word awful more frequently than other authors, this would certainly be worth noting. After all, the word bad is generally a reasonable, and more common, alternative to awful. The fact that our author chooses to use the word awful frequently therefore likely reflects deliberate stylistic choice that we might profitably exploit for identifying the author's writing. What about the word touchdown? The frequency of use of touchdown is a rather poor feature for identifying an author's style. There are actually two distinct reasons that this is so. First, there is no alternative word for expressing the concept touchdown so that its use does not reflect stylistic choice. Second, touchdown is tightly tied to a particular topic so that its frequency of use in one corpus is unlikely to reflect the frequency with which it will be used in other documents, which might concern other topics. By contrast, awful is commonly used across most topics and has plausible alternatives. Some words satisfy one of these two criteria but not both. For example, perspire offers a plausible alternative (sweat) but is not frequently used across a broad range of topics. On the other hand, words like blue and ten are used across topics but don't have common alternatives.To summarize, we seek linguistic features the use of which might reflect deliberate stylistic choice by a given author.Such features will tend to be used across topics and will offer plausible linguistic alternatives. The first criterion is relatively easy to approximate by checking overall frequency of use across different topic areas. We will focus on a formal definition of the second criteria: how can we determine the extent to which a particular linguistic feature offers alternatives? Although the examples we gave are all words, the criteria we define will ...

Computational methods in authorship attribution

“…Many studies since that of Mosteller and Wallace have shown the efficacy of function words for authorship attribution in different scenarios (Morton 1978;Burrows 1987;Karlgren & Cutting 1994;Merriam & Matthews 1994;Kessler et al 1997;Argamon et al 1998;Holmes 1998;de Vel et al 2001;Holmes et al 2001aHolmes et al , 2001bBaayen et al 2002;Binongo 2003;Juola & Baayen 2003;Zhao & Zobel 2005;Argamon & Levitan 2005;Koppel et al 2005Koppel et al , 2006a, confirming the hypothesis that different authors tend to have different characteristic patterns of function word use.…”

“…More recently, Graham et al (2005) and Zheng et al (2006) used neural networks on a wide variety of features. Other studies used k-nearest neighbor (Kjell et al 1995;Hoorn et al 1999;Zhao & Zobel 2005), Naive Bayes (Kjell 1994a;Hoorn et al 1999;Peng et al 2004), rule learners (Holmes & Forsyth 1995;Holmes 1998;Argamon et al 1998;Koppel & Schler 2003;Abbasi & Chen 2005;Zheng et al 2006), support vector machines (De Vel et al 2001;Diederich et al 2003;Koppel & Schler 2003, Abbasi & Chen 2005Koppel et al 2005;Zheng et al 2006), Winnow (Koppel et al 2002;Argamon et al 2003;Koppel et al 2006a), and Bayesian regression Madigan et al 2006;Argamon et al 2008). Further details regarding these studies can be found in the Appendix.…”

“…A number of studies used the output of syntactic text chunkers and parsers to create features, and found that they could considerably improve results based on traditional word based analysis alone (Baayen et al 1996;Stamatatos et al 2000Stamatatos et al , 2001Gamon 2004;van Halteren 2004;Chaski 2005;Uzuner and Katz 2005;Hirst & Feiguina 2007). Many studies have used the frequencies of short sequences of parts-of-speech (or combinations of parts-of-speech and other classes of words) as a simple method for approximating syntactic features for this purpose (Argamon-Engelson et al 1998;Kukushkina et al 2001;De Vel et al 2001;Koppel et al 2002;Koppel & Schler 2003;Chaski 2005;Koppel et al 2005Koppel et al , 2006avan Halteren et al 2005;Zhao et al 2006;Zheng et al 2006).…”

A survey of modern authorship attribution methods

“…The comparison of this method with information gain, a well-known feature selection algorithm examining the discriminatory power of features individually (Forman, 2003), showed that the frequencybased feature set was more accurate for feature sets comprising up to 4,000 features. Similarly, Koppel, Akiva, and Dagan (2006) presented experiments comparing frequencybased feature selection with odds-ratio, another typical feature selection algorithm using discrimination information (Forman, 2003). More important, the frequency information they used was not extracted from the training corpus.…”

Tensor Space Models for Authorship Identification

“…As it has been demonstrated by several authorship identification studies, the frequency of features is a crucial factor for their significance [14,19]. Actually, the frequency information is more important than the discriminatory power of the features when examined individually.…”