Do they work? Validating computerised word frequency estimates against policy series

Budge, Ian; Pennings, Paul

doi:10.1016/j.electstud.2006.04.002

Cited by 63 publications

(53 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…1 and 2 of Budge and Pennings, 2007) but Table 1 (also in Budge and Pennings, 2007) on 'forwards' and 'backwards' estimates which conflict. Incidentally, these are more severe when we use Castles and Mair's (1984) expert scores than when we use the Manifesto scores.…”

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 96%

“…The Conservatives (in Fig. 1c of Budge and Pennings, 2007) are shown as fairly static but that is probably correct. Were we only conducting a pairwise comparison of 1992e1997 as LBG did (2003, 319e321) the lurch rightwards by Labour shown in Fig.…”

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

“…Actually Fig. 1a,b in Budge and Pennings (2007), for the British Labour and Liberal parties, show Wordscores tracing out quite a lot of movement on their part. The Conservatives (in Fig.…”

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

“…Were we only conducting a pairwise comparison of 1992e1997 as LBG did (2003, 319e321) the lurch rightwards by Labour shown in Fig. 1a of Budge and Pennings (2007) would be taken as a success for the Wordscores comparison. So would the move leftwards from 1979 to 1983.…”

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 4 more Smart Citations

Missing the message and shooting the messenger: Benoit and Laver's ‘response’

Budge¹,

Pennings²

2007

Electoral Studies

Self Cite

View full text Add to dashboard Cite

Benoit and Laver (BL; 2007-a-b) ignore the main point made in our article (Budge and Pennings, 2007), about the unreliability of policy estimates made by word frequency methods in the absence of authoritative 'calibrating' texts. Instead they concentrate on the general unreliability of the MRG/CMP Manifesto data (Budge et al., 2001) as a 'benchmark', in spite of the fact that we also used expert scores (Castles and Mair, 1984). Accordingly we divide this reply into two parts. The first deals with questions BL avoid but which provide a context for their criticism of aggregated reference documents. The second responds directly to their datacritiques, showing that they are mostly partial and misleading.1. The general instability of word frequency estimates BL (p. 1) characterize our article as 'criticizing the 'Wordscores' method for computerized content analysis .'. Actually it does not. 'Wordscores' is a useful computer programme 1 which operationalizes a particular approach within the range of possible word frequency techniques. But it is not the only approach. Indeed 'Wordscores' was clearly conceived within the conceptual framework of calibrating and application documents and word weightings provided by Pennings (1999, 2001) and described at the outset of our article. The Laver team abandoned an earlier and cruder approach to computerized analysis (Laver and Garry, 2000) in favour of simplifying and operationalizing this word frequency one. 2 So we have at least two extant applications of word frequency analysis to political texts. 3 The fact that we began with, and returned to, Kleinnijenhuis and Pennings's work at various points demonstrates our general concern with word frequency analysis as an approach rather than Wordscores as such. A check against the Manifesto Left-Right series was Kleinnijenhuis and Pennings's preferred test for their own word-frequency approach. So it was natural Ã Corresponding author. Fax: þ33 20 5986820.E-mail address: pjm.pennings@fsw.vu.nl (P. Pennings). 1 Indeed as we made heavy use of it to support our points about the word frequency approach, we ought to commend the speed with which its authors have made it generally available and in particular Ken Benoit's role in supporting and disseminating it among the professional community.2 Laver et al. (2003, 312) list Kleinnijenhuis and Pennings's article among earlier and quite disparate computerized content analyses but do not refer to it otherwise, or in their response, in spite of its prominence in our discussion.3 Word frequency approaches antedate both sets of authors of course. For example, a project in the Edinburgh Faculty of Divinity in the mid-1950s tried to establish the exact authorship of St Paul's Epistles through word counts. It stopped after concluding they could not all have been written by the same person but could not decide where St Paul was author because they lacked an authoritative reference text indubitably by him. This is the general problem we identify for word-frequency policy estimates.0261-3794/$ -se...

show abstract

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 96%

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

“…Actually Fig. 1a,b in Budge and Pennings (2007), for the British Labour and Liberal parties, show Wordscores tracing out quite a lot of movement on their part. The Conservatives (in Fig.…”

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

Section: The General Instability Of Word Frequency Estimatesmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 3 more Smart Citations