Off the Record: Unrecorded Legislative Votes, Selection Bias and Roll-Call Vote Analysis

Abstract

Scholars often use roll-call votes to study legislative behaviour. However, many legislatures only conclude a minority of decisions by roll call. Thus, if these votes are not a random sample of the universe of votes cast, scholars may be drawing misleading inferences. In fact, theories over why roll-call votes are requested would predict selection bias based on exactly the characteristics of legislative voting that scholars have most heavily studied. This article demonstrates the character and severity of this sampling problem empirically by examining European Parliament vote data for a whole year. Given that many legislatures decided only a fraction of their legislation by roll call, these findings have potentially important implications for the general study of legislative behaviour.

Voting Behaviour in the European Parliament and Economic Governance Reform: Does Nationality Matter?

“…Methodologically, this research been criticised because traditionally roll call votes made up only 15-30% of all votes in the EP, and were likely called for a variety of strategic reasons including the wish to enforce discipline and to signal a particular stance; roll call votes might therefore represent a biased sample of the population of votes (Carrubba et al 2004;Carrubba et al 2006;Høyland 2010). Some of these critiques have been addressed by demonstrating the absence of any strategic selection bias between requested and mandated roll call votes, but the possible bias between roll call votes and secret votes has not been explained so far (Hix et al 2013).…”

Does Bicameralism Promote Stability? Inter-institutional Relations and Coalition Formation in the European Parliament

“…While it is possible that roll-calls are requested for strategic reasons, for example to demonstrate group cohesion or to embarrass opponents (see Carrubba et al 2006), there is unlikely to be a bias in terms of coalition patterns or the frequency with which each group is on the winning side. The analysis is restricted to votes from the 5 th EP parliamentary term (1999)(2000)(2001)(2002)(2003)(2004).…”

Chaotic shop-talk or efficient parliament? The Reichstag, the parties, and the problem of governmental instability in the Weimar Republic

“…It is not possible to calculate exactly how representative the roll-call votes were of all votes, since the total number of votes is not reported separately. I am aware of the problems involved in drawing general conclusions about legislative behavior based on a sample which might not reflect the behavior in all parliamentary debates (Carrubba et al 2006). However, the reported roll-call votes in the Weimar Republic include all important votes of that period, such as votes on the London Ultimate and the Treaty of Versailles.…”