Correlation Versus Interchangeability: The Limited Robustness of Empirical Findings on Democracy Using Highly Correlated Data Sets

Abstract

This article shows that highly correlated measures can produce different results. We identify a democratization model from the literature and test it in over 120 countries from 1951-1992. Then, we check whether the results are robust regarding measures of democracy, time periods, and levels of development. The findings show that measures do matter: while some of the findings are robust, most of them are not. This explains, in part, why the debates on democracy have continued rather than been resolved. More importantly, it underscores the need for more careful use of measures and further testing to increase confidence in the findings. Scholars in comparative politics increasingly are drawn to large-N statistical analyses, often using datasets collected by others. As in any field, we show how they must be careful in choosing the most appropriate measures for their study, without assuming that any correlated measure will do.

Electoral Democracy and Human Development

“…Issues of conceptualization and measurement are, however, inescapably theoretical; they do not flow ineluctably from a measurement model (Adcock & Collier 2001;Borsboom 2006;Munck 2009). And decisions about operationalization often have important consequences (Casper & Tufis 2003), a point that our analyses confirm.…”

Democracy, political risks and stock market performance

“…There has been some criticism of the typically used measures of democracy (Munck and Verkuilen (2002) provide a comprehensive study of the conceptualization, measuring and aggregating problems related to the measures), and we acknowledge that neither of the measures we use to quantify democracy is perfect. Furthermore, Casper and Tufis (2003) warn that even highly correlated democracy measures can produce different results; thus, researchers must justify their measurement choices carefully. Therefore, to take into account as many aspects of democracy as possible and to address data selection issues, we use two different measures for democracy: the Polity index of Polity IV and the democratic accountability index from the Political Risk Service, published in ICRG.…”

FDI and the costs of contract enforcement in developing countries

“…We include a rough proxy for the openness of the political system using the Polity IV dataset (Marshall et al 2004). While there are several (highly correlated) measures of democracy, political openness, and the like, Polity is uniquely well suited for studying political institutions (Casper and Tufis 2003). We include the 2002 Polity score for each country (POLITY) to account for the actual level of political openness in the most recent period.…”