Marketplace of indicators: Inconsistencies between country trends of measures of the rule of law

Abstract

Social scientists can choose from among multiple quality of governance indicators which use different conceptualizations of governance and its components, rely on different data sources, and employ different aggregation and scaling techniques. Despite all differences, these indicators are commonly found to be strongly correlated, which makes the choice of indicator for a given analysis seem inconsequential. We demonstrate that the correlations between governance indicators are indeed high when comparing across countries or using pooled country-year data sets, but are surprisingly low -- sometimes even negative -- within countries. Given the increased interest of researchers in longitudinal analyses with country time series, low agreement between country time trends in the quality of governance is concerning. We illustrate the problem with an analysis of the effect of rule of law on support for democracy, which leads to opposite conclusions depending on which measure of rule of law is used.

Political trust as a cause and consequence of democracy: Longitudinal analysis of European data

“…First, there are many approaches to conceptualizing, operationalizing, and measuring democracy, resulting in a choice of datasets and indicators produced by different research projects and organizations. While across countries these different democracy indicators are relatively consistent, country trajectories are characterized by considerable discrepancies, as Kołczyńska and Bürkner (2020) point out. We use democracy indicators from two datasets created by well respected research groups: the Varieties of Democracy project (Coppedge et al, 2020) and the Democracy Barometer project (Engler et al, 2020), to avoid the risk of results being driven by the choice of a particular index.…”