Measuring Backsliding with Observables: Observable-to-Subjective Score Mapping (OSM)

Abstract

Multiple well-known democracy rating projects—including Freedom House, Polity,and V-Dem—have identified apparent global regression in recent years. These measures rely on partly subjective indicators, which could, in principle, suffer from rater bias. For instance, Little and Meng (2023) argue that shared beliefs driven by the current zeitgeist could lead to shared biases that produce the appearance of democratic backsliding in subjectively coded measures. To assess this argument, and the strength of the evidence for global democratic backsliding, we propose an observable-to-subjective score mapping (OSM) methodology that uses only easily observable features of democracy to predict existing indices of democracy. Applying this methodology to three prominent democracy indices, we find evidence of backsliding, but beginning later and not as pronounced as suggested by some of the original indices. Our approach suggests that particularly the Freedom House measure is out of track with the recent patterns in observable indicators and that there has been a stasis or, at most, a modest decline in the average level of democracy.

An Events-Based Approach to Understanding Democratic Erosion

“…To complement the L&M index, we also compare trends in V-Dem's electoral democracy index to the "objective-to-subjective score mapping" (OSM) index proposed by Weitzel et al (2023) in this symposium. The OSM index aims to overcome some of the potential empirical and conceptual problems with L&M identified by other contributors to the symposium.…”

“…OSM is derived from a random-forest model trained to predict scores on existing "subjective" democracy indices (including V-Dem's) using 26 "easily observable features of democracy" (Weitzel et al 2023, 3). Online appendices C and D compare V-Dem's electoral democracy index to trends in the disaggregated indicators that L&M and Weitzel et al (2023) propose.…”