A moving target? An analysis of the impact of electoral context on polling error variation in both British and international general elections

Abstract

In this article, we argue that electoral context affects the projection mechanisms inherent in polling. This insight applies both to the estimation of party vote shares by pollsters and to poll-driven substantive political expectations. To test this argument, we analyse 794 in-campaign polls covering the UK's 21 post-war general elections, as well as an updated version of Jennings and Wlezien's (2018) international polling dataset. We demonstrate that the election level houses a substantial portion of the observed variance in polling error. This finding is valid across several modelling approaches and a range of measures of polling accuracy both within and beyond the UK. Within the UK, we show that the election level is a particularly important locus of variance when it comes to analysing whether polls give rise to misleading substantive expectations about election results.

Electoral predictors of polling errors

“…In this study, we focus on voter suppression, where parties continually use their discretion to determine voting requirements that disproportionately obstruct specific social groups (Helmke, Kroeger and Paine 2022). Tudor and Wall (2021) A weak, positive association between election-day bias and the State Democracy Index can be found in pane A of figure 6, meaning the lower the level of democracy in a state, the more the Republican candidate is underestimated. This is in line with the "biased" vote hypothesis.…”

“…Yet poll features alone are not enough to account for poll accuracy (DeSart and Holbrook 2003). As Tudor and Wall (2021) demonstrate in their analysis of more than 20,000 polls across 400 national elections worldwide, the bulk of variance in poll accuracy could be observed between (and not within) elections. The crucial question of why the polls failed in a given election but not in others has often remained a matter of speculation.…”

“…We go beyond extant large-scale comparative work (e.g. Jennings and Wlezien 2018;Sohlberg and Branham 2020;Tudor and Wall 2021), and cover a broad range of electoral features that pundits and scholars have suspected of encouraging polling errors, including mobilization, candidacies, polarization, and electoral conduct.…”

Electoral predictors of polling errors

“…Yet poll features alone are not enough to account for poll accuracy (DeSart and Holbrook 2003). As Tudor and Wall (2021) demonstrate in their analysis of more than 20,000 polls across 400 national elections worldwide, the bulk of variance in poll accuracy could be observed between (and not within) elections. The crucial question of why the polls failed in a given election but not in others has often remained a matter of speculation.…”

“…In the selection of potential predictors, we go beyond extant large-scale comparative work (e.g. Jennings and Wlezien 2018;Sohlberg and Branham 2020;Tudor and Wall 2021), and cover a broad range of electoral features that pundits and scholars have suspected of encouraging polling errors, including mobilization, candidacies, polarization, and electoral conduct.…”