Do Online Voter Guides Empower Citizens?

Abstract

Voting Advice Applications (VAAs), which provide citizens with information on the party that best represents their political preferences, are often cited as evidence of the empowering capabilities of digital tools. Aside from the informational benefits of these voter guides, observational studies have suggested a strong effect on political participation and vote choice. However, existing impact evaluations have been limited by a reliance on convenience samples, lack of random assignment, or both. This raises questions about self-selection and the precise mechanisms underlying how voters learn about politics. Here, we provide evidence from a field experiment with survey outcomes conducted with a sample of over 1,000 German citizens in the 2017 federal election campaign. Using linked panel survey and digital trace data combined with a randomized encouragement to complete a VAA, we assess respondents’ compliance with treatment and observe how the use of this tool affects political behavior, attitudes, media consumption, political knowledge, and even social media activity. Our findings reveal that the overwhelming consensus in favor of positive effects on turnout and vote choice should be treated with caution, as we find no such effects. Rather, the actual virtue of VAAs in a complex online information environment lies in increasing knowledge about parties’ positions on issues—exactly the kind of information these tools were designed to provide.

The consequences of online partisan media

“…In this article, we demonstrate a methodological framework for studying the effect of social media and the internet on individuals. By embedding a naturalistic encouragement in an online panel survey with linked digital trace data, our design achieves ecological validity without sacrificing internal validity (45). This approach does not come without costs: our intervention was necessarily a bundle, although multiple survey waves allowed us to prespecify and test for different pathways of influence.…”

Artificial intelligence and voting advice applications

“…The results of these studies were rather mixed, showing a positive effect of VAA usage on electoral turnout in some countries such as Switzerland, the Netherlands, and Finland, but not in others like Greece, while in Germany VAA usage was associated with information-seeking activities but not more active engagement with electoral campaigns such as attending party rallies. Another strand, looked at randomized experiments in countries and regions like Estonia (Vassil, 2011), Finland (Christensen et al, 2021), Germany (Munzert et al, 2020), Hungary (Enyedi, 2016), Italy (Garzia et al, 2017b), Japan (Tsutsumi et al, 2018), Northern Ireland (Garry et al, 2019), Quebec (Mahéo, 2016(Mahéo, , 2017, Switzerland (Pianzola et al, 2019;Stadelmann-Steffen et al, 2023), Taiwan (Liao et al, 2020), Turkey (Andı et al, 2023), Greece, Bulgaria, Romania, Spain, and Great Britain (Germann et al, 2023). Again, while most studies find evidence of VAA effects on voting behavior and political participation (Garry et al, 2019;Pianzola et al, 2019;Liao et al, 2020;Christensen et al, 2021;Germann et al, 2023;Stadelmann-Steffen et al, 2023), other studies find no meaningful effects (Enyedi, 2016;Tsutsumi et al, 2018;Munzert et al, 2020), or effects that were limited to undecided (or centrist) voters (Vassil, 2011;Mahéo, 2016;Andı et al, 2023).…”

Publishing Combined Web Tracking and Survey Data

“…Finally, accompanying the rise of online experiments in the social sciences, researchers have leveraged web tracking data to, e.g., assess compliance with treatment assignments. Some experimental studies include evaluations of the effects of voting advice applications on turnout, vote choice, and knowledge about political platforms (Munzert et al, 2020), the effects of media diets on political behavior and attitudes (Guess et al, 2021), and the effects of monetary incentives in the uptake of COVID-19 contact tracing apps (Munzert et al, 2021).…”

“…Such data can be used to study a wide range of topics. For instance, using web tracking data, researchers have observed which news outlets individuals visit (e.g., Bach et al, 2022;Guess et al, 2021;Reiss, 2022;, assessed compliance in online experiments (Munzert et al, 2020), or evaluated the validity of metrics derived from surveys (Jürgens et al, 2020). The collection of such data happens via online tracking toolssoftware that survey participants install on their devices that logs their internet usage behavior.…”

“…Examples of replication repositories for studies using the data we draw on in this report include an archive forMunzert et al (2020) at https://doi.org/10.7910/DVN/0LCHKX as well as an archive forGuess et al (2021) at https://github.com/NetDem-USC/homepage-experiment.10 However, a term that would probably be more fitting here is "reproduction data" as they enable the reproduction of the results reported in the associated publications (same data, same analysis methods).9 The replication repository for can be found at: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/P3XVFO&version=1.0 8 The replication repository forHaim et al (2021) can be found at: https://osf.io/6xrej/…”