Reporting the polls: the quality of media reporting of vote intention polls in the Netherlands

Abstract

This article examines the media coverage of vote intention polls in the Netherlands. We assess whether the quality of media reporting on polls depends on the availability of information regarding the quality in pollster's reports. Our analysis of the quality looks at three different quality measures: (1) mentioning WAPOR items, such as field dates, sampling method, and polling method, (2) mentioning the margin of error, and (3) the correct interpretation of (in)significant changes and differences. The Netherlands provides an interesting case, because there is variation over time and across pollsters in the way they report their polls as well as the increased popularity of a polling aggregator. Our findings indicate that the overall quality of Dutch coverage of polls is low. When a pollster mentions the margin of error in its report, media reports on that poll are more likely to include this information and to correctly interpret the significance of differences between two parties. This effect is particularly visible when the pollster provides uncertainty intervals in its headline figures.

Who won the election? Explaining news coverage of election results in multi‐party systems

“…We also have to bear in mind that media do not necessarily report election results accurately, especially if these are still preliminary, that is, right after Election Day. Research, for example, shows that news reports are of low quality in Dutch poll coverage when reporting on differences between parties, among other things (Louwerse & van Dijk, 2021). Likewise, poll coverage in Denmark tends to focus on changes at the expense of stability (Larsen & Fazekas, 2020).…”

“…Thus, smaller parties received less media attention. Moreover, and inspired by these pioneering studies (Larsen & Fazekas, 2020; Louwerse & van Dijk, 2021), we recommend that future research considers pre‐election news coverage of polls in its ambition to explain variation in post‐election news coverage. While we focused on previous elections as a reference point for journalists to select and represent parties as winners or losers, comparisons to more recent polls may yield another reasonable yardstick for the evaluation of political parties’ performances.…”

Should we include margins of error in public opinion polls?

“…Still, this information was only shown in small letters, and journalists explicitly referred to the margins of error in only 3 per cent of the cases. In the Netherlands, Louwerse and van Dijk (2022) have found that the reporting of margins of error has increased substantially from 5.9 per cent in 2010 to 64.9 per cent in 2017. For the 2021 German federal election under study in this article, we find that 23 per cent of the poll‐related news coverage mentioned margins of error, but only 2 per cent gave an interpretation of these margins when drawing conclusions about the state of the electoral race 6…”