The Quality of Public Opinion Poll Coverage during German Federal Election Campaigns

Abstract

We analyzed the quality of opinion poll coverage during the German Federal Election Campaigns 2005, 2009, 2013 and 2017 with a quantitative content analysis of the German quality newspapers Die Welt, Frankfurter Allgemeine Zeitung, Süddeutsche Zeitung and Frankfurter Rundschau. From the literature, we derived three indicators of high quality poll coverage: 1) Identification the source of the reported poll, 2) inclusion of the most important methodological information of these polls, and 3) correct interpretation of the poll results with regard to the margin of error. In line with previous research, the poll coverage lacked quality according to all three criteria. We conclude with a discussion of some simple remedies to improve the public opinion poll coverage.

Reporting the polls: the quality of media reporting of vote intention polls in the Netherlands

“…Especially those reports that covered diachronic horse races-the rise or fall of a party between two successive polls-reported or used the margin of error incorrectly. Similarly, Bhatti and Pedersen concluded that Danish journalists interpreted 'statistical noise as meaningful variation ' (2016, p. 136) and for Sweden (Oleskog Tryggvason and Strömbäck 2017, p. 15) and Germany (Vögele and Bachl 2018), similar conclusions were drawn regarding the interpretations of insignificant (diachronic) comparisons as if they were significant. Also in Denmark, newspapers are more likely to report polls about change.…”

“…The inclusion of methodological information is our first measure of quality (Vögele and Bachl 2018;Welch 2002;Andersen 2000;Hardmeier 1999). The most important methodological criteria, 9 based on the extensive list of WAPOR and AAPOR, are (1) sample size, (2) field dates, (3) sampling method, (4) data collection, (5) weighting procedure, and (6) the population.…”

“…The most important methodological criteria, 9 based on the extensive list of WAPOR and AAPOR, are (1) sample size, (2) field dates, (3) sampling method, (4) data collection, (5) weighting procedure, and (6) the population. Just as Vögele and Bachl (2018), we counted how many criteria were included and created an index score ranging from 1 till 6.…”

“…With regard to the inclusion of information about the population of the study, Chang found that almost all Singaporean newspapers articles did so 98% (1999). The inclusion of information about the data collection was least often done by German newspapers: only in 6% of the articles (Vögele and Bachl 2018). The margin of error was not often reported, however.…”