Model Choices in Observational Media Effects Research: A Systematic Review and Validation

Abstract

Media effects research has produced mixed findings about the size and direction of the relationship between media consumption and public attitudes (e.g., Bartels, 1993). We investigate the extent to which model choices contribute to these inconsistent findings. Taking a comparative approach, we first review the use of different models in contemporary studies and their main findings. In order to extend and validate this review, we consider the implications for national election studies attempting to measure media effects in election campaigns and recreate these models with the British Election Study 2005-10 panel data. We compare the direction and size of effects of media content on attitude change across: between-subjects, withinelections models, in which the effects of individual-level variance in media exposure and content are assessed; within-subjects, within-elections models, which compare the effects of variance in media content for the same individual; and within-subjects, between-elections models that allow us to analyze the links between media content and exposure with attitude change over time. Our review shows some notable differences between models in terms of significance of effects (but not effect sizes). We corroborate this finding in the British campaign analysis. We conclude that to check the robustness of claims of media effects in observational data, where possible researchers should examine different model choices when evaluating media effects.Keywords : media exposure, media effects, within-subjects models, between-subjects models, between-elections models 1 Corresponding Author: mschoonvelde@gmail.com. 1For decades, researchers viewed the media as having a minimal effect on opinion and behavior (e.g., Klapper, 1960). Although more recent work has revised those conclusions the empirical record is still remarkably mixed, ranging from claims of "minimal effects" to "massive effects" (Barabas et al., 2015;Mondak, 1995;Zaller, 1996) to an apparent return to "minimal effects" (Bennett & Iyengar, 2008). With such heterogeneity in mind, Bartels described the body of research on media effects as "one of the most notable embarrassments of modern social science" (Bartels, 1993, p. 267), a state of affairs he attributed primarily to a combination of measurement error and the absence of longitudinal research designs capable of detecting media effects (see also Mondak, 1995). Zaller (2002) later raised an additional problem of research designs: the lack of attention to sample size, and in particular the power of different sample sizes to pick up even medium to large media effects on shifts in voting preferences. Research on media effects in observational studies since then has responded to these seminal critiques in paying close attention 2 This research has generally been focused on the US case but we draw on a range of country specific and cross-national studies which are increasing in number, with 2 Citation counts suggest that media effects researchers have focused more on Bartels' criticisms:...