Quantifying media literacy: development, reliability, and validity of a new measure

Abstract

Media literacy has the potential to alter outcomes in various fields, including education, communication, and public health. However, measurement of media literacy remains a critical challenge in advancing this field of inquiry. In this manuscript, we describe the development and testing of a pilot measure of media literacy. Items were formed based on a composite conceptual model and administered to college communications students (n = 34). Each of three media literacy subscales had good internal consistency reliability (α 1 = 0.74, α 2 = 0.79, α 3 = 0.75). Principal components analysis revealed a five-factor structure that corresponded closely with the underlying conceptual model. As was expected, the media literacy scale was significantly correlated with a composite critical thinking measure (r = 0.32, P = .03). This scale may be valuable for the measurement of media literacy and the assessment of media literacy interventions.

Developing a News Media Literacy Scale

“…42 Specifically, their model includes three domains (see Table 1) -authors and audiences (AA), messages and meanings (MM), and representation and reality (RR). The AA domain addresses how authors, primarily motivated by the desire for profit, target specific audiences.…”

“…39 Arke and Primack establish concurrent validity, but their limited sample leads to some doubts of the reliability of their measure. 40 The literature in media literacy suggests relationships among media literacy and other concepts, but the inconsistent operationalizations of the chief concept could lead to unreliable and possibly invalid inferences. Our goal here is to start from the ground up, following guidelines used in psychometric literature, to build a valid and reliable measure.…”

“…For example, while the Primack et al study established content validity through focus groups, theory, and expert guidance, and found criterion-related validity through its ability to predict theoretically related outcomes, it failed to establish concurrent or discriminant validity to show how this construct was similar to and different from related constructs. 38 Bier and colleagues addressed concurrent validity by showing a relationship between smoking media literacy and general media literacy, but their general media literacy scale was constructed using theory only and assessed only through internal consistency. 39 Arke and Primack establish concurrent validity, but their limited sample leads to some doubts of the reliability of their measure.…”

“…36 In another scale development attempt, Arke and Primack used another media literacy framework to establish validity and reliability. 37 While they attempted to establish construct validity by comparing the scale to the related construct of critical thinking, the analysis was based on a very small sample of 34 students.…”

Pedagogical Media Competencies of Preservice Teachers in Germany and the United States: A Comparative Analysis of Theory and Practice

“…for example Hobbs, 2011Hobbs, , 2013Buckingham, 2003;Arke & Primack, 2009;Hobbs & Frost, 2003;Potter, 2008). However, substantial shortcomings are revealed when the international literature on the modeling and measurement of pedagogical media competencies is taken into account.…”