A Systematic Review of Assessment Literacy Measures

Abstract

This work systematically reviews teacher assessment literacy measures within the context of contemporary teacher evaluation policy. In this study, the researchers collected objective tests of assessment knowledge, teacher self‐reports, and rubrics to evaluate teachers’ work in assessment literacy studies from 1991 to 2012. Then they evaluated the psychometric work from these measures against a set of claims related to score interpretation and use. Across the 36 measures reviewed, they found support for these claims was weak. This outcome highlights the need for increased work on assessment literacy measures in the educational measurement field. The authors conclude with recommendations and a resource to inform a research agenda focused on assessment literacy measurement to inform policy and practice.

Assessing Children’s Social Functioning and Reading Proficiency: A Systematic Review of the Quality of Educational Assessment Instruments Used in Norwegian Elementary Schools

“…Thus, our findings support the conclusions of other systematic reviews of assessment measures, namely, that the psychometric evidence is weak or lacking (e.g., Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014). That said, we did not analyse the measurements' original data but rather reviewed the information and the documented materials of the instruments provided to us by the authors and publishers and obtained through the literature search.…”

“…These scales were associated with a mix of adequate, inadequate or not-reported reliability, as well as inadequate overall validity. Furthermore, Gotch and French (2014) found weak psychometric evidence for the 36 educational literacy measures they reviewed. Siddiq et al (2016) reviewed 38 educational assessment instruments that aim to measure students' literacy in information and communication technology; they found that the documentation and reporting of test quality were lacking overall.…”

“…Whereas measures of social functioning are commonly based on informal teachers' ratings and students' self-reports, reading proficiency is measured based on summative formal tests or formative informal assessments and teacher ratings. Findings from systematic reviews show large variations in the methods that schools use to assess children's development in social skills and reading (Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;OECD, 2015 , 2015). Altogether, these reviews show that the educational assessment instruments used in schools vary with respect to a number of dimensions: (1) the level of informal or formal structure (e.g., open notes of teacher ratings versus criterion-based tests); (2) the level of interactivity (e.g., static versus dynamic); (3) whether the assessment is summative or formative (e.g., assessment of learning versus assessment for learning); (4) the assessment structure (e.g., presentation of the items to the test-taker); (5) the response formats (e.g., selected or constructed items); and (6) the item scoring (e.g., hand scoring versus computer-based scoring).…”

“…Reliability is crucial if an assessment is to be useful, as a test that is not reliable can never be a valid instrument (Thorndike & Thorndike-Christ, 2014). Reliability is commonly assessed in terms of internal consistency reliability (the degree to which different test items that probe the same construct produce similar results), test-retest reliability (stability over time), and inter-rater reliability (the degree to which different observers or raters agree).Recent systematic reviews evaluating the quality of educational assessment instruments have revealed a lack of studies of psychometric properties for many of the instruments used in schools (see, for instance, Cordier et al, 2015;Floyd et al, 2015;Gotch & French, 2014;Siddiq, Hatlevik, Olsen, Throndsen, & Scherer, 2016; Statens beredning för medicinsk utvärdering [Swedish Council on Health Technology Assessment], 2014). Although several of the reviewed measures demonstrated good psychometric qualities, many others showed weak or lacking evidence.…”

Exploring How to Model Formative Assessment Trajectories of Posing‐Pausing‐Probing Practices: Toward a Teacher Learning Progressions Framework for the Study of Novice Teachers

“…Experts have typically turned to the concept of assessment literacy to explain differences in teachers' understanding and use of assessment in the classroom (Gotch & French, 2014;Marzano, Pickering, & Pollock, 2001;Plake, Impara, & Fager, 1993;Popham, 2004Popham, , 2008Stiggins, 2001Stiggins, , 2002. Stiggins (1995) states that "Assessment literates know the difference between sound and unsound assessment.…”