Chapter 5: Social, Methodological, and Theoretical Issues Regarding Assessment: Lessons From a Secondary Analysis of PISA 2000 Literacy Tests

Assessment systems as cultural scripts: a sociocultural theoretical lens on assessment practice and products

“…Through the psychometric legacy, normative statistics of educational achievements have become central to education policy regimes (Ozga, 2009) and the strength of correlations between achievement and student background measures, a surrogate for educational quality and equity at the system level (Lingard, Sellar, & Savage, 2014). This 'policy-by-numbers' phenomenon encourages technical solutions to quality in education systems and allows an evolution from evaluation of 'products' (student achievements) to evaluation of the 'producers' (school systems) (Rochex, 2006).…”

Comprehension and content: planning literacy in low socioeconomic and culturally diverse schools

“…The implementation of NCLB in the USA-like its UK and, now quite similarly, Australian counterparts-has raised technical issues of sociodemographic classification and the definition of target populations (Lucas and Beresford 2010), construct definition (Rochex 2006), measurement validity and reliability (Moss et al 2006;Stobart 2008). Nonetheless, post-NCLB data points to two problem scenarios.…”

Cultural, social, and economic capital constructs in international assessments: an evaluation using exploratory structural equation modeling

“…Such practices oversimplify the complex nature of educational systems in which educational outcomes result from interactions and interrelated decisions of different educational actors (i.e., school staff, children, parents, etc.) (Murnane & Willett, 2011;Rochex, 2006). From this perspective, we can claim to have gained great knowledge about the factors related to school effectiveness, and yet, we know little about the mechanisms at work.…”

“…Both promoters (e.g., Teddlie & Reynolds, 2001) and detractors (e.g., Rochex, 2006;Slee, Weiner, & Tomlinson, 1998) of the use of international surveys concur that there is a lack of theory in the research program. These criticisms can be tracked down to the building blocks at the basis of the architecture of the theories subscribed by LSA research.…”

“…Over time, these studies have had a significant impact on educational policy in aspects such as curriculum reforms, educational standards, and educational budget decisions, among other educational initiatives (Grek, 2009;Rutkowski & Rutkowski, 2010;Schwippert, 2007;Schwippert & Lenkeit, 2012). Their influential role in shaping educational policy is reflected by the continued emergence of international assessment studies in different regions, thematic areas, and for different target populations (Rochex, 2006). In addition to the well-established Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), and Programme for International Student Assessment (PISA), some newer regional or language-focused initiatives are now being carried out in Southern and Eastern Africa (The Southern and Eastern Africa Consortium for Monitoring for Assessment of the Quality of Education [LLECE]), and French-speaking countries (Programme on the Analysis of Education Systems [PASEC]).…”