Some Links Between Large‐Scale and Classroom Assessments: The Case of the BEAR Assessment System

Large-scale assessment systems: Design principles drawn from international comparisons1

“…A model of the sequence in which learners progress in conceptual understanding of any one topic, which can be composed on the basis of cognitive theory, and research on students' learning of the topic, can be used both to match tasks to particular student groups, and to set up criteria against which to assess progress in learning. In science, this approach is illustrated by the detailed analysis reported by Smith, Wiser, Anderson, and Krajcik (2006), whilst the value of this approach for both formative and summative purposes is illustrated in the work of the BEAR project (Wilson & Draney, 2004). In Australia, "learning maps" of typical progress in spelling, reading, listening, and spelling have been developed (Forster & Masters, 2004) while in the UK the Graded Assessment in Mathematics (GAIM) project developed hierarchies of learning in mathematics (Brown, 1992).…”

Alternative perspectives on learning outcomes: challenges for assessment

“…To focus on formative assessment, and assuming absence of high-stakes summative pressures, it can be seen from our case-studies that the models of progression needed for formative purposes are very diverse. From studies of this issue in school science and mathematics (Denvir and Brown 1986;Wilson and Draney 2004) detailed models have been developed, based on research studies in the conceptual understanding of mathematics and of science topics, with guidance to teachers on how to use such models in a formative way.…”

Cognitive Diagnosis Using Item Response Models

“…The system consists of four principles, each associated with a practical building block (Wilson, 2005), and in addition, a capstone integrative activity that can take on different aspects under different circumstances (e.g., assessment moderation or standard setting). Its original deployment was as a curriculum-embedded system in science (Wilson & Sloane, 2006), but it has clear and logical extensions to other contexts such as in higher education (Wil-son & Scalise, 2006), in large-scale assessment (Wilson & Draney, 2004), and across other disciplinary areas, such as mathematics (Wilson & Carstensen, 2007).…”

“…The statistical basis for the measurement models used here is a class of models called explanatory item response models (EIRMs; De Boeck & Wilson, 2004), which lays out a large class of models that are suitable for estimating parameters of the psychological space. These are based on generalized linear mixed models and nonlinear mixed models (GLMMs and NLMMs respectively; Verbeke & Molenberghs, 2000), and are estimable with generalized software such as SAS NLMIXED (SAS Institute, 1999) and GLLAMM (Rabe-Hesketh, Skrondal, & Pickles, 2004).…”

“…Several of these are discussed below, along with a sketch of the models one might use to address them. The models are drawn from those described in De Boeck and Wilson (2004), where a scheme for conceptually organizing the models, as well as a generalized statistical approach, is also given. I will not attempt to review that here, as it is beyond the scope of this paper.…”