Introduction to the Biomass Project: An Illustration of Evidence-Centered Assessment Design and Delivery Capability

Steinberg, Lynne; Mislevy, Robert J.; Almond, Russell G.; Baird, Andrew B.; Cahallan, Cara; DiBello, Louis V.; Senturk, Deniz; Yan, Duanli; Chernick, Howard; Kindfield, Ann C. H.

doi:10.1037/e646762011-001

Cited by 8 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach to building alignment into the test development process is not an entirely new idea and is advocated for in the increasingly popular evidence-centered design approach (Mislevy & Haertel, 2006;Mislevy, Steinberg, Almond, & Lukas, 2006;Steinberg et al, 2003). However, to introduce alignment into the test development process requires familiarizing individuals involved at various stages of the process with the different test alignment criteria.…”

Section: Wyse and Vigermentioning

confidence: 95%

How Item Writers Understand Depth of Knowledge

Wyse

Viger

2011

Educational Assessment

View full text Add to dashboard Cite

An important part of test development is ensuring alignment between test forms and content standards. One common way of measuring alignment is the Webb (1997Webb ( , 2007 alignment procedure. This article investigates (a) how well item writers understand components of the definition of Depth of Knowledge (DOK) from the Webb alignment procedure and (b) how consistent their DOK ratings are with ratings provided by other committees of educators across grade levels, content areas, and alternate assessment levels in a Midwestern state alternate assessment system. Results indicate that many item writers understand key features of DOK. However, some item writers struggled to articulate what DOK means and had some misconceptions. Additional analyses suggested some lack of consistency between the item writer DOK ratings and the committee DOK ratings. Some notable differences were found across alternate assessment levels and content areas. Implications for future item writing training and alignment studies are provided.

show abstract

Section: Wyse and Vigermentioning

confidence: 95%

How Item Writers Understand Depth of Knowledge

Wyse

Viger

2011

Educational Assessment

View full text Add to dashboard Cite

show abstract

“…Although rooted in best practices for test design and development that date back 70 years (Bejar, 2008), ECD began to be codified and formally applied in testing programs beginning in the mid-1990s Mislevy, Almond, Yan, & Steinberg, 1999;Mislevy et al, 2003;Mislevy, Steinberg, Breyer, Almond, & Johnson, 2002;Steinberg et al, 2003;Williamson, Bauer, Mislevy, & Behrens, 2003). These authors offer many examples of how ECD could be used, or has been used, but in no way prescribe a single approach for all potential uses.…”

Section: Challenge 2: Ensuring Comparabilitymentioning

confidence: 98%

“…Thus, there is a critical reliance on detailed item writing and form assembly specifications to ensure to the highest degree possible form-to-form comparability from the onset. The AP Program is addressing these two challenges simultaneously through evidence-centered assessment design (ECD) (Mislevy & Riconscente, 2005;Mislevy, Steinberg, Almond, & Lukas, 2006;Steinberg et al, 2003). ECD provides a mechanism to (a) capitalize on advances in the understanding of student learning and assessment, (b) specify item writing and form assembly guidelines, and (c) structure a comprehensive validity argument to support meaningful and comparable inferences from exam scores.…”

Section: Challenge 2: Ensuring Comparabilitymentioning

confidence: 99%

The Promises and Challenges of Implementing Evidence-Centered Design in Large-Scale Assessment

Huff

Steinberg²,

Matts

2010

Applied Measurement in Education

View full text Add to dashboard Cite

The cornerstone of evidence-centered assessment design (ECD) is an evidentiary argument that requires that each target of measurement (e.g., learning goal) for an assessment be expressed as a claim to be made about an examinee that is relevant to the specific purpose and audience(s) for the assessment. The observable evidence required to warrant each claim is also articulated. In turn, the claims and evidence shape the design of assessment opportunities for students to demonstrate what they have learned, whether that opportunity is a classroom activity or a multiple-choice item on a high-stakes assessment. Once identified, the characteristics of these assessment opportunities are referred to as task models, each capable of generating multiple assessment tasks. Taken together, the claims, evidence, and task models constitute the evidentiary argument. The benefits and challenges of implementing ECD in the Advanced Placement Program are addressed. WHY ECD FOR AP?Scores from the Advanced Placement (AP ® ) Exams are valuable tools for credit and placement decisions at postsecondary institutions, and the courses are intended to provide high-school students with an opportunity to engage in

show abstract

“…Based on evidence-centered assessment design, Almond et al (2002) provide a four-phase generic assessment model that is purposed for these types of complex learning environments, including activity selection, presentation, response processing, and summary scoring. Furthermore, in a description of the Biomass project, Steinberg et al (2003) provide an excellent example of innovative, web-based assessment of science inquiry that is highly aligned with several sets of nationally-recognized science standards. This line of research in educational measurement shows promise for the assessment of complex performance data in situated inquiry environments (such as MUVEs) that will allow for progression beyond the static pretestposttest model.…”

Section: Cognitive Loadmentioning

confidence: 99%

Collaboration modality, cognitive load, and science inquiry learning in virtual inquiry environments

Erlandson

Nelson

Savenye

2010

Education Tech Research Dev

View full text Add to dashboard Cite

Educational multi-user virtual environments (MUVEs) have been shown to be effective platforms for situated science inquiry curricula. While researchers find MUVEs to be supportive of collaborative scientific inquiry processes, the complex mix of multi-modal messages present in MUVEs can lead to cognitive overload, with learners unable to effectively process the rich information encountered in virtual space. In this study, we investigated the effect of communication modality on cognitive load and science inquiry learning in students completing a science inquiry curriculum in an educational MUVE. Seventy-eight undergraduate education majors from a large southwestern university participated in this control-treatment study. Significant positive results were found for reducing cognitive load for participants communicating through voice-based chat, although this reduction was not found to influence learning outcomes. We conclude that use of voice-based communication can successfully reduce cognitive load in MUVE-based inquiry curricula.Keywords Virtual environments Á Cognitive load Á Collaboration Á Modality Á VoIP One alternative to classroom-based inquiry that has arisen in recent years is curricula embedded within computer-based educational multi-user virtual environments (MUVEs). MUVEs have emerged as a fertile platform for situated inquiry learning environments, especially concerning science inquiry learning processes (Barab et al. 2005;Nelson et al. 2005;Nelson 2007). Briefly, educational MUVEs are three-dimensional (3-D) virtual worlds that afford learners interaction with other humans and software-based agents, both represented through characters called avatars, as part of an open-ended exploratory environment. In these environments, learners can interact with various situated objects (including images, sounds, and other multimedia content) while communicating and collaborating with other learners to investigate scenarios and solve problems of varying complexity (Nelson and Ketelhut 2007).de Jong (2006) notes that one of the biggest challenges for supporting inquiry learning with technology is the seamless integration of collaborative learning and inquiry learning. Another challenge, cited by Kirschner et al. (2006), is that guided inquiry in technologybased environments is more powerful than wholly open-ended exploration. Both of these challenges for science inquiry learning-appropriately guided inquiry learning and integration of collaboration and inquiry learning-are being addressed by researchers implementing MUVEs (e.g., Bers 1999;Corbit 2002;Slator et al. 2004;Dede et al. 2002). In an even broader sense, research in this area is part of a rapidly expanding focus on the use of educational gaming environments for learning (Gee 2003). MUVEs as situated inquiry environmentsSadler et al. (2007) have outlined the need for emphasis upon socioscientific inquiry, or an approach to scientific inquiry learning that includes social and emotional constructs in addition to the typical cognitive skills involved with the...

show abstract

Introduction to the Biomass Project: An Illustration of Evidence-Centered Assessment Design and Delivery Capability

Cited by 8 publications

References 0 publications

How Item Writers Understand Depth of Knowledge

How Item Writers Understand Depth of Knowledge

The Promises and Challenges of Implementing Evidence-Centered Design in Large-Scale Assessment

Collaboration modality, cognitive load, and science inquiry learning in virtual inquiry environments

Contact Info

Product

Resources

About