Developing a construct‐based assessment to examine students' analogical reasoning around physical models in Earth Science

Abstract

In recent years, science education has placed increasing importance on learners' mastery of scientific reasoning. This growing emphasis presents a challenge for both developers and users of assessments. We report on our effort around the conceptualization, development, and testing the validity of an assessment of students' ability to reason around physical dynamic models in Earth Science. Building from the research literature on analogical mapping and informed by the current perspectives on learning progressions, we present a three-tiered construct describing the increasing sophistication of students' analogical reasoning around the correspondences and non-correspondences between models and the Earth System: at the level of entities (Level 1), configurations in space or relative motion of entities (Level 2), and the mechanism or cause for observed phenomena (Level 3). Grounded in a construct-centered design approach, we describe our process for developing assessments in order to examine and validate this construct, including how we selected topics and models, designed items, and developed outcome spaces. We present the specific example of one assessment centered on moon phases, which was administered to 164 8th and 9th grade Earth Science students as a pre/postmeasure. Two hundred ninety-four responses were analyzed using a Rasch modeling approach. Item difficulties and student proficiency scores were calculated and analyzed regarding their relative performance with respect to the three levels of the construct. The analysis results provided initial evidence in support of the construct as conceived, with students displaying a range of analogical reasoning spanning all three construct levels. It also identified problematic items that merit further examination. Overall, the assessment has provided us the opportunity to better describe and frame the cognitive uses of models by students during learning situations in Earth Science. Implications for instruction and future directions for research in this area are discussed. ß 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 2012

A macroscopic device described by a Boltzmann-like distribution

“…The MecAgit system is simple and inexpensive enough for implementation in classroom teaching, where physical models have been shown to engage students and increase their level of understanding. [32][33] …”

Science Learning Progressions

“…Likewise, assessments play a critical role in informing the development, validation, and use of individual LPs. Such assessments can detect the range of students' abilities, or levels of sophistication, as well as the appropriateness of particular assessment items for evaluating different student groups ( 13,14).…”

Measuring Knowledge Integration Learning of Energy Topics: A two-year longitudinal study

“…Because of their mathematical simplicity and measurement strengths, Rasch-type models have been widely used in large-scale assessments such as the Programme of International Student Assessment (OECD, 2012). Increasingly, researchers draw on Rasch-type models when analyzing classroom-level assessments for formative purposes (Johnson & Tymms, 2011;Liu et al, 2008;Liu, Lee, & Linn, 2011a, 2011bRivet & Kastens, 2012;Siegel, 2007).…”