Using concept inventories to measure understanding

Abstract

Measuring understanding is notoriously difficult. Indeed, in formulating learning outcomes the word 'understanding' is usually avoided, but in the sciences, developing understanding is one of the main aims of instruction. Scientific knowledge is factual, having been tested against empirical observation and experimentation, but knowledge of facts alone is not enough. There are also models and theories containing complex ideas and interrelationships that must be understood, and considerable attention has been devoted across a range of scientific disciplines to measuring understanding. This case study will focus on one of the main tools employed: the concept inventory and in particular the Force Concept Inventory (FCI). The success of concept inventories in physics has spawned concept inventories in chemistry, biology, astronomy, materials science and maths, to name a few. We focus here on the FCI and ask how useful concept inventories are for evaluating learning gains. Finally, we report on recent work by the authors to extend conceptual testing beyond the multiple-choice format.

Forty Years’ Experience in Teaching Fluid Mechanics at Strasbourg University

“…Among the many tests developed for engineering and physics education in the United States [48][49][50][51][52][53][54][55], one is devoted to fluid mechanics [5,6]. Initiated by a cooperative effort between Jay Martin and John Mitchell at the University of Wisconsin-Madison and Ty Newell at the University of Illinois, Champaign-Urbana in 2001, it led to a final version in 2006 with the aim of promoting student understanding of fluid mechanics, as taught in mechanical engineering in the United States.…”

Multiple-choice Learning Assessments for Intermediate Mechanical Engineering Courses: Insights from Think-aloud Interviews

“…The insights we drew from observing students working on the assessments while vocalizing their thoughts were critical to the development process. The interviews helped us (1) identify problematic test items, (2) identify student misconceptions and use them to create distractors couched in language that sounds natural to students, and (3) build evidence for the validity of the assessment.…”

“…We draw a distinction between an MCLA and a concept inventory (CI). A CI is based on a clearly-articulated set of concepts (often narrowly-defined) or expert modes of thinking in a field [1]. Some notable examples include the Force Concept Inventory [2], the Statics Concept Inventory [3], and the Brief Electricity and Magnetism Assessment [4].…”

“…Correlation (Pearson's r) between the MCLA score and course outcomes, and standardized coefficients for the linear model described by Eqn (1)…”

MCS1: A MATLAB Programming Concept Inventory for Assessing First-year Engineering Courses

“…As the use of concept inventories grew in popularity due to the success of FCI [3,4], more assessments were developed in areas such as physics, chemistry, astronomy, geoscience, and others [5]. Concept inventories are often given before and after instruction (referred to as the 'pre-test' and 'post-test' in the literature) [4]. This demonstrates the use of concept inventories as effective assessment tools.…”