Extending modified module analysis to include correct responses: Analysis of the Force Concept Inventory

Abstract

Brewe, Bruun, and Bearden first applied network analysis to understand patterns of incorrect conceptual physics reasoning in multiple-choice instruments introducing the Module Analysis for Multiple-Choice Responses (MAMCR) algorithm. Wells et al. proposed an extension to the algorithm which allowed the analysis of large datasets called Modified Module Analysis (MMA). This method analyzed the network structure of the correlation matrix of the responses to a multiple-choice instrument. Both MAMCR and MMA could only be applied to networks of incorrect responses. In this study, an extension of MMA is explored which allows the analysis of networks involving both correct and incorrect responses. The extension analyzes the network structure of the partial correlation matrix instead of the correlation matrix. The new algorithm, called MMA-P, was applied to the FCI and recovered much of the structure identified by MMA. The algorithm also identified sets of correct answers requiring similar physical reasoning reported in previous studies. Beyond groups of all correct and all incorrect responses, some groups of responses which mixed correct and incorrect responses were also identified. Some of these mixed response groups were produced when a correct response was selected for incorrect reasons; some of the groups helped to explain the gender unfairness previously reported for the FCI.

Assessment of kinematic concepts comprehension: A systematic review

“…3(a), showed that many researchers considered the development of assessment to reveal students' understanding. Especially revealing students' understanding before implementing the learning process [1], [38]- [40].…”

The Curious Construct of Active Learning

“…Finally, an important consideration is how data from CIs and other measures are analyzed. Focusing only on correctness or measures of gain may limit researchers’ and instructors’ understanding of learning (e.g., the type and nature of incorrect answers students give reveals little about the various types of thinking that could have led to the answers; Brewe et al, 2016; Wallace & Bailey, 2010; Wells et al, 2019; Yang et al, 2020).…”

Examining consistency of student errors in vector operations using module analysis

“…An example is the "impetus" module found by Brewe et al, which is dominated by answers students could arrive at following a line of reasoning where a contact force (either constant or diminishing) continues to act on an object after contact has stopped. Recent work has expanded MAMCR to work with larger FCI data sets [19], applied this modified MAMCR to the FMCE [17], and extended it to include students' correct answers on the FCI [18].…”