Multidimensional Item Response Theory and the Conceptual Survey of Electricity and Magnetism

Abstract

While many studies have examined the structure, validity, and reliability of the Force Concept Inventory, far less research has been performed on other conceptual instruments in widespread use in physics education research. This study performs a confirmatory analysis of the Conceptual Survey of Electricity and Magnetism (CSEM) guided by a theoretical model of expert understanding of electricity and magnetism. Multidimensional Item Response Theory (MIRT) with the discrimination matrix constrained to the theoretical model was used to investigate two large datasets (N 1 ¼ 2014 and N 2 ¼ 2657) from two research universities in the United States. The optimal model identified by MIRT was similar, but not identical, for the two datasets and had very good model fit with comparative fit indices of 0.975 and 0.984, respectively. The most parsimonious optimal model required 23 independent principles of electricity and magnetism and was significantly better fitting than a more general model dividing the CSEM into 6 general topics. The optimal models for the two samples were quite similar, sharing 22 of a possible 26 conceptual principles. Most of the overall item difficulties and discriminations were significantly different between the two samples; however, the rank order of the overall difficulty and discrimination were generally similar. There was much more similarity between the discrimination by item of the individual principles. Five items had a difficulty ranking that was substantially different between the two samples, indicating that while generally similar, relative difficulty does depend on the student population and instructional environment.

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

“…"Cart on a Ramp" items (items 8-10) ask students to select the force on a cart as it moves up and down an incline. "Coin Toss -Force" items (items [11][12][13] ask students to select the force on a coin tossed in the air. "Force Graph" items (items [14][15][16][17][18][19][20][21] ask students about the force on a toy car as it moves across a low-friction surface; students select from a number of graphs.…”

“…Recently, substantial efforts have been made to apply quantitative techniques to further understand these instruments including factor analysis [6][7][8], cluster analysis [9], and item response theory [10][11][12][13]. In 2016, Brewe, Bruun, and Bearden [14] introduced a new class of quantitative algorithms to analyze the incorrect answers, network analytic methods [15,16].…”

Multidimensional item response theory and the force and motion conceptual evaluation

“…Study 2 applied the methods introduced in study 1 to the CSEM using datasets drawn from 2 different institutions [7]. The optimal models identified were similar but not identical suggesting that while some differences exist, the optimal models may be representative of student thinking at a number of institutions.…”

Linking and comparing short and full-length concept inventories of electricity and magnetism using item response theory

“…Several recent studies also applied multidimensional IRT to evaluate the construct dimensions of a CI through exploratory and or confirmatory approaches [28][29][30]. Scott and Schumayer [30] used the multidimensional IRT model to explore the factor construct of a FCI dataset [31], which comfirmed an optimal five-factor structure for FCI.…”