Confirmatory factor analysis applied to the Force Concept Inventory

Abstract

In 1995, Huffman and Heller used exploratory factor analysis to draw into question the factors of the Force Concept Inventory (FCI). Since then several papers have been published examining the factors of the FCI on larger sets of student responses and understandable factors were extracted as a result. However, none of these proposed factor models have been verified to not be unique to their original sample through the use of independent sets of data. This paper seeks to confirm the factor models proposed by Scott et al. in 2012, andHestenes et al. in 1992, as well as another expert model proposed within this study through the use of confirmatory factor analysis (CFA) and a sample of 20 822 postinstruction student responses to the FCI. Upon application of CFA using the full sample, all three models were found to fit the data with acceptable global fit statistics. However, when CFA was performed using these models on smaller sample sizes the models proposed by Scott et al. and Eaton and Willoughby were found to be far more stable than the model proposed by Hestenes et al. The goodness of fit of these models to the data suggests that the FCI can be scored on factors that are not unique to a single class. These scores could then be used to comment on how instruction methods effect the performance of students along a single factor and more in-depth analyses of curriculum changes may be possible as a result.

Multidimensional item response theory and the Force Concept Inventory

“…Acceptable model fit is characterized by RMSEA < 0.05 and CFI > 0.96 or TLI > 0.96. For a summary of fit statistics, see Eaton and Willoughby [67].…”

Using psychometric tools as a window into students’ quantitative reasoning in introductory physics

“…Using the model's estimated parameters, goodness-of-fit statistics can be calculated to verify whether or not the proposed model adequately represents the correlational grouping of the question on the assessment [27]. Initially we compared the proposed question groups with the data by applying CFA and using the Confirmatory Fit Index (CFI) and Tucker-Lewis Index (TLI) as indicators of how well the data fit the model (with values above 0.90 indicating good agreement) [20]. For each term, the CFI and TLI were below 0.8, indicating that the data do not support the proposed three groups.…”

“…Multiple-response questions have been used to measure the richness of reasoning associated with the use of mathematics in Junior-level electricity and magnetism [23]. Module analysis helps identify patterns in student responses without constraining the responses to a dichotomous "all right"-or-wrong scoring scheme that is inherent in factor analysis [20,22]. Data were collected in two different terms at a comprehensive public university in the Northwestern United States.…”

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

“…Recently, substantial efforts have been made to apply quantitative techniques to further understand these instruments including factor analysis [6][7][8], cluster analysis [9], and item response theory [10][11][12][13]. In 2016, Brewe, Bruun, and Bearden [14] introduced a new class of quantitative algorithms to analyze the incorrect answers, network analytic methods [15,16].…”