Validity and reliability of the force and motion conceptual evaluation

Ramlo, Susan

doi:10.1119/1.2952440

Cited by 48 publications

(46 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Retaining these items would increase coverage of some kinematic principles while providing coverage of F7 and C5. This would leave a reduced 21-item FCI instrument containing items 1,2,3,4,5,7,8,9,11,13,14,15,17,18,19,20,25,26,27,28, and 30. Blocked items 10 and 16 were removed because there was sufficient coverage of the principles required for their solution.…”

Section: Discussionmentioning

confidence: 99%

“…Ramlo [27] calculated the factor structure of the FMCE [3] finding 3 factors for the pretest; however, these factors contained a mixture of concepts and Ramlo concluded the pretest factor structure was undefined. Three factors were also found for the post-test with items covering similar conceptual topics largely loading onto the same factor.…”

Section: A Factor Analysis and The Fcimentioning

confidence: 99%

See 1 more Smart Citation

Multidimensional item response theory and the Force Concept Inventory

Stewart

Zabriskie

DeVore

2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Research on the test structure of the Force Concept Inventory (FCI) has largely been performed with exploratory methods such as factor analysis and cluster analysis. Multidimensional Item Response Theory (MIRT) provides an alternative to traditional exploratory factor analysis which allows statistical testing to identify the optimal number of factors. Application of MIRT to a sample of N ¼ 4716 FCI post-tests identified a 9-factor solution as optimal. Additional analysis showed that a substantial part of the identified factor structure resulted from the practice of using problem blocks and from pairs of similar questions. Applying MIRT to a reduced set of FCI items removing blocked items and repeated items produced a 6-factor solution; however, the factors still had little relation the general structure of Newtonian mechanics. A theoretical model of the FCI was constructed from expert solutions and fit to the FCI by constraining the MIRT parameter matrix to the theoretical model. Variations on the theoretical model were then explored to identify an optimal model. The optimal model supported the differentiation of Newton's 1st and 2nd law; of one-dimensional and three-dimensional kinematics; and of the principle of the addition of forces from Newton's 2nd law. The model suggested by the authors of the FCI was also fit; the optimal MIRT model was statistically superior.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Factor Analysis and The Fcimentioning

confidence: 99%

Multidimensional item response theory and the Force Concept Inventory

Stewart

Zabriskie

DeVore

2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

show abstract

“…Only one study examined the factor structure of the FMCE. Ramlo found the factor structure of the FMCE pretest was undefined but that a three-factor solution existed for the FMCE post-test [30]. To our knowledge, little research has been performed investigating the reliability, validity, or fairness of either the FMCE or the CSEM.…”

Section: A Reliability and Validitymentioning

confidence: 99%

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Henderson

Miller

Stewart

et al. 2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Gender gaps on the most widely used conceptual inventories created by physics education researchers have been extensively studied. Most of the research exploring the consistent gender gaps has been performed at the student level using the total evaluation score; less research has been performed examining these assessments at the item level and this research has been predominately restricted to the Force Concept Inventory (FCI). Many studies have identified subsets of FCI items as unfair to either men or women. An item is fair if men and women of equal ability in conceptual physics score equally on the item. This study explored the item-level gender fairness of the Force and Motion Conceptual Evaluation (FMCE) and the Conceptual Survey of Electricity and Magnetism (CSEM). Classical test theory and differential item functioning (DIF) analysis were employed to examine item fairness. Fairness was investigated with four large post-test samples, two for the FMCE (n 1 ¼ 3016, n 2 ¼ 3719) and two for the CSEM (n 1 ¼ 2014, n 2 ¼ 2657). Men and women performed significantly differently on the majority of FMCE items but with no more than a small effect size. There were fewer items in the CSEM where men and women performed differently. Using DIF analysis, which assumes that overall test score is an accurate measure of ability, only one item in the FMCE demonstrated large DIF in both samples with that item unfair to women. One additional item showed large DIF in a single sample, also unfair to women. Only one item in the CSEM demonstrated large DIF. The item was unfair to men but this result was not consistent across all samples. The number of large DIF items identified in both the FMCE and the CSEM was substantially smaller than the number of large DIF items identified in the FCI by previous studies.

show abstract

“…There are typically a few weeks remaining in the term after the momentum unit, allowing for postinstruction data collection in addition to the two opportunities during instruction. Past research on students' understanding of N3L has focused on the differences between different instructional methods [10][11][12][13] or N3L as a convenient topic for investigating deeper issues in student understanding [8,[14][15][16]. This paper follows the latter tradition.…”

Section: Introductionmentioning

confidence: 99%

Learning, retention, and forgetting of Newton’s third law throughout university physics

Sayre

Dymek

Clark

et al. 2012

Phys. Rev. ST Phys. Educ. Res.

View full text Add to dashboard Cite

We present data from a between-student study on student response to questions on Newton's third law given in two introductory calculus-based physics classes (Mechanics and Electromagnetism) at a large northeastern university. Construction of a response curve reveals subtle dynamics in student learning not capturable by pretesting and post-testing. We find a significant positive effect of instruction that diminishes by the end of the quarter. Two quarters later, a significant dip in correct response occurs when instruction changes from the vector quantities of electric forces and fields to the scalar quantity of electric potential. When instruction returns to vector topics, performance rebounds to initial values.

show abstract

Validity and reliability of the force and motion conceptual evaluation

Cited by 48 publications

References 22 publications

Multidimensional item response theory and the Force Concept Inventory

Multidimensional item response theory and the Force Concept Inventory

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Learning, retention, and forgetting of Newton’s third law throughout university physics

Contact Info

Product

Resources

About