Concept Inventories: Design, Application, Uses, Limitations, and Next Steps

Klymkowsky, Michael W.; Garvin‐Doxas, Kathy

doi:10.1007/978-3-030-33600-4_48

Cited by 5 publications

(5 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Analyzing students’ understanding is essential to gain insights into their conceptual framework in order to understand where they struggle to comprehend the concepts taught. Despite other formative assessment techniques, concept inventories constitute an ecological yet powerful tool to estimate students’ understanding of multiple topics ( Klymkowsky and Garvin-Doxas, 2020 ). Concept inventories generally comprise a set of multiple-choice questions aiming at unveiling the scientifically incorrect conceptions held by students, using distractor items (wrong items) to yield an estimate of the deepness of the understanding of a concept or principle.…”

Section: Introductionmentioning

confidence: 99%

Understanding Randomness on a Molecular Level: A Diagnostic Tool

Tobler¹,

Köhler

Sinha³

et al. 2023

LSE

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Understanding Randomness on a Molecular Level: A Diagnostic Tool

Tobler¹,

Köhler

Sinha³

et al. 2023

LSE

View full text Add to dashboard Cite

show abstract

“…The Rasch calculation and its outputs involve averaging across participants' answers and across items, so for ensuring a credible CI conclusion, the statistical features of the sampling process should be considered with precaution. Most concerns regarding tests as social measurements can be addressed by utilizing standard guides or practical solutions proposed for specific cases provided by the dedicated literature [14][15][16]. However, their realization could be expensive, time-consuming, or even impossible in practice, which reveals the idea for alternative solutions in several applications.…”

Section: Introductionmentioning

confidence: 99%

“…Notice that by nature, the study of knowledge issues is a dynamic and complex process, so the concept inventories have been expanded and improved impressively regarding their uses, components, and analysis. For example, in Smaill & Rowe [17], Raduta [18], and McColgan et al [19], interesting developments of the CI analysis for several parts of electromagnetism are presented; in Laverty and Caballero [20], it is underlined the need for aligning standardized assessments with modern learning goals such as engagement in scientific practices; in Klymkowsky & Garvin-Doxas [14], it is highlighted the use of interactive assessments in which students are called upon to identify and justify their assumptions; in Sands et al [8], it is stated that concept inventories are not perfect tools, suggesting improvement from the measurement perspective, etc. In this view, there is little room for methodical novelties.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Concept Inventory Analysis by Using Indexes, Optimal Histogram Idea, and the Likert Analysis

Prenga,

Kushta,

Musli

2023

J. Hum. Earth Future

View full text Add to dashboard Cite

Since the introduction of the Force Concept Inventory (FCI) in 1992, the CI tests have been widely used for measuring conceptual knowledge and for studying teaching issues in almost all disciplines and levels of study. A standard concept inventory analysis includes the design of a qualitative test, adequate realization of testing, calibration procedure, and comprehensive analysis of its findings. Usually, the CI test calibration is carried out through the Rasch sociometric technique, which is also used for calculating crucial indicators of knowledge such as item difficulties, students’ abilities, and many more. Whereas the quality of the tests’ design can be guaranteed by using certified and professional CI tests, the statistical adequacy of the testing merits critical attention before going on to the final step of the analysis. Also, the analysis of CI outcomes can be advanced by contemplating auxiliary tools and complementary techniques. In this framework, we propose to enforce the test index validity requirement for qualifying the CI outcomes as local or global. Specifically, the conclusions of CI analysis are acceptable for the whole population from which the sample has been extracted if the test's indexes comply with the validity requirements provided by the index theory. In the case when test indexes are out of validity range and re-conducting them is impractical for some objective circumstances or research specifics, we suggest injecting some new records into the existing one or mixing the data gathered from different samples until the new indexes are in the desired range. Using this methodology, we have reviewed our previous FCI tests, which were initially intended to demonstrate the impairment of learning in the physics discipline triggered by online learning during the pandemic closure. Through this renormalization procedure, we obtained a credible assessment of the understanding of mechanics and electromagnetism in high school students who followed online lectures during the pandemic closure. Also, by using indexes’ validity as an auxiliary tool, we identified that for measuring the knowledge of electromagnetism in students enrolled in branches where physics is a basic discipline, a shortened version of the BEMA test was a better instrument than the corresponding shortened EMCI test. Next, we used the optimal histogram idea borrowed from distribution fitting procedures to identify the natural levels of students’ abilities for solving a certain CI test. Another intriguing proposal presented in this work consists of combining an ad-hoc Likert scale assignment for usual errors in physics exams with the FCI designation of the basic commonsense confusion in mechanics for identifying their pairing features in common exams. We believe that the methods proposed herein can improve CI analysis in more general senses. Doi: 10.28991/HEF-2023-04-01-08 Full Text: PDF

show abstract

“…Since its publication, the MHSCI has been used by instructors at over 50 colleges and universities in the United States and around the world. Instructors report using the MHSCI to evaluate pre-and postcourse understanding of microbiological concepts and misconceptions, for course assessment and curriculum design, to assess learning gains, and as a measurement of course efficacy (3)(4)(5). Faculty members administer the MHSCI online through an individualized link that students access to complete the 23-question multiple-choice assessment as a pre-and posttest.…”

Section: Introductionmentioning

confidence: 99%