The Statistics Concept Inventory: Development and Analysis of a Cognitive Assessment Instrument in Statistics

Allen, Kim

doi:10.2139/ssrn.2130143

Cited by 35 publications

(38 citation statements)

References 116 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Of the 33 items, 18 did not load on any factor. These results were similar to previously reported analyses of the SCI, in which exploratory factor analyses indicated that the items did not load on the developers' constructs, nor were there clear groupings among several of the items (Allen, ). These EFA results suggest that SCI items do not have a strong underlying structure and lack cohesion.…”

Section: Resultssupporting

confidence: 90%

See 1 more Smart Citation

An Analytic Framework for Evaluating the Validity of Concept Inventory Claims

Jorion

Gane

James

et al. 2015

J of Engineering Edu

122

View full text Add to dashboard Cite

Background Concept inventories (CIs) are commonly used in engineering disciplines to assess students' conceptual understanding and to evaluate instruction, but educators often use CIs without sufficient evidence that a structured approach has been applied to validate inferences about student thinking. Purpose We propose an analytic framework for evaluating the validity arguments of CIs. We focus on three types of claims: that CI scores enable one to infer (1) students' overall understanding of all concepts identified in the CI, (2) students' understanding of specific concepts, and (3) students' propensity for misconceptions or common errors. Method We applied our analytic framework to three CIs: the Concept Assessment Tool for Statics (CATS), the Statistics Concept Inventory (SCI), and the Dynamics Concept Inventory (DCI). Results Using our analytic framework, we found varying degrees of support for each type of claim. CATS and DCI analyses indicated that the CIs could reliably measure students' overall understanding of all concepts identified in the CI, whereas SCI analyses provided limited evidence for this claim. Analyses revealed that the CATS could accurately measure students' understanding of specific concepts; analyses for the other two CIs did not support this claim. None of the CI analyses provided evidence that the instruments could reliably measure students' misconceptions and common errors. Conclusions Our analytic framework provides a structure for evaluating CI validity. Engineering educators can apply this framework to evaluate aspects of CI validity and make more warranted uses and interpretations of CI outcome scores.

show abstract

Section: Resultssupporting

confidence: 90%

“…Since the nine items decreased the SCI total score reliability, they therefore do not relate as strongly to the other items on the CI. Both alpha values (full instrument and instrument with items removed) were within the .59 to .77 range obtained in a 2005 administration by the developers with a much smaller sample (Allen, ).…”

Section: Resultssupporting

confidence: 56%

An Analytic Framework for Evaluating the Validity of Concept Inventory Claims

Jorion

Gane

James

et al. 2015

J of Engineering Edu

122

View full text Add to dashboard Cite

show abstract

“…Rather than focusing on computational and procedural knowledge, all items in the test dealt with the conceptual understanding of and reasoning about commonly encountered topics in descriptive statistics. Eight items were adapted from the Statistical Concept Inventory (SCI) (Allen, 2006) and seven items from the Statistical Reasoning Assessment (SRA) (Garfield, 2003). The items were selected with the help of the course instructor based on the goals and the content of the course.…”

Section: Statistics Reasoning Abilitymentioning

confidence: 99%

Statistical Reasoning Ability, Self-Efficacy, and Value Beliefs in a Reform Based University Statistics Course

Olani

Hoekstra

Harskamp

et al. 2017

EJREP

View full text Add to dashboard Cite

show abstract

“…The Foundation Coalition maintained a website to publicize the status of concept inventories in fields related to engineering education (Foundation Coalition, 2001), and a report on the status of concept inventories related to engineering topics was provided by (Evans et al, 2003). Continuing initiatives are under way to develop concept inventories targeted to relevant engineering topics (Allen, 2006;Michel, Jackson, Fortier, & Liu, 2006;Gray et al, 2005;Miller, Streveler, Olds, & Slotta, 2011;Morgan & Richardson, 2005;Shallcross, 2010;Wage, Buck, Wright, & Welch, 2005).Misconceptions related to heat, energy and temperature are widely recognized in the literature (Carlton, 2000;Jasien & Oberem, 2002;Sozbilir, 2003;Thomas, Malaquas, Valente, & Antunes, 1995) and it is therefore useful to have instruments to assess these concept areas. This paper describes the development and validation of one such instrument called the Heat and Energy Concept Inventory (HECI).…”

mentioning

confidence: 99%

Development of the Heat and Energy Concept Inventory: Preliminary Results on the Prevalence and Persistence of Engineering Students' Misconceptions

Prince

Vigeant

Nottis

2012

J of Engineering Edu

110

112

View full text Add to dashboard Cite

BACKGROUNDStudents frequently hold a number of misconceptions related to temperature, heat and energy. There is not currently a concept inventory with sufficiently high internal reliability to assess these concept areas for research purposes. Consequently, there is little data on the prevalence of these misconceptions amongst undergraduate engineering students. PURPOSE (HYPOTHESIS)This work presents the Heat and Energy Concept Inventory (HECI) to assess prevalent misconceptions related to: (1) Temperature vs. Energy, (2) Temperature vs. Perceptions of Hot and Cold, (3) Factors that affect the Rate vs. Amount of Heat Transfer and (4) Thermal Radiation. The HECI is also used to document the prevalence of misconceptions amongst undergraduate engineering students. DESIGN/METHODItem analysis, guided by classical test theory, was used to refine individual questions on the HECI. The HECI was used in a one group, pre-test-post-test design to assess the prevalence and persistence of targeted misconceptions amongst a population of undergraduate engineering students at diverse institutions. RESULTSInternal consistency reliability was assessed using Kuder-Richardson Formula 20; values were 0.85 for the entire instrument and ranged from 0.59 to 0.76 for the four subcategories of the HECI. Student performance on the HECI went from 49.2% to 54.5% after instruction. Gains on each of the individual subscales of the HECI, while generally statistically significant, were similarly modest. CONCLUSIONSThe HECI provides sufficiently high estimates of internal consistency reliability to be used as a research tool to assess students' understanding of the targeted concepts. Use of the instrument demonstrates that student misconceptions are both prevalent and resistant to change through standard instruction. KEYWORDS: concept inventory, heat, misconception INTRODUCTIONIn their seminal work "How People Learn" (Bransford, Brown, & Cocking, 2000) the authors emphasize the critical role in teaching of building from students' current conceptual framework. Since learning may be viewed as a process in which students fit new ideas 412 413 101 (July 2012) 3 Journal of Engineering Education into their existing mental structures, a flawed conceptual framework hinders students' ability to integrate and understand what is being taught. Part of effective instruction, therefore, entails engaging students in ways that uncover their existing conceptual frameworks, and then repairing any misconceptions. Traditional instruction often bypasses these steps, with the result that students leave their courses with many of their misconceptions intact. Decades of research in the sciences demonstrate the modest levels of conceptual change fostered in many classrooms (Center for Development and Learning, 2000;Deslauriers, Schelew, & Wieman, 2011;Lightman & Sadler, 1993;Sahin, 2010;Smith, diSessa, & Roschelle, 1993;Suping, 2003) and more limited studies in engineering indicate a similar pattern (Krause, Decker, & Griffin, 2003;Miller et al., 2006;Steif, Dollar, & D...

show abstract

The Statistics Concept Inventory: Development and Analysis of a Cognitive Assessment Instrument in Statistics

Cited by 35 publications

References 116 publications

An Analytic Framework for Evaluating the Validity of Concept Inventory Claims

An Analytic Framework for Evaluating the Validity of Concept Inventory Claims

Statistical Reasoning Ability, Self-Efficacy, and Value Beliefs in a Reform Based University Statistics Course

Development of the Heat and Energy Concept Inventory: Preliminary Results on the Prevalence and Persistence of Engineering Students' Misconceptions

Contact Info

Product

Resources

About