Confidence versus Performance as an Indicator of the Presence of Alternative Conceptions and Inadequate Problem‐Solving Skills in Mechanics

Abstract

This study investigated the use of performance-confidence relationships to signal the presence of alternative conceptions and inadequate problem-solving skills in mechanics. A group of 33 students entering physics at a South African university participated in the project. The test instrument consisted of 20 items derived from existing standardized tests from literature, each of which was followed by a self-reported measure of confidence of students in the correctness of their answers. Data collected for this study included students' responses to multiple-choice questions and open-ended explanations for their chosen answers. Fixed response physics and confidence data were logarithmically transformed according to the Rasch model to linear measures of performance and confidence. The free response explanations were carefully analysed for accuracy of conceptual understanding. Comparison of these results with raw score data and transformed measures of performance and confidence allowed a re-evaluation of the model developed by Hasan, Bagayoko and Kelley in 1999 for the detection of alternative conceptions in mechanics. Application of this model to raw score data leads to inaccurate conclusions. However, application of the Hasan hypothesis to transformed measures of performance and confidence resulted in the accurate identification of items plagued by alternative conceptions. This approach also holds promise for the differentiation between over-confidence due to alternative conceptions or due to inadequate problem-solving skills. It could become a valuable tool for instructional design in mechanics. IntroductionIn order to ensure a smooth transition between secondary and tertiary education, it is imperative for lecturers of physics to be well informed about the baseline knowledge and understanding of students upon entry to tertiary education. Furthermore, where lack of preparedness is identified lecturers should ideally be able to distinguish between a lack of knowledge about specific concepts and the presence of strong alternative conceptions. It is widely accepted that the instructional design for conceptual change of firmly rooted alternative conceptions will differ markedly from that aimed at acquisition of new knowledge or development of problem-solving skills. Hasan, Bagayoko, and Kelley (1999) proposed that a student's self-reported certainty of response could be used in conjunction with the answer to a conceptual test item to differentiate between lack of knowledge and the presence of misconceptions in mechanics. While acknowledging that their hypothesis is best utilised for individual students, they have expanded it for application to groups of students in an attempt to enable the lecturer to extract this information with relative ease for large groups of students. This paper investigated the use of the relationship between confidence and performance to signal not only the presence of alternative conceptions but also inadequate problem-solving skills in mechanics and as such serve as a practitioner's tool in ...

SHS Students’ Difficulty in Solving Impulsee and Momenyum Problem

“…This analysis method that is also known as Certainty of Response Index (CRI) had been done by some researchers, e.g. Hasan et al (1999) and Potgieter et al (2010). The method used for the analysis of students' understanding level in this research referred to the method employed by Sutopo (2016) who adapted the method utilized by Potgieter et al (2010).…”

“…Hasan et al (1999) and Potgieter et al (2010). The method used for the analysis of students' understanding level in this research referred to the method employed by Sutopo (2016) who adapted the method utilized by Potgieter et al (2010). Based on their answers, the levels of students understanding were grouped into 4 categories.…”

The Confidence-Accuracy Relationship in Diagnostic Assessment: The Case of the Potential Difference in Parallel Electric Circuits

“…The authors concluded that the method could be easily employed to differentiate alternative conceptions from a lack of knowledge. Then, the method was used to distinguish a lack of knowledge from alternative conceptions in mechanics (Planinic, Boone, Krsnik, & Beilfuss, 2006;Potgieter, Malatje, Gaigher, & Venter, 2010); simple direct-current circuits (Peşman & Eryılmaz, 2010;Planinic et al, 2006); radioactivity (Colclough, Lock, & Soares, 2011); diffusion and osmosis (Odom & Barrow, 2007); and environmental education (Arslan, Çiğdemoğlu, & Moseley, 2012). The review of these studies suggests that the Hasan et al 's method can be employed to distinguish a lack of knowledge from alternative conceptions.…”

Conceptual and Procedural Approaches to Mathematics in the Engineering Curriculum – Comparing Views of Junior and Senior Engineering Students in Two Countries

“…Commonly, a respondent is then requested to indicate the degree of certainty she/he has in her/his ability to select and use well-established knowledge, concepts and rules to arrive at each answer (Potgieter, Malatje, Gaigher, & Venter, 2010). Confidence is most often measured on a scale, in our case 1 -4, where 1 implies a total guess and 4 a complete confidence.…”

“…However, if the confidence is high and the answer wrong it indicates a misplaced confidence in the person's knowledge on the particular subject matter, either suggesting a misjudgement of her/his own ability or a sign of existing misconceptions. A confidence measure, together with the correctness or not of a response can thus be used to distinguish between a lack of knowledge and either over-confidence or the existence of a misconception (Potgieter et al, 2010).…”