The interaction of correctness of and certainty in MCQ responses

Tweed, Mike

doi:10.1080/0142159x.2017.1396309

Cited by 3 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our findings highlight the high confidence of medical students in their answers. Indeed, the most frequent confidence statement was "A", consistently with previous studies showing the high confidence in answers in medical and dental students [25,33,39]. This high confidence was more frequently associated with a correct answer.…”

Section: Discussionsupporting

confidence: 86%

“…Along with these descriptive analyses of the interaction between confidence statement and correctness, we examined two parameters described by Tweed [33]. Indeed, the degree of confidence when the answer is correct differs from correctness of the answer when the student is confident.…”

Section: Discussionmentioning

confidence: 99%

“…In this view, interaction between confidence and correctness can be measured directly, but can also be differentiated into two distinct parameters. Indeed, as Tweed suggested it, correctness while being confident (namely "knowledge self-monitoring") and confidence while being correct (namely "efficiency") are two different abilities that should be analyzed separately [33]. They might indeed reflect different information with regards to the student's, hence future practitioner's, behavior.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

et al. 2020

View full text Add to dashboard Cite

Background Multiple-choice question (MCQ) tests are commonly used to evaluate medical students, but they do not assess self-confidence nor penalize lucky guess or harmful behaviors. Based on a scoring method according to the appropriateness of confidence in answers, the study aimed at assessing knowledge self-monitoring and efficiency, and the determinants of self-confidence. Methods A cross-sectional study of 842 s- and third-year medical students who were asked to state their level of confidence (A: very confident, B: moderately confident and C: not confident) during 12 tests (106,806 events). A bonus was applied if the level of confidence matched with the correctness of the answer, and a penalty was applied in the case of inappropriate confidence. Results Level A was selected more appropriately by the top 20% students whereas level C was selected more appropriately by the lower 20% students. Efficiency of higher-performing students was higher when correct (among correct answers, rate of A statement), but worse when incorrect compared to the bottom 20% students (among incorrect answers, rate of C statement). B and C statements were independently associated with female and male gender, respectively (OR for male vs female = 0.89 [0.82–0.96], p = 0.004, for level B and 1.15 [1.01–1.32], p = 0.047, for level C). Conclusion While both addressing the gender confidence gap, knowledge self-monitoring might improve awareness of students’ knowledge whereas efficiency might evaluate appropriate behavior in clinical practice. These results suggest differential feedback during training in higher versus lower-performing students, and potentially harmful behavior in decision-making during clinical practice in higher-performing students.

show abstract

Section: Discussionsupporting

confidence: 86%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Adding the potential safety of responses to item level certainty [30][31][32] has been included in other research programmes. Another criterion to consider might relate to too high a proportion correct for no certainty [43], as this may indicate inefficiency in self-monitoring. It would be possible to generate an additional metric based on certainty when correct [31,32,44], but the analysis we describe prioritises safety (being correct when certain) over efficiency (being certain when correct) [43].…”

Section: Discussionmentioning

confidence: 99%

“…An alternative analysis could have been increasing certainty with increasing levels of correctness [ 28 – 32 ]. Although both these analyses could be valid, our favoured one is better aligned to self-monitoring for safe decision making in practice whereas the latter is better aligned to efficient decision making [ 43 ].…”

Section: Discussionmentioning

confidence: 99%

Defining and tracking medical student self-monitoring using multiple-choice question item certainty

2020

Self Cite

View full text Add to dashboard Cite

Background Self-monitoring is an important component of clinical practice. It is underpinned by the framework of self-efficacy which is concerned with judgments of how well one believes one can achieve or perform a task. This research aimed to develop criteria for adequate self-monitoring, then to measure patterns of self-monitoring, and to explore how these patterns relate to a student’s year in a medical course and to patterns of knowledge. Methods Analysis of individuals’ levels of correctness in answering assessment items and their certainty in correctness may be used to inform assessments of ability to self-monitor. Two criteria were proposed and applied to define adequate self-monitoring. Firstly, increasing proportions correct with increasing levels of certainty. Secondly, having a proportion correct for high certainty responses that was not lower than cohort levels. Student responses in progress tests comprising multiple-choice questions (MCQs) and associated certainty were analysed. Criteria for the presence of adequate self-monitoring and for adequate knowledge were applied to the results of each of four tests conducted over 2 years, and used to categorise patterns of self-monitoring and knowledge. Results Data from 3 year group cohorts totalling 737 students were analysed. The majority (58%) of students demonstrated adequate knowledge and met both criteria for adequate self-monitoring across all four tests. The most advanced year group cohort had the highest rates of adequate knowledge and the highest rates of meeting both self-monitoring criteria. The patterns of self-monitoring were the same as the patterns of knowledge across the four tests for 454 students, but for the remaining 283 the patterns of self-monitoring and knowledge differed. Conclusion Analysis of responses to item level certainty has informed development of a definition of adequate self-monitoring that may be applied to individual student’s responses from a single test, and to track the adequacy of a student’s self-monitoring over time. Patterns of self-monitoring tend to match patterns of knowledge, but not in all cases, suggesting the self-monitoring measure could provide additional information about student ability.

show abstract

Is it time for transition from the subject-based to the integrated preclinical medical curriculum?

Kapitonova¹,

Gupalo²,

Dydykin³

et al. 2020

Russ Open Med J

View full text Add to dashboard Cite

In the 60s of the last century, a number of new universities in the world began to apply an integrated program of medical education, the cornerstone of which was problem-oriented education. Thus, the Flexner model of higher education adopted by that time in most countries of the world, with its characteristic segregation of teaching of the theoretical and clinical disciplines, which had ceased to satisfy the needs of modern healthcare, was gradually replaced by a new system that put the student in the center of the educational process and opened the way to active methods of teaching being focused on the end result – training of graduates whose qualifications most fully satisfy the needs of society. Over the half-century history of its existence, this system has been adopted by most medical universities in different countries of the world, in many of which it has undergone significant modifications in accordance with the needs of national educational standards. Many medical universities in Russia and other countries of the former Soviet Union showed interest in this system, some of the medical faculties of our country accepted certain elements of it. However, up to date no integrated preclinical medical education program has been applied in any of the Russian universities. Hereby we are undertaking an attempt to analyze the reasons and assess the possible perspectives for the transition of medical universities in Russia to teaching of fundamental and biomedical disciplines using the integrated curriculum.

show abstract

The interaction of correctness of and certainty in MCQ responses

Cited by 3 publications

References 5 publications

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

Defining and tracking medical student self-monitoring using multiple-choice question item certainty

Is it time for transition from the subject-based to the integrated preclinical medical curriculum?

Contact Info

Product

Resources

About