Imprecise terms in UK medical multiple‐choice questions: what examiners think they mean

Abstract

Many multiple-choice questions (MCQs) used in medical education in the UK contain undefined, imprecise terms. They are particularly common in true/false items and can be found in classroom tests, published examples of MCQs and, more importantly, in high-stakes examinations which determine a candidate's graduation or membership of a professional body. This study investigated imprecise terms used in some MB BS final examinations and the Part 1 Membership Examination of the Royal College of Physicians. It revealed that imprecise terms occur commonly, yet there is a wide range of opinion among the examiners themselves about their meanings. The numbers and variety of imprecise terms which were found in high-stakes MCQ examinations are described in this paper and details are given concerning the lack of consensus about their meanings as reported by the responsible examiners. A second type of construction error--disproportionately large numbers of 'true' branches--was also recorded. Exemplary practices do exist in MCQ quality assurance, but in the UK they are very much the exception rather than the rule. The findings of this investigation strongly indicate a need for change.

Frequency vs. intensity: which should be used as anchors for self-report instruments?

“…Other studies revealed that even terms like “never” or “always” which were expected to be rated as absolute “0%” or “100%” both were indicated with a range up to 20% [10]. …”

Frequency vs. intensity: Framing effects on patients' use of verbal rating scale anchors

“…In summary, the existing research indicates that it is a rather difficult task to differentiate between frequency or intensity terms for both medical experts and patients with a depressive disorder or fatigue [8,17,18]. Interindividual congruency and intraindividual stability across time of mental representations of such terms was shown to be rather low.…”

De ontwikkeling van een feedbacksysteem voor toetsvragenmakers

“…Wenselijke studentprestaties kunnen dan ook niet goed worden geëvalueerd en op rechtvaardige wijze worden beloond. [6][7] Toetsing binnen de FMW bestaat voor een groot deel uit schriftelijke toetsen met geprecodeerde toetsvragen. De kwaliteit van deze laat echter vaak te wensen over.…”