A comparison of multiple-choice tests and free-response tests in examinations of clinical competence

Abstract

This paper reports a study which compared the performance of different groups of students and doctors on identical and equivalent tests set in an objective-type format and in a free-response format. The tests were designed to ensure that the content was relevant to clinical practice at the hospital intern level. In all test situations candidates' scores were significantly higher in the objective tests than in the free-response tests. This difference was greater for the more junior and less competent students than for the more competent doctors. The cueing effect of the options was thought to be the main factor responsible for the difference in performance. The results of a questionnaire survey demonstrated that students were aware of the deficiencies in multiple-choice tests. A large majority of the students believed that the free-response tests gave a more accurate assessment of their clinical ability. It was found that in these tests, aimed at measuring aspects of clinical competence, multiple-choice questions appeared to overestimate the candidate's ability to an extent that made them less suitable than free-response questions for this purpose. It was also found that free-response tests, of the type used in this study, provide a suitable alternative to multiple-choice tests for use in the written section of clinical examinations. It was concluded that the written component of the final examination in the medical course should have a preponderance of free-response items over multiple-choice items.

Assessment in a problem-based medical curriculum

“…Further serious problems with the scoring method were exposed. The effect of cueing by the optional answers was demonstrated to increase the scores markedly (Goran et al, 1973;Martin, 1975;Newble et al, 1979;Norman & Feightner, 1981). Also it proved very difficult to assign score weights, and find a balance among different answering strategies (McGuire, 1974;Schwabbauer, 1975).…”

The use of an essay examination in evaluating medical students during the surgical clerkship

“…As such the inflated mean scores implicit in MCQ exams may overestimate the clinical skills a medical student is perceived to have. 7,8 Irrespective of the potential flaws in MCQ exams, they have managed to dominate the method in which medical students' objective knowledge is assessed due to the inherent simplicity of administration. These forms of examinations are logistically M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT easier to administer, more cost-effective and test more clinical situations per unit time.…”

Rating Scale Measures in Multiple-Choice Exams: Pilot Studies in Pharmacology

“…ere is controversy whether MCQ only assess lower levels of knowledge like recall of isolated facts, encourage trivialization of knowledge, or lend to rote learning [4]. One main concern, when using these MCQ-based examinations, is whether really knowledge is assessed or other patterns like structure of the personality, the experience with the test format ("test sophistication" [5]), or the decision to take an educated guess.…”