A comparison of short and multiple choice questions in the evaluation of students of biochemistry

Abstract

Summary Premedical and other undergraduate students were given an examination consisting of a section of short questions and a section of multiple choice questions of the one in five type. After correction for random guessing, the average multiple choice mark was close to the average short question mark. Thus the theoretical ‘average’ student did equally well in both types of examination. Individual multiple choice marks were plotted against the corresponding short question mark. Although the two sets of marks were highly correlated (P < 0.001), there was still a wide scatter around the best‐fit, least‐square line (r= 0.63). Thus dependence on one type of examination alone could have seriously penalized some students. The least‐squares line was extrapolated to determine the multiple‐choice mark of a theoretical ‘idiot’ student, defined as one obtaining zero marks in the short question section of the examination. This student did better than predicted on the basis of random guessing. On the other hand a theoretical ‘genius’ student, defined as one obtaining top marks in the short question section, did less well than predicted. Thus multiple choice examinations, even when corrected for the random guessing component, may overestimate weaker students and underestimate stronger students.

The assessment of an examination in pathology consisting of multiple-choice, practical and short essay questions

“…Forsdyke (1978), comparing results of multiple‐choice and short essay questions in biochemistry examinations reported no significant difference between the two. With more experience with practical and short essay items, our students would probably give a similar result.…”

“…Furthermore, plots of the scores of individual sections versus the criterion and calculations of the respective best‐fit least‐squares lines showed that different parts of the student population (above and below average relative to the criterion) were not significantly advantaged or disadvantaged by the different item types. These results are not in agreement with those published by Forsdyke (1978). Construct validity . Evidence confirming the presence of constructs that were identified with each of the examination sections but also contributed to the general construct—achievement in pathology—was required.…”

“…Furthermore, plots of the scores of individual sections versus the criterion and calculations of the respective best‐fit least‐squares lines showed that different parts of the student population (above and below average relative to the criterion) were not significantly advantaged or disadvantaged by the different item types. These results are not in agreement with those published by Forsdyke (1978).…”

Samuel Butler and human long term memory: Is the cupboard bare?

“…For example, one's knowledge influences one's perception of one's environment (selective attention). The fact that cues can facilitate retrieval (Forsdyke, 1978), suggests that often the factor limiting retrieval is the retrieval process itself, rather than presence in the store.…”

The multiple choice tutorial: its use in the reinforcement of fundamentals in medical education

“…The multiple choice question (MCQ) is well established and widely used in a large number of medical schools and it is considered by many examiners to be an effective method of assessment. The MCQ has many advantages over the conventional essay question, and there has been evidence to show a high degree of correlation between MCQ and short answer question performance (Forsdyke, 1978). There are also disadvantages (Newble, Baxter & Elmslie, 1979).…”