Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine?

Abstract

BackgroundWritten assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity.DiscussionAn extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format.SummaryThe evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-014-0249-2) contains supplementary material, which is available to authorized users.

A descriptive analysis of extended matching questions among third year medical students

“…29 Furthermore, good quality, wellconstructed distractors, does reduce cueing. 30 Hence the importance of EMQs with more distracters. However distractors can't just be "fillers" they need to be good and plausible.…”

State-of-the-art usage of simulation in anesthesia

“…The goal of summative assessment is to assess student learning at the end of an instructional unit by comparing it against some standard [71] and means that they can either pass or fail. For an assessment to be summative, it must be both reliable (meaning that the test represents the true abilities of the person) and valid (the test must accurately represent the nature of what is intended to be measured) [72,73].…”

Clinical REsearch During Outbreaks (CREDO) Training for Low- and Middle-Income Countries

“…Our training model was designed with the workshops occurring before and after the e-learning component to introduce and then solidify knowledge gained in the online component. Our assessment model combined multiple-choice questions in the e-learning sessions with simulation exercises in the workshop sessions to provide a balance in testing between recall and critical or creative thinking ( 7 ). …”