Contemplation on marking scheme for Type X multiple choice questions, and an illustration of a practically applicable scheme

Abstract

Ever since its inception 100 years back, multiple choice items have been widely used as a method of assessment. It has certain inherent limitations such as inability to test higher cognitive skills, element of guesswork while answering, and issues related with marking schemes. Various marking schemes have been proposed in the past but they are not balanced, skewed, and complex, which are based on mathematical calculations which are typically not within the grasp of medical personnel. Type X questions has many advantages being easy to construct, can test multiple concepts/application/facets of a topic, cognitive skill of various level of hierarchy can be tested, and unlike Type K items, they are free from complicated coding. In spite of these advantages, they are not in common use due to complicated marking schemes. This is the reason we explored the aspects of methods of evaluation of multiple correct options multiple choice questions and came up with the simple, practically applicable, nonstringent but logical scoring system for the same. The rationale of the illustrated marking scheme is that it takes into consideration the distracter recognition ability of the examinee rather than relying on the ability only to select the correct response. Thus, examinee's true knowledge is tested, and he is rewarded accordingly for selecting a correct answer and omitting a distracter. The scheme also penalizes for not recognizing a distracter thus controlling guessing behavior. It is emphasized that if the illustrated scoring scheme is adopted, then Type X questions would come in common practice.

Comparison of performance in physiology subject assessed by short essay type questions and multiple choice questions of first year medical students

“…In present study, we analyses the effectiveness of MCQs and OSPE type of assessment in first year medical curriculum. 9 Our data showed that objective type of examinations results in to greater scoring over conventional type of method in theory and practical. In previous studies 10,11 authors observed that the score obtained in MCQs was similar to the score of SEQs and concluded that the students who performed well in the essay were also likely to do well in the MCQs.…”

Effect of different scoring approaches upon credit assignment when using Multiple True‐False items in dental undergraduate examinations

“…Credit will only be given at or above chance performance (≥ n /2 statements answered correctly). Additional fractional points are awarded as the number of correct answers increases Count ‐ 3: Compared to the Count‐2 approach, the minimum number of correct answers required for partial scoring might be increased and credit would be given only above chance performance (> n /2 statements answered correctly) …”

“…Additional fractional points are awarded as the number of correct answers increases Count ‐ 3: Compared to the Count‐2 approach, the minimum number of correct answers required for partial scoring might be increased and credit would be given only above chance performance (> n /2 statements answered correctly) “Vorkauf ‐ Method”: Half‐credit might be awarded even if only 1 answer is incorrect, otherwise 0 credit or 1 credit (full credit) is awarded …”

“…Multiple response (MR): The most restrictive rating gives credit (1 point) only if all options are answered correctly. This approach is called Multiple response (MR), Number Right scoring or Dichotomous scoring (DS) Correction for Guessing: As opposed to all approaches above, this approach uses a scale from −1 to 1.…”

“…Up to now, different scoring approaches leading to different examination scores have been developed: MTF/PS 1/n, Count‐2, Count‐3, “Vorkauf‐Method”, PS 50, Dichotomized MTF, “Blasberg‐Method”, Multiple response (MR)/Number Right scoring/Dichotomous scoring (DS), Correction for Guessing/Negative Marking (NM), “Ripkey‐Method”, Morgan‐Method, Balanced Scoring Method and Let Omit Method . As all MTF statements have to be answered separately, most scoring approaches, except for the MR method, value partial knowledge.…”