True-false analysis reveals inherent flaws in multiple true-false tests

Abstract

Background: Multiple true-false tests (MTF), a component of our assessment system, have consistently generated low scores and many failures. This was attributed to the negative marking scheme. However, no study was conducted to explore the issue further. Item analysis revealed that students omitted false options more frequently and answered them wrongly more frequently than true options. The aim of this study was to determine the performance discrepancy between true and false options of MTF tests and the reasons for such discrepancy and the poor performance of MTF in general.Methods: The student performance of past 7 years of year-3 medicine end-of-posting examinations was analysed. The item analysis reports of 23 MTF tests were used to determine the significance of the differences in omission rates, correct-answer rates and the discrimination index of true and false options.Results: There were statistically significant differences in the omission rates, correct-answer rates and discrimination index values of true and false options. This study revealed that the false options consistently let down student performance. Although negative marking could be partly blamed for the situation, no justification could be found for the use of false options to test knowledge.Conclusions: Some publications endorse MTF, but many highlight its drawbacks. The use of false options in MTF was seen as an inherent defect in this instrument. As viable alternatives like VSAQ and Constructed Response Tests are in the horizon, we conclude that MTF ought to be discarded as an assessment instrument.

Multiple Choice Questions Versus Very Short Answered Questions in the Evaluation of Students of Veterinary Pathology

“…Students' performance in MTF questions in our study was also quite higher. This is inconsistent to the finding of Puthiaparampil et al (18) who reported low student scores in MTF question exam. In addition to the reasons that push students to prefer BSA questions, partial credit for any corrected choices enabled students to achieve higher scores in MTF.…”

Very short answer questions: a viable alternative to multiple choice questions

“…Test-wiseness and guessing, which are unavoidable in MCQ tests are of no concern in VSAQ. It has also been observed that students found false options harder than true ones to answer, and omission rates due to uncertainty are high in MTF [21]. The adverse effect of using false options was also highlighted by Wood [10].…”

Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions

“…Test-wiseness and guessing, which are unavoidable in MCQ tests are of no concern in VSAQ. It has also been observed that students found false options harder than true ones to answer, and omission rates due to uncertainty are high in MTF [20]. The adverse effect of using false options was also highlighted by Wood [10].…”