Assessment Analysis: How It Is Done

Puthiaparampil, Thomas

doi:10.15694/mep.2017.000142

Cited by 4 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The bars show percentages of items falling in the three categories: ≤0.0 = no or negative discrimination; 0.1-0.19 = fair and ≥ 0.2 good and excellent distribution combined. Item analysis of VSAQ was manually performed on Excel Worksheet using the 'UNIMAS Formulae'[18]…”

mentioning

confidence: 99%

Very short answer questions: a viable alternative to multiple choice questions

Puthiaparampil

Rahman

2020

BMC Med Educ

Self Cite

View full text Add to dashboard Cite

Background: Multiple choice questions, used in medical school assessments for decades, have many drawbacks such as hard to construct, allow guessing, encourage test-wiseness, promote rote learning, provide no opportunity for examinees to express ideas, and do not provide information about strengths and weakness of candidates. Directly asked, directly answered questions like Very Short Answer Questions (VSAQ) are considered a better alternative with several advantages. Objectives: This study aims to compare student performance in MCQ and VSAQ and obtain feedback. from the stakeholders. Methods: Conduct multiple true-false, one best answer, and VSAQ tests in two batches of medical students, compare their scores and psychometric indices of the tests and seek opinion from students and academics regarding these assessment methods. Results: Multiple true-false and best answer test scores showed skewed results and low psychometric performance compared to better psychometrics and more balanced student performance in VSAQ tests. The stakeholders' opinions were significantly in favour of VSAQ. Conclusion and recommendation: This study concludes that VSAQ is a viable alternative to multiple-choice question tests, and it is widely accepted by medical students and academics in the medical faculty.

show abstract

mentioning

confidence: 99%

Very short answer questions: a viable alternative to multiple choice questions

Puthiaparampil

Rahman

2020

BMC Med Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…MTF shows a fair distribution, BAQ higher than MTF and VSAQ about 70% of items having the highest discrimination power. Item analysis of VSAQ was manually performed on Excel Worksheet using the UNIMAS formulae[17].…”

mentioning

confidence: 99%

Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions

Puthiaparampil

Rahman

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background Multiple choice questions, used in medical school assessments for decades, have many drawbacks, such as: hard to construct, allow guessing, encourage test-wiseness, promote rote learning, provide no opportunity for examinees to express ideas, and do not provide information about strengths and weakness of candidates. Directly asked and answered questions like Very Short Answer Questions (VSAQ) is considered a better alternative with several advantages. Objectives This study aims to substantiate the superiority of VSAQ by actual tests and obtaining feedback from the stakeholders. Methods Conduct multiple true-false, one best answer and VSAQ tests in two batches of medical students, compare their scores and psychometric indexes of the tests and seek opinions from students and academics regarding these assessment methods. Results Multiple true-false and best answer test scores showed skewed results and low psychometric performance compared to better psychometrics and more balanced student performance in VSAQ tests. The stakeholders’ opinions were significantly in favour of VSAQ. Conclusion and recommendation This study concludes that VSAQ is a viable alternative to multiple choice question tests, and it is widely accepted by medical students and academics in the medical faculty.

show abstract

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Kanzow¹,

Schmidt²,

Kanzow³

2023

JMIR Med Educ

View full text Add to dashboard Cite

Background Single-choice items (eg, best-answer items, alternate-choice items, single true-false items) are 1 type of multiple-choice items and have been used in examinations for over 100 years. At the end of every examination, the examinees’ responses have to be analyzed and scored to derive information about examinees’ true knowledge. Objective The aim of this paper is to compile scoring methods for individual single-choice items described in the literature. Furthermore, the metric expected chance score and the relation between examinees’ true knowledge and expected scoring results (averaged percentage score) are analyzed. Besides, implications for potential pass marks to be used in examinations to test examinees for a predefined level of true knowledge are derived. Methods Scoring methods for individual single-choice items were extracted from various databases (ERIC, PsycInfo, Embase via Ovid, MEDLINE via PubMed) in September 2020. Eligible sources reported on scoring methods for individual single-choice items in written examinations including but not limited to medical education. Separately for items with n=2 answer options (eg, alternate-choice items, single true-false items) and best-answer items with n=5 answer options (eg, Type A items) and for each identified scoring method, the metric expected chance score and the expected scoring results as a function of examinees’ true knowledge using fictitious examinations with 100 single-choice items were calculated. Results A total of 21 different scoring methods were identified from the 258 included sources, with varying consideration of correctly marked, omitted, and incorrectly marked items. Resulting credit varied between –3 and +1 credit points per item. For items with n=2 answer options, expected chance scores from random guessing ranged between –1 and +0.75 credit points. For items with n=5 answer options, expected chance scores ranged between –2.2 and +0.84 credit points. All scoring methods showed a linear relation between examinees’ true knowledge and the expected scoring results. Depending on the scoring method used, examination results differed considerably: Expected scoring results from examinees with 50% true knowledge ranged between 0.0% (95% CI 0% to 0%) and 87.5% (95% CI 81.0% to 94.0%) for items with n=2 and between –60.0% (95% CI –60% to –60%) and 92.0% (95% CI 86.7% to 97.3%) for items with n=5. Conclusions In examinations with single-choice items, the scoring result is not always equivalent to examinees’ true knowledge. When interpreting examination scores and setting pass marks, the number of answer options per item must usually be taken into account in addition to the scoring method used.

show abstract

Assessment Analysis: How It Is Done

Cited by 4 publications

References 0 publications

Very short answer questions: a viable alternative to multiple choice questions

Very short answer questions: a viable alternative to multiple choice questions

Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Contact Info

Product

Resources

About