Rating Scale Measures in Multiple-Choice Exams: Pilot Studies in Pharmacology

Melzer, Andreas; Gergs, Ulrich; Lukás, Josef; Neumann, Joachim

doi:10.1155/2018/8615746

Cited by 4 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, 87.6% (211) students from 241 eligible students entered the first online study, and 77.5% (186) from 240 eligible students took part at the second online test one year later. e same observations hold true in formative written tests, and it may therefore be generalized that a bonus system is always necessary for student participation rates of formative tests in tertiary education [18]. Students were not aware that the very same questions would be given one year later (February 2019) in the online test because in previous years, different questions were provided in the second online test.…”

Section: Methodsmentioning

confidence: 86%

Comparison of Online Tests of Very Short Answer versus Single Best Answers for Medical Students in a Pharmacology Course over One Year

Neumann

Simmrodt

Teichert

et al. 2021

Education Research International

Self Cite

View full text Add to dashboard Cite

Single best answers (single choice, SC) are the classical tools used in medical examinations on all levels of education. In contrast, very short answer (VSA) probably requires sound prior knowledge and deeper learning than SC, and VSA should make cueing and guessing impossible. Hence, in a basic pharmacology course, we wanted to compare the SC and VSA formats directly at the end of the course and one year later. Medical students (n = 211) were given a formative online test. Two groups were randomly formed (A and B). Participants in group A were first given fifteen single choices (one out of five) pharmacology questions and thereafter fifteen very short answer questions (open question which were to be answered online and semiautomatically assessed). Very similar questions with regard to learning objectives but in opposite order were given to group B. After one year, about half of students from group A were again given the very same questions (AA) or the opposite questions (AB). Likewise, group B was again tested with the opposite questions (BA) or the same (BB). The SC questions in groups A, AA, AB, B, BA, and BB were in sum easier to answer than the corresponding VSA questions. Repeating the test after one year with the same students increased retention of right answers by about 1.5 points. In summary, direct comparison questions in the VSA format are more difficult for our students to answer than questions in the SC format, conceivably because cueing and guessing are eliminated. Knowledge retention is present by repeating the very same examination format online. Retention of knowledge is higher when starting with VSA (group B) both for a subsequent SC format or a VSA format. These data would argue for more use of the VSA format at least in pharmacology examinations.

show abstract

Section: Methodsmentioning

confidence: 86%

Comparison of Online Tests of Very Short Answer versus Single Best Answers for Medical Students in a Pharmacology Course over One Year

Neumann

Simmrodt

Teichert

et al. 2021

Education Research International

Self Cite

View full text Add to dashboard Cite

show abstract

“…S-MCQs were more associated with A and C statements, which represent the more radical confidence statements. Indeed, there is higher chance that a student answering a S-MCQ will know for sure if his answer is correct or incorrect [42].…”

Section: Discussionmentioning

confidence: 99%

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

et al. 2020

View full text Add to dashboard Cite

Background Multiple-choice question (MCQ) tests are commonly used to evaluate medical students, but they do not assess self-confidence nor penalize lucky guess or harmful behaviors. Based on a scoring method according to the appropriateness of confidence in answers, the study aimed at assessing knowledge self-monitoring and efficiency, and the determinants of self-confidence. Methods A cross-sectional study of 842 s- and third-year medical students who were asked to state their level of confidence (A: very confident, B: moderately confident and C: not confident) during 12 tests (106,806 events). A bonus was applied if the level of confidence matched with the correctness of the answer, and a penalty was applied in the case of inappropriate confidence. Results Level A was selected more appropriately by the top 20% students whereas level C was selected more appropriately by the lower 20% students. Efficiency of higher-performing students was higher when correct (among correct answers, rate of A statement), but worse when incorrect compared to the bottom 20% students (among incorrect answers, rate of C statement). B and C statements were independently associated with female and male gender, respectively (OR for male vs female = 0.89 [0.82–0.96], p = 0.004, for level B and 1.15 [1.01–1.32], p = 0.047, for level C). Conclusion While both addressing the gender confidence gap, knowledge self-monitoring might improve awareness of students’ knowledge whereas efficiency might evaluate appropriate behavior in clinical practice. These results suggest differential feedback during training in higher versus lower-performing students, and potentially harmful behavior in decision-making during clinical practice in higher-performing students.

show abstract

Using Precourse Formative Written Testing in a Pharmacology Class Greatly Increases Medical Students’ Performance in Final Written Summative Tests

Neumann

Simmrodt

Gergs

2020

Education Research International

Self Cite

View full text Add to dashboard Cite

We wanted to test the progress of medical students at our university in a pharmacology course. The formal teaching was given as lectures to the full class of students. We gave the very same written test of multiple-choice (MC) questions (single best choice) to third-year medical students before and after a one semester course of basic pharmacology. The initial voluntary test (containing 30 MC questions) was taken by 79% of the eligible students (n = 147), a week before pharmacology lectures had started. Defining a passing grade of 60% of right answers, only 2% of the students passed the test. The range was between 5 and 21 points. The final, now obligatory, written test at the end of the course (one week after the last lecture in pharmacology) was taken by all students in the semester (n = 179) and was passed by 95%, of students, again defined by the same passing score. Here, the points obtained ranged from 12 to 29. Over the time of the semester, the attendance in the lectures dropped dramatically to less than 10% of the students. Hence, progress tests are useful, but they hardly measure the gain in knowledge through attendance in the pharmacology lecture (the intervention); they also measure other sources of knowledge, such as textbook reading or memorizing only the initial questions and looking up the answers.

show abstract

Rating Scale Measures in Multiple-Choice Exams: Pilot Studies in Pharmacology

Cited by 4 publications

References 22 publications

Comparison of Online Tests of Very Short Answer versus Single Best Answers for Medical Students in a Pharmacology Course over One Year

Comparison of Online Tests of Very Short Answer versus Single Best Answers for Medical Students in a Pharmacology Course over One Year

Knowledge self-monitoring, efficiency, and determinants of self-confidence statement in multiple choice questions in medical students

Using Precourse Formative Written Testing in a Pharmacology Class Greatly Increases Medical Students’ Performance in Final Written Summative Tests

Contact Info

Product

Resources

About