Rater Errors in a Clinical Skills Assessment of Medical Students

Abstract

The authors used a many-faceted Rasch measurement model to analyze rating data from a clinical skills assessment of 173 fourth-year medical students to investigate four types of rater errors: leniency, inconsistency, the halo effect, and restriction of range. Students performed six clinical tasks with 6 standardized patients (SPs) selected from a pool of 17 SPs. SPs rated the performance of each student in six skills: history taking, physical examination, interpersonal skills, communication technique, counseling skills, and physical examination etiquette. SPs showed statistically significant differences in their rating severity, indicating rater leniency error. Four SPs exhibited rating inconsistency. Four SPs restricted their ratings in high categories. Only 1 SP exhibited a halo effect. Administrators of objective structured clinical examinations should be vigilant for various types of rater errors and attempt to reduce or eliminate those errors to improve the validity of inferences based on objective structured clinical examination scores.

Gender as a Factor in the Prediction of Performance in Botswana General Certificate of Secondary Education Physical Education Examinations by Coursework and Forecast Grades among Senior Secondary School Students

“…In other words to what extent could one count on what the student has done in school, as reflected by his/her performance, in admitting him/ her to the next level of education, or in employing him/her in an area that requires the same or related skills? Iramaneerat (2007) stated that different countries use different measures to evaluate academic readiness of students. Botswana's Ministry of Education in collaboration with Botswana Examination Council recommended class or theory based sessions in addition to practice-based sessions for performance of coursework in physical education.…”

Back to the Future: Electronic Marking of Objective Structured Clinical Examinations and Admission Interviews Using an Online Management Information System in Schools of Health Sciences

“…However, current coefficients do not achieve the standards suggested in other research literature on the subject e.g. OSCEs [17,[19][20][21][22] The generalizability of results is only appropriate in OSCEs with a minimum of 15-18 stations [23][24][25].…”

Evaluation of the Consistency Between Clinical and Preclinical Observers in the Formation of the History Taking Skills of Fourth Year Medical Students of Marmara University

“…[12][13] Göz lem ci ye ait, tu tar lı lık ta et ki li ola bile cek özel lik ler den bi ri si, göz lem ci nin de ğer lendir di ği is tas yo nun ya pı lan dı rıl ma sın da al dı ğı rol dür. [12][13] Bi zim ça lış ma mız da kli nik göz lem ci ler, KUG prog ra mı göz lem ci le ri nin ha zır la dı ğı de ğer -len dir me reh be ri ne ge ri bil di rim de bu lun muş lar ve her bir mad de üze rin de uz laş ma sağ lan mış tır; ancak kli nik göz lem ci ler de ğer len dir me reh be ri nin ve do la yı sıy la tü re til di ği öğ re nim reh be ri nin ilk oluş tu rul ma aşa ma sın da bu lun ma mış lar dır. Bu durum kli nik ve KUG göz lem ci le ri ara sın da ger çek -leş ti ri len de ğer len dir me yi "sa hip len me " 12 dü zeyin de fark lı lık oluş ma sı na ve iki grup ara sın da güç -lü bir tu tar lı lı ğın olu şa ma ma sı na kat kı da bu lun muş ola bi lir.…”

“…13 Bu du rum da ge nel ola rak da ha ka tı bir tu tum ta kı nan göz lem ci ye gö re mü sa maha kar tu tum ta kı nan göz lem ci ler, da ha yük sek not ver me eği li min de ol mak ta dır lar. İki grup ara sın daki tu tar sız lık eği li mi nin ola sı bir ne de ni de ki mi eği ti ci le rin da ha mü sa ma ha kar de ğer len dir me le ri ola bi lir.…”