2009
DOI: 10.1111/j.1365-2923.2009.03438.x
|View full text |Cite
|
Sign up to set email alerts
|

The effect of differential rater function over time (DRIFT) on objective structured clinical examination ratings

Abstract: Systematic biases, such as DRIFT, may compromise internal validity in an OSCE. Further work is needed to confirm this finding and to explore whether DRIFT also affects ratings on summative OSCEs. If confirmed, the factors contributing to DRIFT, and ways to reduce these, should then be explored.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
3
1
1

Citation Types

1
33
1

Year Published

2012
2012
2024
2024

Publication Types

Select...
8

Relationship

0
8

Authors

Journals

citations
Cited by 46 publications
(35 citation statements)
references
References 4 publications
1
33
1
Order By: Relevance
“…Numerous studies have documented the fact examination bias is significantly reduced in a welldesigned OSCE [20,23] However, some researchers believe that OSCEs are vulnerable to systematic biases due to examiner fatigue, incorporating frequent rest periods for the examiners and involving multiple examiners to evaluate the examinee's performance based on a standard checklist could reduce the possibility of such errors [36,37]. In this study, the majority of the faculty members (70%) felt that they could stay focused while observing the same case repeatedly.…”
Section: Resultsmentioning
confidence: 99%
“…Numerous studies have documented the fact examination bias is significantly reduced in a welldesigned OSCE [20,23] However, some researchers believe that OSCEs are vulnerable to systematic biases due to examiner fatigue, incorporating frequent rest periods for the examiners and involving multiple examiners to evaluate the examinee's performance based on a standard checklist could reduce the possibility of such errors [36,37]. In this study, the majority of the faculty members (70%) felt that they could stay focused while observing the same case repeatedly.…”
Section: Resultsmentioning
confidence: 99%
“…We contend that there are a number of reasons why the aforementioned assumptions ought to be challenged: 1) examiners are typically not randomised into cycles, but are allocated chiefly based on geography and convenience; 2) examiners in different localities may develop different tacit performance norms, different shared expectations or hidden curricula (Hafferty & Franks, 1994); or different standards of practice (Weiner et al, 1995), which are known to inversely relate to assessors' severity of judgements (Kogan, Hess, Conforti, & Holmboe, 2010); 3) timing of cycles may also influence the standard of judgement (Hope & Cameron, 2015;McLaughlin et al, 2009). …”
Section: Many Facet Rasch Model (Mfrm)mentioning
confidence: 99%
“…Moreover, adjusting for examiner variance altered the pass/fail decisions of 11% of students in their study. Other studies have suggested that examiners' scores may be biased depending on the timing of OSCEs (Hope & Cameron, 2015;McLaughlin, Ainslie, Coderre, Wright, & Violato, 2009), the performance of other candidates (Yeates, Moreau, & Eva, 2015) or by different geographical locations (Sebok, Roy, Klinger, & De Champlain, 2015). Consequently, it is not sufficient to simply conduct an OSCE, and believe that the resulting scores are a fair representation of students' performance given the known influences of construct irrelevant variance.…”
Section: Introductionmentioning
confidence: 99%
“…1 A study of undergraduate medical OSCE students found no evidence that the duration of examining in a communication OSCE station influenced examiners and the marks they awarded. 2 However, McLaughlin et al 3 reported that the point of entry to an OSCE circuit was significantly associated with scoring and could be a factor that may compromise the reliability of the marks awarded and the internal validity of an OSCE examination. Removing the first 2 stations from a candidates' final scoring in an attempt to eliminate examiner "warm up" did not influence this socalled differential rating over time, an effect that might be due to examiner fatigue as the OSCE continues.…”
Section: Introductionmentioning
confidence: 99%
“…Removing the first 2 stations from a candidates' final scoring in an attempt to eliminate examiner "warm up" did not influence this socalled differential rating over time, an effect that might be due to examiner fatigue as the OSCE continues. 3 It is well known that examiner stringency or leniency, colloquially known as "hawk" and "dove" behavior, might also influence the mark awarded in any particular OSCE station. [4][5][6][7] This can be minimized by pairing of examiners in those examinations or stations that are dual manned.…”
Section: Introductionmentioning
confidence: 99%