Impact of examiners on candidate scores: An introduction to the use of multifacet Rasch model analysis for oral examinations

Oral assessment and postgraduate medical examinations: establishing conditions for validity, reliability and fairness

“…Interaction represents one of oral assessment's main strengths since it allows examiners to probe a candidate's reasoning, ethics and level of knowledge (Lunz and Stahl 1993). However, interaction also highlights the nature of oral assessment as an interpersonal event; it gives rise to the possibility that the social interaction entailed in this form of assessment may distort communication and affect both a candidate's performance and how that performance is perceived by the examiners (Abrahamson 1983).…”

“…The two principal issues regarding content in oral assessment are (i) to ensure that oral assessment is used to assess those aspects of a candidate's performance that are best assessed using this medium (Levine and McGuire 1970;Lunz and Stahl 1993;Muzzin and Hart 1985) and (ii) that there is the utmost clarity regarding the role of the candidate's communicative ability in the process. Where communication and/or language ability is not being assessed, it must not be allowed to influence the examiners' judgement.…”

'Home' Improvement: The Task for Measurement Specialists

“…Although these theories have not been popular, the many-facet Rasch model (Linacre, 1989) is one IRT model that should be explored further in physical education and exercise science. It is currently being used to solve measurement problems associated with performance assessment in occupational therapy (Fisher, 1994), medicine (Lunz & Stahl, 1993), and industry (Rehfeldt, 1994). An overview of this model will follow, so its application potential in physical education and exercise science may be understood more fully.…”

The Presence and Impact of Local Item Dependence on Objective Structured Clinical Examinations Scores and the Potential Use of the Polytomous, Many-Facet Rasch Model

“…9 Interest in OSCEs in the medical education field increased as research on oral examinations revealed considerable concern regarding low reliability and validity of oral scores. [10][11][12] In general, the OSCE has candidates rotate through a series of stations where performance skills are assessed. The OSCE methods of assessment can include clinical observation, use of standardized patients (SPs), oral interactions, and written components.…”

“…Inconsistency of raters on oral examinations has been studied extensively. 2,10,[18][19][20][21][22] In 1993, Lunz and Stahl 10 reviewed an oral examination administered by the American Board of Urology. They found that examinees with identical raw scores often had differing ability measures after removing the effects because of rater stringency/leniency and case difficulty.…”