The oral examination: a study of academic and non-academic factors

Thomas, C. S.; Mellsop, Graham; Callender, K.; Crawshaw, Josie; Ellis, Pete; Hall, Anne; Macdonald, J. A.; Silfverskiöld, Peter; Romans, Sarah

doi:10.1111/j.1365-2923.1993.tb00297.x

Cited by 47 publications

(42 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…7 As well, several authors allude to the importance of trained examiners. 3,8 Only one faculty member at each site was a current board examiner. The remainder varied in experience in practice examinations.…”

Section: Discussionmentioning

confidence: 99%

The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good

Kearney

Puchalski

Yang

et al. 2002

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

Purpose: In response to the Royal College's request to improve the validity and reliability of oral examinations, the Examination Board in anesthesia proposed a structured oral examination format. Prior to its introduction, we studied this format in two residency programs to determine reliability of the examiners.Methods: Twenty faculty and 26 residents from two Canadian residency programs participated (Sites A and B). Pairs of examiners scored five or six residents examined consecutively on two standardized questions using global rating scales with anchored performance criteria. Residents' performances were scored independently during the examination (Time 1) and later from a videotaped recording (Time 2). Correlations between scores of the pairs of examiners and between scores of each examiner were determined.Results: Correlations demonstrating inter-rater agreement between examiners at Site A ranged from -.324 to .915 (mean .506) at Time 1. At Time 2, correlations ranged from .64 to .887 (mean .791). At Site B correlations ranged from .279 to .989 (mean .707) at Time 1 and at Time 2 correlations ranged from -.271 to .924 (mean .477). Correlations demonstrating intra-rater agreement of examiners at Site A ranged from .054 to .983 (mean .723) and at Site B correlations ranged from -.055 to .974 (mean .662).Correlations >0.4 were seen in 80% of the scores and >0.7 in 50% indicating fair to good intra-rater and inter-rater reliability using this format.Conclusions: Despite the limitations of our study our results compare favourably with those previously reported in anesthesia. We recommend the adoption of this format to the Royal College of Physicians and Surgeons of Canada Examination Board. Objectif : C'est à la demande du Collège royal, d'améliorer la validité et la fiabilité des examens oraux, que le Bureau des examinateurs en anesthésie a proposé un modèle d'examen oral structuré. Avant sa mise en application, nous l'avons testé dans deux programmes de rési-dence afin de déterminer la fiabilité des examinateurs. Méthode : Vingt facultés et 26 résidents de deux programmes canadiens ont participé à l'étude (Sites A et B). Des paires d'examinateurs ont utilisé une échelle de notation globale comportant des critères de rendement définis pour évaluer cinq ou six résidents appelés à répon-dre consécutivement à deux questions normalisées. Les résultats des résidents ont été cotés séparément pendant l'examen (Temps 1) puis, à partir d'un enregistrement vidéo (Temps 2). Les corrélations entre les scores des paires d'examinateurs et entre les scores de chaque examinateur ont été établies.Résultats : Les corrélations démontrant une concordance interexaminateurs au Site A sont 324 à 0,915 (moyenne de 0,506) au Temps 1. Au Temps 2, de 0,64 à 0,887 (moyenne de 0,791). Au Site B, elles sont de 0,279 à 0,989 (moyenne 0,707) au Temps 1,271 à 0,924 (moyenne de 0,477

show abstract

Section: Discussionmentioning

confidence: 99%

The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good

Kearney

Puchalski

Yang

et al. 2002

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

show abstract

“…Correcting for such errors would change the pass/fail decisions for about 6% of the examinees [6]. Marks awarded to candidates by different examiners indicates low reliability between ratings and agreement between examiners is often poor [7].…”

Section: Introductionmentioning

confidence: 99%

Introduction of Structured Oral Examination as A Novel Assessment tool to First Year Medical Students in Physiology

Shenwai

Patil

2013

JCDR

View full text Add to dashboard Cite

Background: The traditional viva examination which is still predominantly used in most of the medical institutions as one of the assessment methods has some demerits like lack of standardization, objectivity & reliability. The process involves many faculty members from respective departments. Hence there can be variations in the time allotted to each student, number of questions asked, and difficulty level of the questions. These can be resolved by structuring the oral examination to make it a better assessment tool. The present study was conducted to introduce structured oral examination (SOE) as a novel assessment tool to first year M.B.B.S. students in Physiology and evaluating the process by taking feedback from the students and faculty. Material and Methods:All the subjects [first year M.B.B.S. students (n=100)] were initially assessed by traditional viva. Feedback in the form of a questionnaire was collected from the students. Questionnaire included various questions based on Likert scale and numerical value for each response was decided. Topic for the structured oral examination was decided

show abstract

“…This anxiety, particularly in males, is associated with a lower oral score [7] . Therefore, the examiner's initial contact with the student should be as friendly as possible.…”

Section: Introduction Of Oral Examinationmentioning

confidence: 99%

“…The final rating should be discussed thoroughly by all examiners particularly when there are discrepancies in marking. Ideally, the final mark should equal the mean of marks given by all examiners because of the poor inter-rater reliability of marking by different oral examiners [7,11] .…”

Section: Ratingmentioning

confidence: 99%

Guidelines for the Administration of Oral Examinations: Review and Opinion

Jan¹,

Al-Buhairi²

2001

med

View full text Add to dashboard Cite

ABSTRACT. Practical oral examinations are important and are frequently a difficult part of medical school and board examinations. They represent an accurate and direct mean of assessing the student's interaction with patients and their clinical and technical skills. This paper reviews an outline for the administration of oral examinations. The review is based on the medical literature, detailed discussions with many senior examiners from different medical systems, and the author's personal experience. In summary, although the examiner's judgment is crucial, some general rules remain important for fair and consistent evaluation of students. Firstly, the examiner's attitude should be as friendly as possible and his or her objectives are to assess the student's medical competence and practical safety and does not help them pass or fails them. Secondly, scoring should be based on a model answer or well-accepted medical practices for a consistent rating. Finally, each examiner should give an independent score before discussing the final rating with the other examiners.

show abstract

The oral examination: a study of academic and non-academic factors

Cited by 47 publications

References 10 publications

The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good

The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good

Introduction of Structured Oral Examination as A Novel Assessment tool to First Year Medical Students in Physiology

Guidelines for the Administration of Oral Examinations: Review and Opinion

Contact Info

Product

Resources

About