The OSCE in a pre-registration osteopathy program: Introduction and psychometric properties

Abstract

ObjectiveTo investigate the psychometric properties of the objective structured clinical examination (OSCE) conducted in the final year of a pre-professional osteopathy program. A variety of metrics are used to determine the reliability of the examination. This is the first study to describe an OSCE in a non-USA osteopathy program. MethodsData from the OSCE conducted in 2011 was collated and analysed to establish the pass/fail rates, cost of the examination, internal consistency, and inter-examiner reliability. The examination was conducted over two days with students completing 5 stations on day 1 and 4 stations on day 2. Each station was of 15 minutes duration and there were 2 examiners per station. ResultsForty-eight students and 31 examiners were involved in the examination. Twenty six students failed at least one station with six students failing three or more stations.Cronbach's alpha was greater than 0.80 for all stations indicating that each is internally consistent and over 50% of the variance in the students' total score for a station was due to the students themselves. Inter-examiner reliability was poor to fair however there was good to excellent agreement for the global rating at each station. The total cost of conducting the examination was $AUD12 933.20. ConclusionThe results of the study suggest that the OSCE format is an appropriate method for assessing clinical competency in osteopathic education. The OSCE should be used in conjunction with other forms of assessment to develop and overall picture of the students' clinical competency. There are however some modifications that are required to improve the examination and these will be the subject of further studies. MethodsData from the OSCE conducted in 2011 was collated and analysed to establish the pass/fail rates, cost of the examination, internal consistency, and variance components. The examination was conducted over two days with students completing 5 stations on day 1 and 4 stations on day 2. Each station was of 15 minutes duration and there were 2 examiners per station. ResultsForty-eight students and 31 examiners were involved in the examination. Twenty-six students failed at least one station with six students failing three or more stations. Cronbach's alpha was greater than 0.80 for all stations indicating that each is internally consistent and over 50% of the variance in the students' total score for a station was due to the students themselves. The total cost of conducting the examination was $AUD12 933.20. ConclusionThe results of the study suggest that the OSCE format is an appropriate method for assessing clinical competency in osteopathic education. The OSCE should be used in conjunction with other forms of assessment to develop and overall picture of the students' clinical competency.Some modifications to the format of the assessment are required to improve the examination and these will be the subject of further studies.

Clinical assessment during a global pandemic – Transitioning to a COVID safe hybrid OSCE

“…OSCEs usually involve 6-12 stations with either continuous or multiple cases to assess the domains of clinical competence in osteopathic practice. OSCEs in osteopathy have been shown to offer a psychometrically sound assessment of clinical competence [2].…”