Reliability of the OSCE for Physical and Occupational Therapists

Abstract

[Purpose] To examine agreement rates between faculty members and clinical supervisors as OSCE examiners. [Subjects] The study subjects were involved physical and occupational therapists working in clinical environments for 1 to 5 years after graduating from training schools as OSCE examinees, and a physical or occupational therapy faculty member and a clinical supervisor as examiners. Another clinical supervisor acted as a simulated patient. [Methods] The agreement rate between the examiners for each OSCE item was calculated based on Cohen’s kappa coefficient to confirm inter-rater reliability. [Results] The agreement rates for the behavioral aspects of the items were higher in the second than in the first examination. Similar increases were also observed in the agreement rates for the technical aspects until the initiation of each activity; however, the rates decreased during the middle to terminal stages of continuous movements. [Conclusion] The results may reflect the recent implementation of measures for the integration of therapist education in training schools and clinical training facilities.

Continuing competence assessment and maintenance in occupational therapy: Scoping review with stakeholder consultation

“…The number of participants ranged from five (Tan Torres, ) to 391 (Andersen, , ). Two sources did not state the sample size (Sakurai et al ., , b). Professional self‐identification included occupational therapy practitioners (a total of 2030) and other allied health professionals (174, including physical therapy practitioners).…”

“…Five studies focussed on the development of three different tools for assessment of continuing competence: the Objective Structured Clinical Examination (OSCE) ( n = 2), Chart‐Stimulated Recall tool (CSR) ( n = 2), and Multisource Feedback (MSF) system ( n = 1). During the OSCE, participants’ clinical skills are assessed as they rotate through stations, perform specific tasks with simulated patients, and receive ratings by university faculty and other clinicians (Sakurai et al ., , b). The CSR evaluation method combines interview with the occupational therapy practitioner, an audit of randomly selected charts, and a rating of the practitioner's ‘performance on a variety of items that reflect various components of competence, e.g.…”

Metodologias Ativas em Fisioterapia: Estudo de Confiabilidade Interexaminador do Método Osce

“…Os índices de confiabilidade interexaminador alcançados neste estudo concordam com os de outros autores 1,23,24 e reafirmam que o Osce é um método com reprodutibilidade de satisfatória a excelente para a avaliação em situações práticas. Sakurai e colaboradores 13 examinaram a concordância interexaminador do Osce ao avaliarem estudantes de graduação e profissionais de Fisioterapia e Terapia Ocupacional no Japão, e revelaram alta reprodutibilidade na avaliação dos aspectos comportamentais e técnicos dos examinados. Observou-se uma discrepância entre a nota final da estação 1 e as demais estações.…”

“…O Osce é parte integrante do processo avaliativo de muitos programas de residência médica no exterior e de exames de certificação profissional 10,11 . Embora muito se discuta na literatura a respeito desta forma de avaliação, pouco se sabe sobre sua aplicabilidade entre acadêmicos de Fisioterapia, sendo as evidências de reprodutibilidade interexaminador ainda incipientes 12,13 . Desta forma, o objetivo deste trabalho é relatar a experiência da aplicação do Osce nas disciplinas da área de atenção fisioterapêutica do aparelho locomotor, do curso de Fisioterapia da Faculdade de Ciências da Saúde do Trairí, da Universidade Federal do Rio Grande do Norte (Facisa/UFRN), e demonstrar a confiabilidade interexaminador dos instrumentos de avaliação da simulação da prática fisioterapêutica.…”

OSCE-based Clinical Skill Education for Physical and Occupational Therapists

“…Compared with conventional written examinations, the OSCE enables examiners to assess clinical skills in the psychomotor, emotional, and cognitive domains, and clarify points to improve7 ) . Saito et al.…”