Performance-Based Assessment in Child and Adolescent Psychiatry Residency Training

Abstract

Objective This study aimed to examine the application of the Objective Structured Clinical Examination (OSCE) to the assessment of competency among child and adolescent psychiatry (CAP) residents and to analyze the feedback from the residents and the examiners. Methods The OSCE was administered to 53 CAP residents based on three seniority levels over a 14-year period. The results of 147 OSCEs applied to residents and the feedback received were evaluated. OSCE scores were calculated based on the scores given by the examiners and standardized patients (SPs). Results Examiners' communication skills scores were significantly higher than examiners' task performance scores but were not significantly different than the SPs' scores. Intraclass correlation coefficients indicated that examiners and SPs were very consistent in their assessments among themselves. The scores given by the examiners and the SPs were not different between genders except for female residents' communication skills scores given by SPs in the OSCE-senior. With regard to the feedback on the OSCE, it was determined that examiners gave significantly higher scores than residents on every item except for "neutrality of the examiners." Conclusions A standard OSCE including different station types was structured to assess the progressive clinical skills of residents over the years. Using the OSCE contributed to CAP residency training far beyond assessment, creating a useful educational experience for both the trainers and the residents. Despite the challenge experienced related to SPs, the OSCE was found to be useful in improving training programs.

Child and Adolescent Psychiatry: New Concepts and New Strategies for the Future

“…This last essay by Agrawal [24] and all of the other papers in this special collection [1,[8][9][10][11][12][13][14][15][16][17][18][20][21][22][23] remind us that child and adolescent psychiatry training is not just of interest to child and adolescent psychiatrists. All psychiatrists, physicians, and most other professionals serve in roles that influence, whether directly or indirectly, the mental health of young people and adults who have had chronic conditions that began in youth.…”

“…As training must cover an ever-expanding scope of curricular topics and clinical skills, and especially with models that either shorten or do not expand the current length of training, competency-based assessment tools will become increasingly relevant. Aras and Demirgören [23] described a 14-year positive experience from the Dokuz Eylül University Faculty of Medicine in Turkey with the use of an objective structured clinical examination in child and adolescent psychiatry resident training.…”

Applications of Clinical Informatics to Child Mental Health Care: a Call to Action to Bridge Practice and Training

“…In this issue, Aras [43] discusses the use of objective structured clinical examinations in measuring child psychiatry trainees' competency in communicating with children and families, highlighting the evolving challenge to assess live clinical performance. In practice, trainees are encouraged to learn the skill of "synchronous documentation," involving reviewing health records, writing clinical documentation, entering orders, resolving decision support prompts, and billing workflows, all while interfacing with the child and family inperson or through telehealth.…”

Objective Structured Clinical Examination: Underused Tool in Epilepsy Fellowship Programs?

“…2 The main drawback is that OSCEs have a greater associated burden in terms of cost, space requirements, and examiner time commitment when compared to other examination modalities. 2 In addition to undergraduate medical education, OSCEs have also been shown as an effective assessment and educational tool in many other training contexts, including medical residency (both medical and surgical), [3][4][5][6] fellowship, 7,8 nursing, 9 pharmacy, 10 and dentistry. 11 While OSCEs have classically been done in person, the COVID-19 pandemic led to a transition to virtual OSCEs, achieved with the aid of teleconferencing applications.…”