The Hierarchical Clustering of Clinical Psychology Practicum Competencies: A Multisite Study of Supervisor Ratings

Abstract

Competency evaluation rating forms are widely used to assess a range of global and specific psychology practitioner competencies during and at the end of clinical placements. Surprisingly, there is little research examining the dimensional structure or the hierarchical clustering of items on these ratings. The current, multisite study Key words: competency assessment, field placement, halo bias, leniency bias, psychology internships, psychology practitioner competencies, supervisor evaluations, supervisor ratings. [Clin Psychol Sci Prac 22: 390-403, 2015] Field placements are a central aspect of training programs in professional psychology. The structure, duration, casework, and supervision requirements of these placements vary across programs and across countries, but multiple placements are typically required by training programs and mandated by regulatory bodies to provide a breadth of professional experiences for trainees

Brief Online Training with Standardised Vignettes Reduces Inflated Supervisor Ratings of Trainee Practitioner Competencies

“…It was hypothesised that CΨPRS ratings would be lower in the Trained compared to the Untrained group. In accordance with previous literature (Gonsalvez et al, 2015), we expected differences in ratings across domains, but we did not expect an interaction between training and domain.…”

“…This is consistent with other research demonstrating the effectiveness of training in improving rater reliability and accuracy across varied disciplines (Chafouleas et al, 2015; Jelley & Goffin, 2001; Schanche et al, 2010; Stamoulis & Hauenstein, 1993; Støre‐Valen et al, 2015; Thornton & Zorich, 1980). It may be that providing clear frames of reference indirectly attenuates vulnerability to rater biases that is commonly observed in CERFs (Dudek et al, 2005; Gonsalvez et al, 2015; Stamoulis & Hauenstein, 1993). In other words, high ratings potentially driven by rater bias can be reduced to more accurate levels by providing objective and clear criteria (via behaviourally descriptive vignettes) that are matched to a trainee's expected stage of development.…”

“…The CΨPRS used in this research is an online trainee assessment and is a revision of earlier versions of the CΨPRS (Gonsalvez et al, 2015). It is a 60‐item rating scale comprising 10 overall domain items and 50 subdomain items measuring specific skills within each of the 10 domains ( Counselling, Clinical Assessment, Case Conceptualisation, Intervention, Ethical Attitude and Behaviour, Scientist‐Practitioner Approach, Professionalism, Psychological Testing, Reflective Practice, and Response to Supervision .…”

“…To this end, there has been significant effort directed towards developing valid and reliable competency evaluation rating forms (CERFs). Typically, these assessment forms use Likert scales to rate trainee competence across a range of foundational (e.g., professional skills, ethical attitude and behaviour) and functional (e.g., case conceptualisation, intervention) domains (Fouad et al, 2009; Gonsalvez et al, 2015; Rodolfa, Bent, Eisman, Nelson, & Ritchie, 2005). …”

“…Our study takes a similar vignette‐based training approach, but in our case, supervisors rate the level of trainee competency described in the vignettes, and expert ratings are provided as feedback. The central question is whether training attenuates the high scores given using the CΨPRS, hypothesised to be driven by rater bias (Gonsalvez et al, 2015). …”