Trainee self-assessment of cognitive behaviour therapy competence during and after training

Abstract

Large-scale cognitive behavioural therapy (CBT) training and implementation programmes, such as the pioneering Improving Access to Psychological Therapies (IAPT) initiative in the UK, aim to develop a workforce of competent therapists who can deliver evidence-based interventions skilfully. Self-awareness of competence enables CBT therapists to accurately evaluate their clinical practice and determine professional development needs. The accuracy of self-assessed competence, however, remains unclear when compared with assessments conducted by markers with expertise in CBT practice and evaluation. This study investigated the relationship between self- and expert-rated competence – assessed via therapy recordings rated on the Cognitive Therapy Scale Revised (CTS-R) scale – for a large sample of IAPT CBT trainees during training and, for the first time, at post-training follow-up. CBT trainees (n = 150) submitted therapy recordings at baseline, mid-training and end-of-training. At 12+ month follow-up, a subset of former trainees (n = 30) submitted recordings from clinical practice. There were positive relationships (r = .27 to .56) between self and expert CTS-R scores at all time points. The proportion of tapes demonstrating significant agreement between self and expert ratings (CTS-R difference <5 points) increased significantly across training and remained stable at follow-up. Findings indicate that accurate self-awareness of competence can be developed during structured CBT training and retained in the workplace. These outcomes are encouraging given the importance of self-awareness to CBT practice and accreditation. Future investigation into the development and maintenance of accurate self-awareness of competence is warranted. Key learning aims (1) What is the relationship between self-ratings and expert ratings of CBT competence during training and at post-training follow-up? (2) Does agreement between self and expert competence ratings improve with CBT training? (3) How does agreement between self and expert ratings change across training for more- and less-competent trainees? (4) Can accurate self-awareness of competence be retained post-training in the workplace?

Prospective psychotherapists’ bias and accuracy in assessing their own facilitative interpersonal skills.

“…Studies showed inconsistent results regarding the relationship between therapists’ professional experience and self-assessment bias (Beale et al, 2020; Jaeken et al, 2017; Loades & Myles, 2016; Parker & Waller, 2015). Our study showed no significant association, with a tendency toward higher self-assessments for trainee psychotherapists in contrast to psychology students.…”

“…The Dunning–Kruger effect has since been confirmed in a variety of studies, which showed that mental health professionals tended to overestimate their counseling skills (measured through Skilled Counseling Scale or Cognitive Therapy Scale Revised ) before training. After training, they viewed their skills more critically, which led to accurate estimations (i.e., closer to observer-rated levels of competence) or underestimation of skills (Beale et al, 2020; Lepkowski et al, 2009; Little et al, 2005; Urbani et al, 2002). In a study by Jaeken et al (2017), not all trainees overestimated their skills before training.…”

Use of self-practice/self-reflection (SP/SR) exercises for competency-based training and assessment in CBT

“…This is supported by research findings that those objectively rated as the least competent practitioners may demonstrate the greatest over-estimation, as they lack awareness of their own deficiencies (Brosan et al, 2008;Dunning et al, 2003). Consequently, this may result in the clinician failing to recognise a need for training or supervision to improve their practice, and the delivery of less effective or even harmful therapy to their clients (Beale et al, 2020;Brosan et al, 2008;McManus et al, 2012). For less experienced trainees the self-assessment of competency has been more variable.…”

“…With the wide dissemination of cognitive behavioural therapy (CBT) training, especially with the advent of stepped care initiatives, finding methods for competency-based training is important to ensure delivery of effective services (Beale et al, 2020). While there is considerable research on the theory and application of CBT to the treatment of mental health difficulties (Butler et al, 2006;David and Cristea, 2018), there has, until recently, been little research on how training methods relate to the development of trainees' competence in CBT (Bennett-Levy and Lee, 2014).…”

The why, what, when, who and how of assessing CBT competence to support lifelong learning

“…Self-assessment is a core aspect of reflective practice that allows therapists to manage their own learning and facilitates identification of professional development needs. The ability to accurately self-assess competence is also a meta-competence that can be improved with training and supervisory feedback (Beale et al, 2020;Brosan et al, 2008;Loades and Myles, 2016). Self-assessing competence by rating performance within therapy or role-play sessions may be a useful selfdevelopment strategy, especially when used in combination with objective assessor or supervisory ratings.…”