An improved competence rating scale for CBT Supervision: Short-SAGE

Reiser, Robert P.; Cliffe, Tom; Milne, Derek

doi:10.1017/s1754470x18000065

Cited by 16 publications

(26 citation statements)

References 37 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study examined the feasibility of the MMRSS as a valid and easy to use supervision tool, and preliminary results indicated this tool is psychometrically sound and clinically useful. Future studies are needed to evaluate this tool in a range of supervisory settings including testing the psychometric properties in comparison with other newly developed tools such as short-SAGE (Reiser et al, 2018) and to examine the ability of the MMRSS to detect changes in skills throughout supervision.…”

Section: Discussionmentioning

confidence: 99%

A new tool for rating cognitive behavioural supervision – preliminary findings in a clinical setting

Møller

Rosenberg

Hvenegaard

et al. 2020

tCBT

View full text Add to dashboard Cite

Cognitive behavioural therapy (CBT) is an evidence-based psychotherapy and one of the most widely used treatments for mental health problems. It is generally acknowledged that supervision improves the quality of treatment although systematic descriptions and empirical evaluation of supervision have been sparse. Moreover, there are relatively few valid and reliable instruments to evaluate supervision. Based on a comprehensive review of the supervision literature, six competency domains were identified to cover the scope of CBT supervision: Theory, Focus, Learning strategy, Techniques, Structure, and Interpersonal style. The Moeller, Moerch, Rosenberg Supervision Scale (MMRSS) was developed to evaluate supervisor performance within each of these domains after observation of supervision. The present study examined the psychometric properties of the MMRSS (inter-rater reliability and construct validity), the clinical utility, and satisfaction when using MMRSS to evaluate CBT supervision. CBT supervisors (n = 8) were recruited for the study and provided videos of group supervision. A total of 21 videos were rated using the MMRSS and the Supervisory Competency Scale (SCS) by two independent raters. Supervisees and supervisors completed a satisfaction questionnaire to capture their experience of using the MMRSS during supervision of supervision. The MMRSS showed acceptable internal consistency and validity. Several domains in MMRSS (Structure, Learning strategy, and Interpersonal style) correlated significantly with the corresponding domains in the SCS for cognitive supervision. Preliminary results indicate that the MMRSS may be a valid and clinically useful tool to evaluate CBT supervision, although further systematic evaluation is needed. Key learning aims (1) To understand that empirically founded evaluation of cognitive behavioural supervision is essential for good training. (2) To argue that a modern view of supervision places an emphasis on learning principles. (3) To describe the Moeller, Moerch, Rosenberg Supervision Scale (MMRSS) and the scale’s preliminary psychometric properties. (4) To describe the supervisors’ and supervisees’ reported satisfaction using the MMRSS.

show abstract

Section: Discussionmentioning

confidence: 99%

A new tool for rating cognitive behavioural supervision – preliminary findings in a clinical setting

Møller

Rosenberg

Hvenegaard

et al. 2020

tCBT

View full text Add to dashboard Cite

show abstract

“…SAGE teaching is included in the Generic Supervision course, and the Clinical Supervision Workstream regularly hosts SAGE Masterclasses, facilitated by experts such as one of the SAGE authors, Derek Milne. The NESSST-CBT materials have recently been revised to include the 'Short SAGE', with items reduced from 22 to 14 to facilitate easier use by clinicians (Reiser et al, 2018). Alongside this, NES have introduced clearer, briefer and more consistent tracking of delegates to aid the measuring of 'Behaviour' (Kirkpatrick Level 3) rather than confidence, as a more robust measure of impact (Eccles and Mittman, 2006).…”

Section: Discussionmentioning

confidence: 99%

“…The CBT Supervision Manual requires to be periodically updated in order to be consistent with current clinical supervision research and practice. The Manual was last updated in early 2019, as mentioned to include the 14-item Short SAGE (Reiser et al, 2018), as well as the REACTS Supervisee Feedback measure (Milne et al, 2012), in addition to including the REACTS measure (Milne et al, 2012) instead of the TARS (Davis et al, 1989) for delegate feedback. There has been recent substantial investment and development in NES digital services.…”

Section: Future Directionsmentioning

confidence: 99%

Improving CBT supervision. Four years of implementing NES Specialist Supervision Training for CBT in Scotland

O’Neill¹,

Albiston

Ferguson

et al. 2020

tCBT

View full text Add to dashboard Cite

NHS Education for Scotland (NES) plays a lead role in training the NHS Psychological Therapies workforce across Scotland. Ferguson et al. (2016) outlined the challenges, opportunities and proposed evaluation of the NES Specialist Supervision Training in Cognitive Behavioural Therapy (NESSST-CBT). The aims of the training were to provide an evidence-based, flexible and learner-focused training in CBT specific supervision competencies. This paper will provide an update on the evaluation of the training using Kirkpatrick’s Impact Evaluation Model (1967, 1987). Results indicate that: (1) delegates rated the training experience positively in various ways; (2) delegates described increases in their confidence and competence in using structured measures of CBT and supervision; (3) a majority of delegates completing a 3-month follow-up questionnaire described continued use of a structured CBT measure in supervision and for self-reflection; and (4) 392 psychological therapists in Scotland have now been formally trained in CBT specific supervision skills. NESSST-CBT continues to adapt and improve as a resource for staff as NES moves forward in its Digital Strategy for Scotland’s NHS and partnership staff. Further implications of this are discussed, as well as limitations of the study. Key learning aims (1) Readers will be able to further understand the multi-faceted role of NHS Education for Scotland in implementing CBT supervision training in Scotland. (2) Readers will be able to list three key outcomes from the feedback data on 4 years of a specialist supervision blended-learning training for CBT supervision. (3) Readers will be able to identify three key limitations of the study and recommendations for future research.

show abstract

“…Cognitive Behaviour Therapy, 47, 206-228. Reiser, R. P., Cliffe, T., & Milne, D. L. (2018). An improved competence rating scale for CBT Supervision: Short-SAGE.…”

Section: Discussionmentioning

confidence: 99%

“…As clinical supervision is difficult to reliably assess, the authors suggest a minimum of three to four supervision sessions in order to receive a reasonable assessment of competence. Recently, SAGE has been refined into a shorter 14-item version ( Short –SAGE), which has undergone some preliminary psychometric evaluation supporting its usability (Reiser et al, 2018). However, rating therapist or supervisor competence is challenging, and poor inter-rater reliability is a common problem in observational coding instruments.…”

Section: Introductionmentioning

confidence: 99%

Can we agree on the quality of clinical supervision? Inter-rater reliability of the Short–SAGE (Supervision: Adherence and Guidance Evaluation) scale

Beckman

Spännargård

Alfonsson

2020

tCBT

View full text Add to dashboard Cite

Clinical supervision is a cornerstone in psychotherapist training, but research in this area is hampered by a lack of validated tools for assessing supervision quality. Short–SAGE (Supervision: Adherence and Guidance Evaluation) is an observational instrument designed for evaluating supervision in cognitive behavioural therapy. The aim of this study was to evaluate the inter-rater reliability of Short–SAGE. Four experienced clinical psychologists participated in three 3-hour Short–SAGE coding training sessions, followed by an additional meeting and coding instructions. In a cross-over design, codings of 20 supervision sessions were then assessed with intraclass correlations (ICC), for both the 3- and 7-point scales of the instrument. In the single measure analyses for both scales, only one item showed ICC in the good range, and the rest of the 14 item ICCs were in the poor to fair range. Moreover, on the 3-point scale, five of the 14 inter-rater correlations were non-significant. For research and training purposes, validated tools to assess supervision quality are highly needed. However, instruments for measuring adherence and/or competence are of little value if the coders do not attain inter-rater reliability. Whether quality of supervision is associated with improvements in supervisees’ competencies is not yet clear. Short–SAGE provides a tool that may enable empirical research in this area. Further studies are needed to assess whether extensive training can improve the inter-rater reliability of Short–SAGE. Key learning aims (1) Readers will be aware of the urgent need for validated tools to assess clinical supervision quality. (2) Readers will be familiar with some existing tools for assessing the quality of clinical supervision. (3) Readers will be able to identify common problems in the development of instruments for assessing clinical supervision.

show abstract

An improved competence rating scale for CBT Supervision: Short-SAGE

Cited by 16 publications

References 37 publications

A new tool for rating cognitive behavioural supervision – preliminary findings in a clinical setting

A new tool for rating cognitive behavioural supervision – preliminary findings in a clinical setting

Improving CBT supervision. Four years of implementing NES Specialist Supervision Training for CBT in Scotland

Can we agree on the quality of clinical supervision? Inter-rater reliability of the Short–SAGE (Supervision: Adherence and Guidance Evaluation) scale

Contact Info

Product

Resources

About