Implementation Climate and Time Predict Intensity of Supervision Content Related to Evidence Based Treatment

Abstract

Objective: Children infrequently receive evidence-based treatments (EBTs) for mental health problems due to a science-to-practice implementation gap. Workplace-based clinical supervision, in which supervisors provide oversight, feedback, and training on clinical practice, may be a method to support EBT implementation. Our prior research suggests that the intensity of supervisory focus on EBT (i.e., thoroughness of coverage) during workplace-based supervision varies. This study explores predictors of supervisory EBT intensity.Methods: Participants were twenty-eight supervisors and 70 clinician supervisees. They completed a baseline survey, and audio recorded supervision sessions over 1 year. Four hundred and thirty eight recordings were coded for supervision content. We chose to explore predictors of two EBT content elements due to their strong evidence for effectiveness and sufficient variance to permit testing. These included a treatment technique (“exposure”) and a method to structure treatment (“assessment”). We also explored predictors of non-EBT content (“other topics”). Mixed-effects models explored predictors at organizational/supervisor, clinician, and session levels.Results: Positive implementation climate predicted greater intensity of EBT content coverage for assessment (coefficient = 0.82, p = 0.004) and exposure (coefficient = 0.87, p = 0.001). Intensity of exposure coverage was also predicted by more time spent discussing each case (coefficient = 0.04, p < 0.001). Predictors of greater non-EBT content coverage included longer duration of supervision sessions (coefficient = 0.05, p < 0.001) and lower levels of supervisor EBT knowledge (coefficient = −0.17, p = 0.013). No other supervisor- or clinician-level variables were significant predictors in the mixed effects models.Conclusion: This was the first study to explore multi-level predictors of objectively coded workplace-based supervision content. Results suggest that organizations that expect, support and reward EBT are more likely to have greater intensity of EBT supervision coverage, which in turn may positively impact clinician EBT fidelity and client outcomes. There was evidence that supervisor knowledge of the EBT contributes to greater coverage, although robust supervisor and clinician factors that drive supervision are yet to be identified. Findings highlight the potential effectiveness of implementation strategies that simultaneously address organizational implementation climate and supervisor practices. More research is needed to identify mechanisms that support integration of EBT into supervision.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

“…Across both samples, scores on the EBCSS subscales demonstrated acceptable internal consistency and evidence of structural validity. Construct validity evidence was generated in aim 1 by showing that scores on the EBCSS subscales were higher in agencies with higher levels of EBP implementation climate, an outcome supported by theory and prior research [28]. Aim 2 provided construct validity evidence.…”

“…The ICS assesses the extent to which clinicians share perceptions that they are expected, supported, and rewarded to use EBP in their clinical work with clients. Scores on the ICS have demonstrated excellent reliability and evidence of construct-related validity [49][50][51][52], including positive associations with EBP-related content in clinical supervision [28]. Items were rated on a Likert-type scale from 0 ("not at all") to 4 ("a very great extent").…”

“…Coderrated observational measures, such as the Supervision Process Observational Coding System [12], can be considered gold-standard measures with strong evidence of reliability and validity [12,23]; however, the requirements of coding audio-recorded sessions using trained raters (a rare practice outside of training clinics or clinical trials) significantly limits their pragmatism [24,25]. Measures that rely on clinician or supervisor report are more feasible [26][27][28][29]; however, available measures are either too narrow, focusing in great depth on only a single clinical intervention, or too broad, assessing only the duration, format, and general functions of supervision (e.g., crisis assessment) rather than the use of specific supervision techniques that facilitate implementation across clinical interventions. Furthermore, many measures lack strong evidence of score reliability or validity.…”

Clinical supervision approach predicts evidence-based trauma treatment delivery in children’s mental health

“…An important future direction is to explore characteristics of the setting, supervisor, clinician, and session that may influence the selection or elicitation of a particular approach. Related studies have largely failed to identify clinician or supervisor characteristics that influence practice, but have found that either the organizations’ climate toward EBTs ( 57 , 58 ) or being situated within a private vs. public agency ( 18 ) influence supervision practice. Nonetheless, numerous factors could plausibly influence supervisors’ technique use.…”

Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

“…The ICS has also shown positive correlations with implementation outcomes, such as knowledge and attitudes toward EBPs [ 11 , 17 ], intensity of supervisory focus on evidence-based treatment [ 6 ], and clinicians’ use of EBPs [ 11 ]. As measuring actual use of EBPs is challenging [ 18 ], measuring clinicians’ intention to use newly learned EBP can be a good alternative, as intentions are proposed to be determinants of behavior [ 19 – 21 ] and thus important when making a decision to adopt EBPs.…”