What Really Matters for Supervision Training Workshops? A Realist Evaluation

Abstract

Purpose Supervision training supports health care supervisors to perform their essential functions. Realist evaluations are increasingly popular for evaluating complex educational interventions, but no such evaluations exist appraising supervision workshops. Building on an earlier realist synthesis of supervision training, the authors evaluated whether supervision workshops work, for whom and under what circumstances, and why. Method The authors conducted a 2-stage realist evaluation during 2018–2019 to refine and develop program theory. The intervention involved half-day, face-to-face supervision workshops as part of an Australian state-wide government-funded program for health care and human services supervisors. Data collection involved realist interviews with 10 workshop developers (stage 1) and 43 supervisors (stage 2). The authors employed team-based data analysis using realist logic to refine and develop program theory by identifying contexts, mechanisms, outcomes, and context-mechanism-outcome configurations. Results Despite their brevity, the supervision workshops had many reported benefits for supervisors (e.g., improved satisfaction) through various perceived mechanisms pertaining to pedagogy (e.g., mixed pedagogies), workshops (e.g., optimal duration), and individuals (e.g., supervisor engagement). However, they also yielded negative reported outcomes (e.g., suboptimal knowledge gains) brought about by assorted perceived mechanisms related to pedagogy (e.g., suboptimal peer learning), workshops (e.g., content irrelevance), and individuals (e.g., suboptimal facilitator competence). Such mechanisms were thought to be triggered by diverse contexts including supervisors’ levels of experience, sector, and workplace supervision cultures. Conclusions While the findings partly support the realist synthesis of supervision training and previous realist evaluations of faculty development, this realist evaluation extends this literature considerably. Health care educators should employ mixed pedagogies (e.g., didactic teaching, peer learning), relevant content, optimal workshop duration, and competent/engaging facilitators. Educators also need to tailor workshops according to supervisors’ contexts including the sectors and supervision cultures in which supervision is practiced, and supervisors’ levels of experience (e.g., experienced supervisors appreciated workshop brevity).

A roadmap to realist interviews in health professions education research: Recommendations based on a critical analysis

“…Therefore, we have written this current Focus on Research Methods paper, primarily for those relatively new to realist research, based on our knowledge of the methodology literature, as well as our own experience of realist and qualitative interviewing. [10][11][12] We hope this paper helps evaluators to conduct and report realist interviews optimally, as well as reviewers and editors to critically appraise realist interviews more easily. See Box 3 for an overview of this paper's illustrative cases.…”

“…Purposive and theoretical sampling is key for realist interviews, with participants selected based on their ability to contribute to the gleaning, refining and/or consolidation of programme theory. 5,6,17,27,29 Furthermore, participant diversity is important to help shed more fulsome light on programme theory, with programme practitioners (e.g., programme designers and programme facilitators) typically being interviewed first (often for theory gleaning purposes), and then latterly programme users (e.g., programme participants) being interviewed, 5,6,29 as was the case in Nguyen et al 10 If programme theory suggests that programmes might work differently for different stakeholders (e.g., medicine versus nursing practitioners) and/or from different organisational settings (e.g., primary versus secondary care), then those sub-groups of interest should inform ongoing sampling. 5,6 No suggested minimum number of realist interviews exists.…”

“…Realist evaluation of supervision training 10,11 Realist evaluation of faculty development 12 Interview schedules (available on request)…”

“…Interestingly, nine studies also mentioned some form of thematic analysis, whereby contexts, mechanisms and outcomes were conceptualised as themes, or the steps in framework analysis (a form of thematic analysis) were employed to organise the realist configurational analyses. Finally, only five studies [10][11][12]34,40 reported meeting the reporting standards for realist evaluations. 18 In conclusion, we identified several limitations in published realist interview studies within the HPER field, meaning that reporting standards for realist evaluations were sometimes unmet.…”

“…Although qualitative research interviewing is well‐articulated in the HPER literature, 7–9 realist interviewing is not, despite its increasing use as a realist evaluation data collection method (see Table 1 later). Therefore, we have written this current Focus on Research Methods paper, primarily for those relatively new to realist research, based on our knowledge of the methodology literature, as well as our own experience of realist and qualitative interviewing 10–12 . We hope this paper helps evaluators to conduct and report realist interviews optimally, as well as reviewers and editors to critically appraise realist interviews more easily.…”