Using Realist Evaluation to Understand Process Outcomes in a COVID-19-Impacted Yoga Intervention Trial: A Worked Example

Abstract

Realist evaluation offers a valuable way to understand how interventions function and thus how they can be improved and locally adapted. Consequently, realist evaluation is increasingly conducted in parallel with intervention trials. It comprises a clear philosophical foundation and view of causality, pragmatic mixed data collection methods, and a theory-driven approach in which hypothesised program theories are tested and refined. However, detailed methods for data analysis are seldom well-described in realist studies and no clear method for analysing and presenting realist evaluation data has yet emerged. In this methodological paper we use the worked example of our realist process evaluation of the SAGE yoga trial to illustrate an applied process of data analysis and presentation of findings. We show how we drew on other realist studies for ideas, provide examples of six key tasks involved in conducting a realist process evaluation (including coding data and structuring results) and describe strategies that did not work and our rationale for rejecting them. This detailed account of the decisions and methods that worked for us is intended to provide a practical and informed point of departure for researchers conducting a realist evaluation.

Do the implementation processes of a school-based daily physical activity (DPA) program vary according to the socioeconomic context of the schools? a realist evaluation of the Active at school program

“…These initial theories generally combine elements of substantive theories with stakeholders’ assumptions about how and why the program may work, derived from their research and/or professional expertise. These realist theories are used to reveal the underlying logic of programs and are then tested and refined empirically through data collection and analysis [ 23 ]. A central tenet of the realist approach is that programs work differently in different contexts.…”

“I wouldn’t have joined if it wasn’t online”: understanding older people’s engagement with teleyoga classes for fall prevention

“…Audio recordings from focus groups and interviews were transcribed verbatim and uploaded to NVivo [53] where they were coded deductively using the framework of realist evaluation concepts and program theories developed in phase 1 of the process evaluation [52]. Eight inductive subcodes were introduced to capture new emphases in the data.…”

“…In phase one of the process evaluation we used a realist evaluation methodology to explore what worked for whom in the different program formats offered by the hybrid SAGE program. This is detailed in our methodology paper [52] but, in summary, it involved data collection across several sources, including interviews with 21 purposively sampled participants and three SAGE yoga instructors, and 46 participant feedback forms. We started by identifying the proximal impacts or process outcomes that we wanted to explain (such as strong adherence and positive self-reported impacts identified in feedback forms), and developed initial program theories to explain these.…”

‘My words become my hands’: Yoga instructors’ experiences of adapting teleyoga in the SAGE fall prevention trial—A qualitative analysis

“…Our process evaluation was conducted in two phases and used a realist evaluation methodology to explore what worked for whom in the different programme formats of the SAGE programme (i.e., hybrid and teleyoga). Our methodology and subsequent theory development have been detailed elsewhere 25 , 44 and involved data collection across several sources and time points. The process evaluation was led by two experienced (more than 5 years), PhD qualified, female qualitative health researchers (AH and HG).…”