If We Measure It, They Will Come: A Realist Evaluation Approach in a Therapeutic Jurisprudence Context

Abstract

Realist evaluation asks what works best for whom under what circumstances and why. Therapeutic jurisprudence seeks to explore how the therapeutic design and application of the law can support therapeutic and anti-therapeutic outcomes. This article presents a proposed evaluation framework utilizing realist evaluation in a therapeutic jurisprudence context for family court interventions. Three key topics that make up the framework are described: mediation and moderation to document what works best for whom under what circumstances, analytical strategies to achieve good enough measurement of relevant constructs in the theory, and context-mechanism-outcome pattern hypotheses and generative causation used in realist evaluation. Key PointsThe realist evaluation approach asks, "what works best for home under what circumstances and why" and is proposed as the foundation for meaningful evaluation of family court interventions.Therapeutic jurisprudence, with its emphasis on therapeutic application of the law, and how legal actors can impact therapeutic and anti-therapeutic outcomes, fits well within this realist evaluation framework, and can enrich how interventions are conceptualized and measured. Most family court interventions (and most interventions generally) involve complex interactions within varied con-texts that can impact their success or failure. The context-mechanism-outcome pattern hypotheses that are the foundation of realist evaluation incorporate this complexity and allow the important simultaneous measurement of process and outcome patterns. Mediation gets at "how" an intervention works, while moderation gets at "when" and "for whom". Building mediation and moderation variables into the context-mechanism-outcome hypotheses is a way to incorporate the complexity found in family court interventions and fits well within the realist evaluation approach.Measurements used in programs are never perfect but can be "good enough". Designing a "good enough" way to measure the key components of a family court intervention should be seen as a three to five year multi-study process, rather than one study at a time.The combined expertise of family court professionals and evaluation researchers is essential to designing and testing reliable, valid, and relevant ways to measure family court program mechanisms and outcomes that will ultimately be utilized in practice to measure the success of interventions, but also to identify when and for whom interventions need to be improved.

Searching for context: a review of “what works” reviews of interventions to prevent youth offending using the EMMIE Framework

“…12 For example, Petrucci (2021) illustrates that an intervention’s success or failure can involve complex interactions within varied contexts, so the context-mechanism-outcome hypotheses of realist evaluation can incorporate such complexity and allow for process and outcome patterns to be measured simultaneously, enriching the conceptualisation and measurement of intervention.…”

“…Successionism examines the causal relationship between dependent and independent variables, such that observing these causal associations between (in/dependent) variables becomes a closed system that rules out alternative explanations and experimental approaches become the gold standard.12 For example,Petrucci (2021) illustrates that an intervention's success or failure can involve complex interactions within varied contexts, so the context-mechanism-outcome hypotheses of realist evaluation can incorporate such complexity and allow for process and outcome patterns to be measured simultaneously, enriching the conceptualisation and measurement of intervention.…”