The Troubled Families Programme: Learning about policy impact through realist case study research

Abstract

This article aims to critically explore how qualitative case study research that is founded on realist principles can fundamentally enhance social policy evaluation methodologies and, in turn, provide improved learning for policy makers and practitioners. We suggest these methodological advantages are accrued through the careful construction of theory-based explanations of 'how' policy programmes work thereby addressing the limitations of quasi-experimental methods -namely a focus on and prioritisation of outcome measures. The paper situates this key argument within wider, long-standing debates about evidence-based policy-making and what constitutes 'evidence' of impact in social policy. It does so through reflection on the contentious and contradictory knowledge claims that surround the Troubled Families Programme and evaluative claims regarding its efficacy. In conclusion and looking forward, we suggest that there remains much scope to combine 'intensive' qualitative case studies with 'extensive' quantitative measures within local and national evaluations of complex, multi-dimensional social policies, such as the Troubled Families Programme.

Evaluating social policies: Do methodological approaches determine the policy impact?

“…The agency also uses its own evidence hierarchy to judge the quality of current studies, where RCTs are considered the best evidence available, followed by multivariate analyses with controls for observed differences and effect evaluations based on individual panel data. Similar is found in the UK, where randomized control trials are regarded as the gold standard in several official evaluations in the field of social policy (Parr & Churchill, 2019).…”

Social policy and embedded evaluation: Assessing the impact of a food insecurity project in the United Kingdom

“…The government claimed that the intervention, which initially targeted social support for 120,000 families living on low incomes and with complex problems, had a 100% success rate (DCLG, 2015). However, the independent evaluation suggested that the impact in terms of increasing employment, improving health outcomes, and reducing antisocial behaviour was limited and questioned the robustness of the evaluation design and the data collected (Day et al, 2016; Parr & Churchill, 2019). It is also notable that one of the key factors identified in the collapse of the high‐profile United Kingdom children's charity Kids Company , which aimed to help children living in poverty, was a lack of robust internal or independent evaluation evidence to assess the impact and cost effectiveness of the outcomes of education and employment interventions (House of Commons, 2016).…”

Intensive intervention with families experiencing multiple and complex challenges: An alternative to child removal in a bi‐ and multi‐cultural context?

“…In the short term, they are likely to require additional resources, and assessing cost-effectiveness can only take place over time because savings occur with reduced duplication of services and reduced dependence on services longterm. Recognizing a full range of potential outcomes, including 'soft' outcomes such as increases in families' subjective experience of wellbeing and increased engagement with services, has been highlighted (Batty, 2014;Parr & Churchill, 2019). A more nuanced outcome framework, distinguishing between transformative change, stabilizing, and crisis management, to allow for the recognition of significant results that fall short of transformative change is recommended (Batty & Flint, 2012).…”