Capturing complexity in the evaluation of a major area-based initiative in community empowerment: what can a multi-site, multi team, ethnographic approach offer?

Abstract

In recent years there has been growing emphasis on the need to develop ways of capturing 'complexity' in the evaluation of health initiatives in order to produce better evidence about 'how' and under what conditions such interventions work. Used alone, conventional methods of evaluation, that attempt to reduce phenomena intervention processes and outcomes to a small number of discrete and finite variables, are typically not well suited to this task. Among the research Therecommunity there have been increasing calls among the research community to take more seriously qualitative methods as an alternative or complementary approach toin the evaluation of health initiativesintervention evaluation. Ethnography has been identified as being particularly well suited to the purpose of capturing the full messiness that ensues when health interventions are introduced into compexcomplex settings (or systems). In this paper we draw reflect on our experience of taking a long term multi-site, multi team, ethnographic approach to captureing complex, dynamic system processes in the first phase of an evaluation of a major area-based community empowerment initiative being rolled out in 150 neighbourhoods in England. Through reflection on the practice of conducting multi-site, multi-team ethnography in the first phase of this evaluation, Wwe consider the utility of our approach for capturing the complexity inherent to understanding the changes that ensue when the initiative is delivered into multiple diverse contexts/systems as well as the opportunities and challenges that emerge in the research process.

Complexity and Community Context: Learning from the Evaluation Design of a National Community Empowerment Programme

“…The findings presented here confirm the importance of studying community ecology in evaluations of community engagement and empowerment [24]. The use of longitudinal qualitative research to examine local contexts is in keeping with other UK community empowerment evaluations [28,29,48]. The fieldwork drew on naturalistic traditions of qualitative research [38], however we recognise that incorporating ethnographic methods, with research teams working for longer times within the study communities, may have led to better more nuanced understandings of the community systems [48].…”

“…Overall, learning from this study emphasises the value of case study design in exploring the critical links between context, participants and intervention. It also supports the need for further robust qualitative studies to understand the development and impacts of community empowerment interventions [28,29,48].…”

“…That the 'system' was not a professional system with a common language of partnership made the higher levels of abstraction more difficult. The importance of managing the tension between the generation of 'thick descriptions' of community contexts and building a more general programme theory is also noted in an account of a multi-site ethnographic evaluation of an area-based community empowerment initiative [48]. The three critical elements that helped manage this tension in our study were the application of a theory of change, the centrality of case studies and keeping community (lay) perspectives central to interpretations.…”

“…The use of longitudinal qualitative research to examine local contexts is in keeping with other UK community empowerment evaluations [28,29,48]. The fieldwork drew on naturalistic traditions of qualitative research [38], however we recognise that incorporating ethnographic methods, with research teams working for longer times within the study communities, may have led to better more nuanced understandings of the community systems [48]. Although the design attempted to draw on multiple data sources to strengthen evidence, it did not include comparator (non-intervention) communities.…”

Innovations in Addressing Mental Health Needs in Humanitarian Settings: A Complexity Informed Action Research Case Study

“…Scholars and practitioners still wanting to advance a scientific basis for social and behavioral action to improve healthcare are advocating new approaches that leave behind some of the problematic aspects of traditional theory and research. These efforts have been described with terms such as complexity, complexity science, systems, complex systems, and complex adaptive systems (Braithwaite et al, 2017(Braithwaite et al, , 2018Greenhalgh and Papoutsi, 2018;Khan et al, 2018;Long et al, 2018;Churruca et al, 2019;Henderson et al, 2019;Orton et al, 2019;Sturmberg and Bircher, 2019;South et al, 2020;Younger, 2020). Understanding these approaches is challenging in large part due to substantial inconsistency or confusion regarding their underlying epistemological and ontological assumptions (Alhadeff-Jones, 2008;Poli, 2013;Long et al, 2018).…”