Methodological Challenges of Mixed Methods Intervention 					Evaluations

Boeije, Hennie; Drabble, Sarah J; O’Cathain, Alicia

doi:10.1027/1614-2241/a000101

Cited by 17 publications

(17 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To get a grip on the complexity of the setting, we include a process evaluation. Process measures can be used to explain differences in effectiveness between clusters (e.g., schools), and understand how and under which conditions an intervention is effective [27,28].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The Unique Extended Selection Cohorts Design for the Evaluation of the School-Based Jump-In Intervention on Dietary Habits: A Study Protocol

Takens

Busch

Ujcic-Voortman

et al. 2020

IJERPH

View full text Add to dashboard Cite

Background: To promote healthy dietary and physical activity behaviour among primary school children, the city of Amsterdam structurally implements the school-based Jump-in intervention in over half of its primary schools. Previously shown to be effective in stimulating physical activity and outside recess play, our study is the first to evaluate Jump-in's effect on children's dietary behaviour. Evaluating the effectiveness and implementation process of an intervention in a real-life setting requests an alternative study design. Methods: we chose a mixed-methods, quasi-experimental Extended Selection Cohorts design to evaluate Jump-in's effectiveness and implementation process. Children and parents from the first ten primary schools that enrolled in the programme in 2016-2017 were invited to participate. The primary outcomes were children's dietary behaviour and behavioural determinants, assessed by child and parent questionnaires, and photographs of the food and drinks children brought to school. Process indicators, contextual factors and satisfaction with the programme were assessed by interviews with health promotion professionals, school principals, school project coordinators, and teachers; focus group discussions with parents and children; and document analysis. Discussion: Conducting research in a real-life setting is accompanied by methodological challenges. Using an Extended Selection Cohorts design provides a valuable alternative when a Randomized Controlled design is not feasible.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The process evaluation focused on the implementation lessons learned and to better grasp how and under which conditions the intervention may be effective or not [27,28]. Figure 2 illustrates the process measurements.…”

Section: Process Evaluationmentioning

confidence: 99%

The Unique Extended Selection Cohorts Design for the Evaluation of the School-Based Jump-In Intervention on Dietary Habits: A Study Protocol

Takens

Busch

Ujcic-Voortman

et al. 2020

IJERPH

View full text Add to dashboard Cite

show abstract

“…A growing body of evidence -in both the health and social sciences -supports conducting process evaluations of complex interventions [e.g. 25,29]. However, few PA evaluations have captured the complexity of behaviour change systems [30].…”

Section: The Importance Of Process Evaluationmentioning

confidence: 99%

A whole system approach to increasing children’s physical activity in a multi-ethnic UK city: a process evaluation protocol

Hall

Seims

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Engaging in regular physical activity requires continued complex decision-making in varied and dynamic individual, social and structural contexts. Widespread shortfalls of physical activity interventions suggests the complex underlying mechanisms of change are not yet fully understood. More insightful process evaluations are needed to design and implement more effective approaches. This paper describes the protocol for a process evaluation of the JU:MP programme, a whole systems approach to increasing physical activity in children and young people aged 5-14 years in North Bradford, UK. Methods: This process evaluation, underpinned by realist philosophy, aims to understand the development and implementation of the JU:MP programme and the mechanisms by which JU:MP influences physical activity in children and young people. It also aims to explore behaviour change across wider policy, strategy and neighbourhood systems. A mixed method data collection approach will include semi-structured interview, observation, documentary analysis, surveys, and participatory evaluation methods including reflections and ripple effect mapping. Discussion: Not only is this an innovative approach to process evaluation but it will also feed into iterative programme development to generate evidence-based practice and deliver practice-based evidence. This paper advances knowledge regarding the development of process evaluations for evaluating systems interventions, and emphasises the importance of process evaluation.

show abstract

“…Furthermore, while the paper discusses several challenges in detail, we acknowledge that more can be said about the challenges of sampling, 38 resources 3 and other methodological quality criteria in MMR 39 and MMR methodology in general. 40,41 The case study was drawn from a completed evaluation but findings have not yet been published in peer-reviewed journals. We can therefore not be certain that our triangulation protocol will be acceptable in the field.…”

Section: Limitationsmentioning

confidence: 99%

Using mixed methods in health services research: A review of the literature and case study

Haan

Eijk-Hustings

Vrijhoef

2020

J Health Serv Res Policy

View full text Add to dashboard Cite

Objective To provide an overview of the challenges of conducting mixed methods research (MMR) in the context of health services research (HSR) and to discuss a case study example of the triangulation procedures used in a MMR study on task-shifting in the Netherlands. Method A narrative literature review of publications between January 2000 and February 2020 on the use of mixed methods in the context of HSR and a description of the triangulation procedures in an HSR study with a MMR design on task shifting in the Netherlands. Results The narrative review identified eight challenges. Those related to publishing and appraisal of MMR within HSR were most frequently reported (e.g. ‘lack of MMR examples’, ‘lack of recognition’ and ‘triangulation issues’). Also, practical problems for conducting MMR within HSR were identified (e.g. ‘lack or resources’, ‘teamwork’ and ‘lack of quality criteria’). Methodological challenges (e.g. ‘sampling’ and ‘paradigm wars’) were less frequently reported as a challenge. Conclusion While increasing in popularity, there remain a range of challenges for the design, conduct and reporting of MMR designs in HSR. Using a triangulation protocol can potentially help address some of these challenges.

show abstract

Methodological Challenges of Mixed Methods Intervention Evaluations

Cited by 17 publications

References 45 publications

The Unique Extended Selection Cohorts Design for the Evaluation of the School-Based Jump-In Intervention on Dietary Habits: A Study Protocol

The Unique Extended Selection Cohorts Design for the Evaluation of the School-Based Jump-In Intervention on Dietary Habits: A Study Protocol

A whole system approach to increasing children’s physical activity in a multi-ethnic UK city: a process evaluation protocol

Using mixed methods in health services research: A review of the literature and case study

Contact Info

Product

Resources

About