BackgroundThe last two decades have seen a growing recognition of the need to expand the impact evaluation toolbox from an exclusive focus on randomized controlled trials to including quasi-experimental approaches. This appears to be particularly relevant when evaluation complex health interventions embedded in real-life settings often characterized by multiple research interests, limited researcher control, concurrently implemented policies and interventions, and other internal validity-threatening circumstances. To date, however, most studies described in the literature have employed either an exclusive experimental or an exclusive quasi-experimental approach.MethodsThis paper presents the case of a study design exploiting the respective advantages of both approaches by combining experimental and quasi-experimental elements to evaluate the impact of a Performance-Based Financing (PBF) intervention in Burkina Faso. Specifically, the study employed a quasi-experimental design (pretest-posttest with comparison) with a nested experimental component (randomized controlled trial). A difference-in-differences approach was used as the main analytical strategy.DiscussionWe aim to illustrate a way to reconcile scientific and pragmatic concerns to generate policy-relevant evidence on the intervention’s impact, which is methodologically rigorous in its identification strategy but also considerate of the context within which the intervention took place. In particular, we highlight how we formulated our research questions, ultimately leading our design choices, on the basis of the knowledge needs expressed by the policy and implementing stakeholders. We discuss methodological weaknesses of the design arising from contextual constraints and the accommodation of various interests, and how we worked ex-post to address them to the best extent possible to ensure maximal accuracy and credibility of our findings. We hope that our case may be inspirational for other researchers wishing to undertake research in settings where field circumstances do not appear to be ideal for an impact evaluation.Trial registrationRegistered with RIDIE (RIDIE-STUDY-ID-54412a964bce8) on 10/17/2014.

Background: As countries reform health financing systems towards universal health coverage, increasing concerns emerge on the need to ensure inclusion of the most vulnerable segments of society, working to counteract existing inequities in service coverage. To this end, selected countries in sub-Saharan Africa have decided to couple performance-based financing (PBF) with demand-side equity measures. Still, evidence on the equity impacts of these more complex PBF models is largely lacking. We aimed at filling this gap in knowledge by assessing the equity impact of PBF combined with equity measures on utilization of maternal health services in Burkina Faso. Methods: Our study took place in 24 districts in rural Burkina Faso. We implemented an experimental design (clusterrandomized trial) nested within a quasi-experimental one (pre- and post-test design with independent controls). Our analysis relied on self-reported data on pregnancy history from 9999 (baseline) and 11 010 (endline) women of reproductive age (15-49 years) on use of maternal healthcare and reproductive health services, and estimated effects using a difference-in-differences (DID) approach, purposely focused on identifying program effects among the poorest wealth quintile. Results: PBF improved the utilization of few selected maternal health services compared to status quo service provision. These benefits, however, were not accrued by the poorest 20%, but rather by the other quintiles. PBF combined with equity measures did not produce better or more equitable results than standard PBF, with specific differences only on selected outcomes. Conclusion: Our findings challenge the notion that implementing equity measures alongside PBF is sufficient to produce an equitable distribution in program benefits and point at the need to identify more innovative and contextsensitive measures to ensure adequate access to care for the poorest. Our findings also highlight the importance of considering changing policy environments and the need to assess interferences across policies.

In spite of the wide attention performance-based financing (PBF) has received over the past decade, no evidence is available on its impacts on quantity and mix of service provision nor on its interaction with parallel health financing interventions. Our study aimed to examine the PBF impact on quantity and mix of service provision in Burkina Faso, while accounting for the parallel introduction of a free healthcare policy. We used Health Management Information System data from 838 primary-level health facilities across 24 districts and relied on an interrupted time-series analysis with independent controls. We placed two interruptions, one to account for PBF and one to account for the free healthcare policy. In the period before the free healthcare policy, PBF produced significant but modest increases across a wide range of maternal and child services, but a significant decrease in child immunization coverage. In the period after the introduction of the free healthcare policy, PBF did not affect service provision in intervention compared with control facilities, possibly indicating a saturation effect. Our findings indicate that PBF can produce modest increases in service provision, without altering the overall service mix. Our findings, however, also indicate that the introduction of other health financing reforms can quickly crowd out the effects produced by PBF. Further qualitative research is required to understand what factors allow healthcare providers to increase the provision of some, but not all services and how they react to the joint implementation of PBF and free health care.