Lessons learned from using linked administrative data to evaluate the Family Nurse Partnership in England and Scotland

Abstract

Introduction“Big data” – including linked administrative data – can be exploited to evaluate interventions for maternal and child health, providing time- and cost-effective alternatives to randomised controlled trials. However, using these data to evaluate population-level interventions can be challenging. ObjectivesWe aimed to inform future evaluations of complex interventions by describing sources of bias, lessons learned, and suggestions for improvements, based on two observational studies using linked administrative data from health, education and social care sectors to evaluate the Family Nurse Partnership (FNP) in England and Scotland. MethodsWe first considered how different sources of potential bias within the administrative data could affect results of the evaluations. We explored how each study design addressed these sources of bias using maternal confounders captured in the data. We then determined what additional information could be captured at each step of the complex intervention to enable analysts to minimise bias and maximise comparability between intervention and usual care groups, so that any observed differences can be attributed to the intervention. ResultsLessons learned include the need for i) detailed data on intervention activity (dates/geography) and usual care; ii) improved information on data linkage quality to accurately characterise control groups; iii) more efficient provision of linked data to ensure timeliness of results; iv) better measurement of confounding characteristics affecting who is eligible, approached and enrolled. ConclusionsLinked administrative data are a valuable resource for evaluations of the FNP national programme and other complex population-level interventions. However, information on local programme delivery and usual care are required to account for biases that characterise those who receive the intervention, and to inform understanding of mechanisms of effect. National, ongoing, robust evaluations of complex public health evaluations would be more achievable if programme implementation was integrated with improved national and local data collection, and robust quasi-experimental designs.

Intensive home visiting for adolescent mothers in the Family Nurse Partnership in England 2010–2019: a population-based data linkage cohort study using propensity score matching

“…There are a number of challenges inherent to evaluations using administrative data, and findings in this report should be interpreted in the context of three main limitations. 45 First, although we carefully designed and assessed the propensity score analysis strategy, our approach is subject to the assumption that conditional on the propensity score, the distribution of characteristics between BMJ Public Health groups was balanced. However, there may be residual confounding as we could only control for the fairly crude maternal risk factors associated with enrolment in FNP that are captured in administrative data.…”

“…These challenges are relevant to other studies aiming to use administrative data to evaluate public health interventions. 45 Second, outcomes captured in administrative data can be difficult to interpret. For example, the increased rates of unplanned admissions and A&E attendances in the mother and child associated with FNP may be viewed as contradictory to the aims of improving child health, but may actually reflect appropriate care seeking as a result of advice and support from family nurses.…”