Evaluating Infrastructure Development in Complex Home Visiting Systems

Abstract

In recent years, increased focus on the effectiveness and accountability of prevention and intervention programs has led to greater government funding for the implementation and spread of evidence-based health and human service delivery models. In particular, attention has been paid to programs that require significant infrastructure investment and systems change to support large-scale replication. For conceptual and methodological reasons, such systems change initiatives can be a challenge to evaluate. To overcome these challenges, this article outlines a mixed methods approach to systems change evaluation and offers a case study of how this approach has been used to evaluate the development of system infrastructure supporting the implementation, spread, and sustainability of evidence-based home visiting projects. The approach combined systems concepts (boundaries, relationships, perspectives, ecological levels, and dynamics) and qualitative methods (project site visits, telephone interviews, reviews of project documents and logic models) with quantitative methods (a web-based partner survey) to directly measure the projects' system properties and contextual dynamics, and to assess how these factors were associated with the projects' infrastructure development. In the case study, the projects worked at four ecological levels (organization, community, state, and national) to build eight types of infrastructure (planning, collaboration, operations, workforce development, fiscal capacity, community and political support, communications, and evaluation). The evaluation found that the size of the projects' partner networks was not as important as the quality of their collaboration or their sharing of common goals in the projects' infrastructure development.

Retention and attrition in a home visiting programme: looking back and moving forward

“…Specifically, investigators were interested in identifying families in a five-county region that were most likely to leave the First Teacher system through attrition and the potential reasons for choosing to leave or stay in the programme. Given the complexity of the research phenomenon, investigators selected a mixed-methods design to inform the research process and to provide robust and useful research results (Hargreaves, Cole, Coffee-Borden, Paulsell, & Boller, 2013).…”

Tools for Evaluating the Implementation of Complex Education Interventions

“…The texts highlight the interactive, nested, and emergent nature of phenomena in complex systems and suggest useful methods for investigating and learning within evaluations of complex systems. Evaluations that build from complex systems frameworks yield important insights into the networked nature of systems change (M. Hargreaves et al, 2013).…”