Evaluating quality improvement at scale: design considerations for routine reporting to executive board of a healthcare organisation in the UK National Health Service

Abstract

Background Quality improvement (QI) in healthcare is a cultural transformation process that requires long-term commitment from the executive board, a critical theme in emerging accounts of QI success in the UK National Health Service (NHS). To help sustain long-term commitment from the executive board, an organisation-wide picture of QI applications and their impact needs to be made routinely visible.Method We developed a retrospective evaluation drawing inputs from the resident QI team of a healthcare organisation and academic colleagues in the field of implementation and improvement science, as well as peer-reviewed and grey literature on what constitutes success for QI in healthcare. Formative feedback on content relevance, acceptability, and feasibility issues were used to guide evaluation design. The evaluation was conducted as an online survey so that the data accrual process resembles routine reporting to help surface implementation challenges. A purposive sample of QI projects was identified to maximise contrast between projects that were or were not successful as determined by the resident QI team. To hone strategic focus in what should be reported, we also compared factors that might affect project outcomes. For understanding implementation issues, we reviewed data quality to surface challenges in the design and sustainability of routine reporting for the executive board.Results Out of 52 QI projects, 10 led to a change in routine practice (henceforth referred to as adoption). Details of project outcomes were limited. Project team outcomes, indicative of capacity building, were not systematically documented. Service user involvement, quality of measurement plan, fidelity and documentation of plan-do-study-act (PDSA) cycles had a major impact on adoption. The proximal impact of these process factors on adoption was consistently more apparent than the distal impact of input and contextual factors.Conclusions Designing a routine reporting framework is an iterative process involving continual dialogue with frontline staff and improvement specialists to navigate data accrual demands. A retrospective evaluation, as in this study, can yield empirical insights for dialogue about the routine visibility of QI applications and their organisation-wide impact, thereby honing the implementation science of QI in a healthcare organisation.