Hospital-level evaluation of the effect of a national quality improvement programme: time-series analysis of registry data

Abstract

Background and objectivesA clinical trial in 93 NHS hospitals evaluated a quality improvement programme for emergency abdominal surgery, designed to improve mortality by improving the patient care pathway. Large variation was observed in implementation approaches and the main trial result showed no mortality reduction. Our objective therefore was to evaluate whether trial participation led to care-pathway implementation and to study the relationship between care-pathway implementation and use of six recommended implementation strategies. MethodsWe performed a hospital-level time-series analysis using data from the Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial. Care-pathway implementation was defined as achievement of >80% median reliability in ten measured care-processes. Mean monthly process performance was plotted on run-charts. Process improvement was defined as an observed run-chart signal, using probability-based 'shift' and 'runs' rules. A new median performance level was calculated after an observed signal. ResultsOf 93 participating hospitals, 80 provided sufficient data for analysis, generating 800 process measure charts from 20,305 patient admissions over 27 months. No hospital reliably implemented all ten processes. Overall, only 279 of the 800 processes were improved (3 [2-5] per hospital) and 14/80 hospitals improved more than six processes. Mortality-risk documented (57/80 [71%]), lactate measurement (42/80 [53%]) and cardiac-output guided fluid therapy (32/80 [40%]) were most frequently improved. Consultant-led decision making (14/80 [18%]), consultant review before surgery (17/80 [21%]) and time to surgery (14/80 [18%]) were least frequently improved. In hospitals using ≥5implementation strategies, 9/30 (30%) hospitals improved 6 care processes compared with 0/11 hospitals using ≤2 implementation strategies. ConclusionOnly a small number of hospitals improved more than half of the measured care-processes, more often when at least 5 of 6 implementation strategies were used. In a longer-term project this understanding may have allowed us to adapt the intervention to be effective in more hospitals. BACKGROUNDAs the volume of surgical procedures performed worldwide continues to increase [1,2] the need for improvement in the quality and safety of surgical care has become a global healthcare priority [3][4][5]. This is of particular importance considering both the increasing age and complexity of the surgical population and the global mortality burden associated with surgery [6,7]. Emergency abdominal surgery is a commonly performed procedure worldwide, with high mortality rates, and wide variations in the standards of care [8][9][10][11]. The Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial was performed to test whether a national quality improvement (QI) programme to implement a carepathway could reduce 90-day mortality following emergency abdominal surgery [12].The EPOCH trial intervention consisted of an evidence based care-pathway designed to improve patient o...

Cost-effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery: Learning from 15,856 patients

“…The QI programme in the EPOCH trial required an extensive care pathway with 37 components to be implemented (9) and there were wide variations in these elements at individual hospitals and thus local adaptations in each hospital were needed. According to the ethnographic evaluation of this QI programme (32), hospital staff often had little or no additional time to improve patient care and consequently, the objective of the EPOCH QI programme was not readily achieved because the care pathway was not implemented as fully as intended (33). As such the lack of cost-effectiveness identified here may be due, to a large extent, on implementation failure resulting in a loss of intervention effectiveness.…”

Revitalising audit and feedback by understanding and responding to local contexts

“…In addition to the study designs described by Foy et al, an understanding of relevant contextual factors using qualitative methods can help explain how national audits are perceived locally and used for improvement (or not). Recent evaluations of a quality improvement intervention in the UK using data from the National Emergency Laparotomy Audit, for example, showed the importance of social barriers to change 234…”

Challenge of ensuring access to high-quality emergency surgical care for all

“…In this issue of BMJ Quality and Safety, Stephens and colleagues provide insight into the first problem of ineffective implementation. 4 The authors used a hospitallevel time-series analysis and run charts to determine whether or not participation in the EPOCH trial led to actual changes in the 10 processes of care targeted by trial. 4 They also characterised the relationship between care-pathway implementation and the use of six suggested implementation strategies.…”

“…4 The authors used a hospitallevel time-series analysis and run charts to determine whether or not participation in the EPOCH trial led to actual changes in the 10 processes of care targeted by trial. 4 They also characterised the relationship between care-pathway implementation and the use of six suggested implementation strategies. Ultimately, they found that only a small number of hospitals improved even half of the 10 targeted processes, but the greatest number of process improvements occurred at the hospitals that used five or more of the suggested implementation strategies.…”

“…5 This collaborative provided change management and leadership coaching as well as quarterly feedback in the form of run charts as suggested by Stephens and colleagues. 4 The ELC trial demonstrated that iterative improvements in care bundle implementation were temporally associated with iterative improvements in mortality and hospital length of stay, thereby suggesting that the processes of care under investigation could be associated with better outcomes. Unfortunately, this study does not prove causation as the lack of a concurrent control group makes it impossible to account for secular trends-a limitation common to many preimplementation/postimplementation studies.…”