Intelligent Information: A National System for Monitoring Clinical Performance

Abstract

Objective. To use statistical process control charts to monitor in‐hospital outcomes at the hospital level for a wide range of procedures and diagnoses. Data Sources. Routine English hospital admissions data. Study Design. Retrospective analysis using risk‐adjusted log‐likelihood cumulative sum (CUSUM) charts, comparing each hospital with the national average and its peers for in‐hospital mortality, length of stay, and emergency readmission within 28 days. Data Collection. Data were derived from the Department of Health administrative hospital admissions database, with monthly uploads from the clearing service. Principal Findings. The tool is currently being used by nearly 100 hospitals and also a number of primary care trusts responsible for purchasing hospital care. It monitors around 80 percent of admissions and in‐hospital deaths. Case‐mix adjustment gives values for the area under the receiver operating characteristic curve between 0.60 and 0.86 for mortality, but the values were poorer for readmission. Conclusions. CUSUMs are a promising management tool for managers and clinicians for driving improvement in hospital performance for a range of outcomes, and interactive presentation via a web‐based front end has been well received by users. Our methods act as a focus for intelligently directed clinical audit with the real potential to improve outcomes, but wider availability and prospective monitoring are required to fully assess the method's utility.

Evaluation of a national surveillance system for mortality alerts: a mixed-methods study

“…Since then, there have been several international papers investigating the use of control charts to monitor the quality of hospital care in other settings; 56,62,66,68 however, these studies were on clinical audit data and not on administrative hospital data. Bottle and Aylin 35 published a paper in 2007 that investigated the use of statistical control charts for monitoring clinical performance, including in-hospital mortality. The methodology described was the basis for the national surveillance system for mortality alerts that is evaluated in this report.…”

“…Table 1 lists the studies identified in the literature search on mortality surveillance as an indicator of quality. Hospital mortality monitoring studies were international, covering the UK, 2,14,35,51,53 the USA, 49,52,59,64,67,68,70 the Netherlands, 57,63 Canada 55,69 and Australia and New Zealand. 50,54,56,58,[60][61][62]65,66 Studies were for a range of conditions and settings; these included surgical, 2,51,59,64,66,70 intensive care, 50,52,54,61,63,65 transplantation 53,67,68 and all cause.…”

“…35 Diagnoses are determined from the primary diagnosis within an admission. The ICD-10 codes are combined into clinically meaningful groups using the agency for Healthcare Research and Quality's Clinical Classification Software (CCS).…”

“…An OPCS grouping system was developed at the Imperial College Unit through an expert panel, a literature review and user input. 35 A list of the conditions and procedures covered by the alerting system is provided in Appendix 2.…”

Hospital mortality under surgical care

“…In the UK revelations from the Bristol inquiry into paediatric cardiac surgery deaths from 1999-2001, 1 combined with changes in the way the healthcare is commissioned and purchased through 'payment by results', have led to a drive to identify measures by which NHS trusts can be judged. 2 Until 2005 UK hospitals were compared using the Healthcare Commission's 'star' rating system, which combined indicators covering a range of administrative, financial management and clinical services. Since 2005 hospitals are judged through the 'annual health check', whereby performance is measured against existing national targets such as whether the patient waited more than 4 hours in accident and emergency.…”