Using hospital standardised mortality ratios to assess quality of care — proceed with extreme caution

Scott, Ian; Brand, Caroline; Phelps, Grant; Barker, Anna; Cameron, Peter

doi:10.5694/j.1326-5377.2011.tb03150.x

Cited by 51 publications

(46 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the UK, Canada, the Netherlands and the United States, HSMR has been used for many years within organisations to monitor performance and response to various quality and safety programs [39]. It has also been used by a variety of stakeholders to compare performance between hospitals [40].…”

Section: Discussionmentioning

confidence: 99%

Towards National Surgical Surveillance in the UK – A Pilot Study

Agha

2012

PLoS ONE

View full text Add to dashboard Cite

ObjectiveThe Bristol heart inquiry in the United Kingdom (UK) highlighted the lack of standards for evaluating surgical performance and quality. In 2009, the World Health Organisation (WHO) proposed six standardised metrics for surgical surveillance. This is the first study to collect and analyse such metrics from a cohort of National Health Service (NHS) Trusts in England, helping to determine their feasibility and utility in measuring surgical performance, its impact on public health and mortality, and for tracking surgical trends over time.MethodsFreedom of Information Act 2000 (FOI) requests for WHO standardised surgical metrics were made to 36 NHS Trusts in England during July to November 2010. Additional data on Hospital Standardised Mortality Ratio (HSMR), Patient Safety Score and Abdominal Aortic Aneurysm (AAA) volume and mortality was obtained from Dr Foster Health and The Guardian Newspaper. Analysis was performed using mixed-effect logistic regression.Results30/36 trusts responded (83%). During 2005–9, 5.4 million operations were performed with a 24.2% increase in annual number of operations. This rising volume within hospitals was associated with lower mortality ratios. A 10% increase in operative volume was associated with a lower day of surgery death rate (DDR OR = 0.94, p = 0.056) and post-operative inpatient 30-day mortality (PDR30 OR = 0.93, p = 0.001). For every 10,000 more operations that an NHS Trust does, a 4% drop in PDR30 mortality was achieved. A 10% increase in the volume of elective AAAs was associated with lower elective AAA (OR = 0.96, p = 0.032) and emergency AAA (OR = 0.95, p = 0.009) PDR30 mortality. Lower DDR mortality was noted for emergency AAA mortality (OR = 0.95, p = 0.025) but not elective AAAs (OR = 0.97, p = 0.116).ConclusionStandarised surgical metrics can provide policy makers and commissioners with valuable summary data on surgical performance allowing for statistical process control of a complex intervention. This study has shown their collection is feasible albeit using FOI and the first to show a statistically significant volume-outcome relationship for surgery as a whole within hospitals. It adds weight to the argument that patients are safer in larger hospitals or those that become larger by growing their patient base. Together with other measures, such metrics can help build a picture of surgical surveillance in the UK and potentially lead us to safer surgery.

show abstract

Section: Discussionmentioning

confidence: 99%

Towards National Surgical Surveillance in the UK – A Pilot Study

Agha

2012

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…32 33 Overlooking variations in ward performance results in blanket hospital grades that are unreliable 34 and aggregated hospital outcomes that are not accurate indicators of quality of care. [33][34][35][36][37][38][39][40][41][42] Aggregate measures obscure the need for greater investment in ward specific datasets. Without them, major policy decisions-on the adequacy of resources for safe staffing, for example-will be based on incorrect inferences drawn from averaged, hospital-wide data.…”

Section: Variations In Ward Care Are Ignoredmentioning

confidence: 99%

Rethinking medical ward quality

Pannick

Wachter

Vincent

et al. 2016

BMJ

View full text Add to dashboard Cite

Rethinking medical ward qualityFor quality to improve, we need to embrace the complexities of general medical inpatient care, say Samuel Pannick and colleagues Medical wards deliver the majority of acute inpatient care in health systems worldwide. This care is expensive, costing the NHS around £5bn (€5.5bn; $6.2bn) a year, a quarter of its inpatient expenditure.1 Improving the performance of medical wards is an international priority, 2 3 not only because of the scale of care that they deliver. Their core workload-treating complex, increasingly frail patients in a time pressurised setting-represents the broader challenges facing healthcare. 4 Yet major gaps remain in our understanding of how wards perform. [5][6][7] Safety and quality interventions have been most effective in improving standardised clinical tasks in the operating theatre and intensive care unit, such as the insertion of central venous catheters. The processes of ward care require a more nuanced approach to improvement. Medical patients' clinical syndromes often fall between traditional diagnostic categories, 8 and specific organisational challenges exist for the teams that care for them. We discuss the unique properties of medical wards and the problems they face, before setting out a vision for ward improvement that embraces the complexity of ward care. The medical ward is a different animalImportant differences exist between medical wards and other clinical settings, from haemodialysis units to operating theatres. Medical ward teams care for a particularly heterogeneous group of patients, with no single best pathway for diagnosis or treatment. Staff are skilled in the management of a diverse range of conditions, from pyelonephritis to gastrointestinal bleeding and terminal cancer. Many patients arrive without a diagnosis; indeed, empirical treatment can be concluded with no definitive diagnosis ever established. This sets medical wards apart from other hospital settings, which typically manage more narrowly defined patient populations with more predictable care trajectories. With such heterogeneity, medical ward teams may struggle to articulate their clinical and business aims and are better defined by their interpersonal networks and the flow of information within them. 9Episodes of medical ward care can be long, involving large, dynamic, multidisciplinary teams. Team members are often dispersed throughout the hospital; physicians and allied health professionals are rarely located together on one unit with nurses and their patients. Frequent handovers are made more difficult by the absence of a central procedure around which a structured care narrative can be formed.Errors are common and often serious-medical ward patients have the same risks of preventable and fatal adverse events as those in intensive care, 7 10 and preventable hospital deaths are disproportionately caused by failures in general ward care. 6 Crucially, ward failures are different from the procedural misadventures of the operating theatre or intensive care unit, 10 re...

show abstract

“…In these circumstances, the assumption of a constant association with mortality is known as the ‘constant risk fallacy’ and may result in risk adjustment paradoxically worsening the effect of differences in case mix upon risk-adjusted mortality 19. These limitations of risk-adjustment methods have led to criticism of the use of risk-adjusted outcomes to judge quality of care 20 21…”

Section: What Are the Limitations Of Risk Adjustment?mentioning

confidence: 99%

What do hospital mortality rates tell us about quality of care?

2013

View full text Add to dashboard Cite

Hospital mortality rates could be useful indicators of quality of care, but careful statistical analysis is required to avoid erroneously attributing variation in mortality to differences in health care when it is actually due to differences in case mix. The summary hospital mortality indicator is currently used by the English National Health Service (NHS). It adjusts mortality rates up to 30 days after discharge for patient age, sex, type of admission, year of discharge, comorbidity, deprivation and diagnosis. Such risk-adjustment methods have been used to identify poor performance, most notably at mid-Staffordshire NHS Foundation Trust, but their use is subject to a number of limitations. Studies exploring whether variation in risk-adjusted mortality can be explained by variation in healthcare have reached conflicting conclusions. Furthermore, concerns have been raised that the proportion of preventable deaths among hospital admissions is too small to produce a reliable 'signal' in risk-adjusted mortality rates. This provides hospital managers, regulators and clinicians with a considerable dilemma. Variation in mortality rates cannot be ignored, as they might indicate unacceptable variation in healthcare and avoidable mortality, but they also cannot be reliably used to judge the quality of healthcare, based on current evidence.

show abstract

Using hospital standardised mortality ratios to assess quality of care — proceed with extreme caution

Cited by 51 publications

References 28 publications

Towards National Surgical Surveillance in the UK – A Pilot Study

Towards National Surgical Surveillance in the UK – A Pilot Study

Rethinking medical ward quality

What do hospital mortality rates tell us about quality of care?

Contact Info

Product

Resources

About