Ranking hospitals: do we gain reliability by using composite rather than individual indicators?

Hofstede, Stefanie N.; Ceyisakar, Iris; Lingsma, Hester F.; Kringos, Dionne; Mheen, Perla J Marang-van de

doi:10.1136/bmjqs-2017-007669

Cited by 23 publications

(33 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The HARM score is a composite quality measurement, motivated by the fact that simple rates of risk‐adjusted morbidity or mortality may not reliably reflect hospital performance with surgery. In the literature, several composite surgical quality measurements have been described. All of these metrics are fairly new, and none seems to have been firmly established.…”

Section: Discussionmentioning

confidence: 99%

Benchmarking of abdominal surgery: a study evaluating the HARM score in a European national cohort

et al. 2020

View full text Add to dashboard Cite

Background: Reliable, easily accessible metrics of surgical quality are currently lacking. The HARM (HospitAl length of stay, Readmission and Mortality) score is a composite measure that has been validated across diverse surgical cohorts. The aim of this study was to validate the HARM score in a national population of patients undergoing abdominal surgery.Methods: Data on all abdominal surgery in Norwegian hospitals from 2011 to 2017 were obtained from the Norwegian Patient Registry. Readmissions and 30-day postoperative complications as well as deaths in and out of hospital were evaluated. The HARM scoring algorithm was tested after adjustment by establishing a newly proposed length of stay score. The correlation between the HARM score and complications, as well as the ability of aggregated HARM scores to discriminate between hospitals, were analysed. Risk adjustment models were developed for nationwide hospital comparisons.Results: The data consisted of 407 113 primary operations on 295 999 patients in 85 hospitals. The HARM score was associated with complications and complication severity (Goodman-Kruskal value 0⋅59). Surgical specialty was the dominating variable for risk adjustment. Based on 1-year data, the risk-adjusted score classified 16 hospitals as low HARM score and 16 as high HARM score of the 53 hospitals that had at least 30 operations. Conclusion:The HARM score correlates with major outcomes and is associated with the presence and severity of complications. After risk adjustment, the HARM score discriminated strongly between hospitals in a European population of abdominal surgery.

show abstract

Section: Discussionmentioning

confidence: 99%

Benchmarking of abdominal surgery: a study evaluating the HARM score in a European national cohort

et al. 2020

View full text Add to dashboard Cite

show abstract

“…We considered scenarios in which the prevalences of the three indicators across all hospitals were 0.05, 0.10, and 0.25 (Pr( Y 1 ij = 1) = 0.05, Pr( Y 2 ij = 1) = 0.10, and Pr( Y 3 ij = 1) = 0.25) as this is typical the range of prevalences occurring frequently in practice. For instance, Hofstede et al found that the median hospital-specific rate of in-hospital mortality amongst patients with colorectal carcinoma was 4.9%, while the median acute readmission rate for stroke patients was 6.1% [23]. They found that the median in-hospital mortality rate for patients with heart failure was 11.0%, while the acute readmission rate for colorectal carcinoma patients was 10.7%.…”

Section: Methodsmentioning

confidence: 99%

“…Lawson examined the rankability of SSI following colorectal surgery and found that the mean rankability was 0.65 for superficial SSI, 0.40 for deep/organ-space SSI, and 0.59 for any SSI [22]. Hofstede et al examined the rankability of in-hospital mortality for a variety of conditions or procedures [23]. They found that rankability ranged from 0.01 for patients with osteoarthritis undergoing total hip arthroplasty/total knee arthroplasty to 0.71 following hospitalization for stroke.…”

Section: Introductionmentioning

confidence: 99%

Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?

Austin¹,

Ceyisakar

Lingsma

et al. 2019

BMC Med Res Methodol

View full text Add to dashboard Cite

Background Report cards on the health care system increasingly report provider-specific performance on indicators that measure the quality of health care delivered. A natural reaction to the publishing of hospital-specific performance on a given indicator is to create ‘league tables’ that rank hospitals according to their performance. However, many indicators have been shown to have low to moderate rankability, meaning that they cannot be used to accurately rank hospitals. Our objective was to define conditions for improving the ability to rank hospitals by combining several binary indicators with low to moderate rankability. Methods Monte Carlo simulations to examine the rankability of composite ordinal indicators created by pooling three binary indicators with low to moderate rankability. We considered scenarios in which the prevalences of the three binary indicators were 0.05, 0.10, and 0.25 and the within-hospital correlation between these indicators varied between − 0.25 and 0.90. Results Creation of an ordinal indicator with high rankability was possible when the three component binary indicators were strongly correlated with one another (the within-hospital correlation in indicators was at least 0.5). When the binary indicators were independent or weakly correlated with one another (the within-hospital correlation in indicators was less than 0.5), the rankability of the composite ordinal indicator was often less than at least one of its binary components. The rankability of the composite indicator was most affected by the rankability of the most prevalent indicator and the magnitude of the within-hospital correlation between the indicators. Conclusions Pooling highly-correlated binary indicators can result in a composite ordinal indicator with high rankability. Otherwise, the composite ordinal indicator may have lower rankability than some of its constituent components. It is recommended that binary indicators be combined to increase rankability only if they represent the same concept of quality of care.

show abstract

“…In BMJ Quality and Safety , Hofstede et al 8 have addressed a common situation where providers (such as hospitals, general practices or community teams) are ranked according to their performance on a quality indicator. Rankings are used often to make direct performance comparisons between providers and used to identify positive or negative outliers.…”

mentioning

confidence: 99%

“…Hofstede et al 8 examine whether it is possible to improve the reliability of rankings based on quality measures. Two strategies are assessed: combining indicator data across several years to increase the number of events (eg, reporting readmission rates based on the number of admissions occurring over a multi-year period rather than a single year) or generating a composite measure by combining information from two or more quality indicators.…”

mentioning

confidence: 99%

Composite measures of healthcare quality: sensible in theory, problematic in practice

Friebel

Steventon

2018

BMJ Qual Saf

View full text Add to dashboard Cite

Ranking hospitals: do we gain reliability by using composite rather than individual indicators?

Cited by 23 publications

References 28 publications

Benchmarking of abdominal surgery: a study evaluating the HARM score in a European national cohort

Benchmarking of abdominal surgery: a study evaluating the HARM score in a European national cohort

Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?

Composite measures of healthcare quality: sensible in theory, problematic in practice

Contact Info

Product

Resources

About