Conflicting measures of hospital quality: Ratings from “Hospital Compare” versus “Best Hospitals”

Abstract

BACKGROUNDIn April 2005 the Centers for Medicare and Medicaid Services launched “Hospital Compare,” the first government‐sponsored hospital quality scorecard. We compared the ranking of U.S. News and World Report's “Best Hospitals” with Hospital Compare performance ratings.METHODSWe examined Hospital Compare scores for core measures related to care for acute myocardial infarction (AMI), congestive heart failure (CHF), and community‐acquired pneumonia (CAP). We calculated composite scores for the disease‐specific sets of core measures and a composite combined score for the 14 core measures (across 3 diseases) and determined national score quartile cut points for each set. We then characterized the quartile distribution of Hospital Compare scores for the Best Hospitals for care of cardiac conditions and respiratory disorders in each year, as well as for the Best Hospital “Honor Roll” institutions.RESULTSAMI scores were available for 2165 hospitals, CHF scores for 3130, and CAP scores for 3462. In both 2004 and 2005, fewer than 50% of the Best Hospitals for cardiac care rated in the top quartile of Hospital Compare scores for AMI and CHF. Among the Best Hospitals for care of respiratory disorders, fewer than 15% scored in the top Hospital Compare quartile for CAP. Among Honor Roll institutions, only 5 (of 14 hospitals in 2004; of 16 in 2005) ranked in the top quartile for the combined core measure score.CONCLUSIONSHospital Compare scores are frequently discordant with Best Hospital rankings, which is likely attributable to the markedly different methods each rating approach employs. Such discordance between major quality rating systems paints a conflicting picture of institutional performance for the public to interpret. Journal of Hospital Medicine 2007;2:128–134. © 2007 Society of Hospital Medicine.

Physician-leaders and hospital performance: Is there an association?

“…Halasyamani and Davis, 2007;McGaghie and Thompson, 2001) than others (e.g. Wang, Wang, Lichtman, Bradley, Normand and Krumholz, 2007;Souba, 2008;Philibert, 2009 (2007) found an imperfect correlation for cardiac diagnoses and respiratory disorders between the highest-ranked hospitals in USNWR and those in ‗Hospital Compare'.…”

Hospital quality measurement— Perplexing for professionals, let alone for patients

“…In this issue of the Journal of Hospital Medicine, Halasyamani and Davis report that there is relatively poor correlation between the Hospital Compare scores of the Centers for Medicare and Medicaid Services (CMS) and U.S. News and World Report's "Best Hospitals" rankings. 1 The authors note that this is not necessarily surprising, as the methodologies of these rating systems are quite different, although their purposes are functionally similar.…”

“…In Halasyamani and Davis's study, only 2.9% of hospitals reported all 14 Hospital Compare core performance measures used in their analysis. 1 Evidence suggests that poor performance is a strong disincentive to voluntarily report quality measures to the public. 5 And because there is no evidence that this type of transparency initiative will drive volume to higher-quality programs, publicly reporting quality measures may not provide a strong enough incentive for hospitals to allocate resources to the improvement of the quality of care they deliver in these specific areas.…”

National Hospital Ratings Systems Share Few Common Scores And May Generate Confusion Instead Of Clarity

“…Some research indicates that being named to U.S. News & World Report's "Best Hospitals" list is associated with lower thirty-day mortality, 4 but other studies have found no association between the U.S. News list and readmissions, 5 wide variation on a number of indicators, 6 and discrepancies with other ratings systems such as the Centers for Medicare and Medicaid Services' (CMS's) Hospital Compare. 7 Hospital rating systems use a variety of methods for distinguishing "high" performers from "low" performers, often creating the paradox of hospitals' simultaneously being considered best and worst depending on the rating system used. [8][9][10] For example, 43 percent of hospitals classified as having below-average mortality by one risk-adjustment method were classified as having above-average mortality by another method.…”