Short Term Survival after Admission for Heart Failure in Sweden: Applying Multilevel Analyses of Discriminatory Accuracy to Evaluate Institutional Performance

Abstract

BackgroundHospital performance is frequently evaluated by analyzing differences between hospital averages in some quality indicators. The results are often expressed as quality charts of hospital variance (e.g., league tables, funnel plots). However, those analyses seldom consider patients heterogeneity around averages, which is of fundamental relevance for a correct evaluation. Therefore, we apply an innovative methodology based on measures of components of variance and discriminatory accuracy to analyze 30-day mortality after hospital discharge with a diagnosis of Heart Failure (HF) in Sweden.MethodsWe analyzed 36,943 patients aged 45–80 treated in 565 wards at 71 hospitals during 2007–2009. We applied single and multilevel logistic regression analyses to calculate the odds ratios and the area under the receiver-operating characteristic (AUC). We evaluated general hospital and ward effects by quantifying the intra-class correlation coefficient (ICC) and the increment in the AUC obtained by adding random effects in a multilevel regression analysis (MLRA). Finally, the Odds Ratios (ORs) for specific ward and hospital characteristics were interpreted jointly with the proportional change in variance (PCV) and the proportion of ORs in the opposite direction (POOR).FindingsOverall, the average 30-day mortality was 9%. Using only patient information on age and previous hospitalizations for different diseases we obtained an AUC = 0.727. This value was almost unchanged when adding sex, country of birth as well as hospitals and wards levels. Average mortality was higher in small wards and municipal hospitals but the POOR values were 15% and 16% respectively.ConclusionsSwedish wards and hospitals in general performed homogeneously well, resulting in a low 30-day mortality rate after HF. In our study, knowledge on a patient’s previous hospitalizations was the best predictor of 30-day mortality, and this information did not improve by knowing the sex and country of birth of the patient or where the patient was treated.

Clinic variation in glycaemic control for children with Type 1 diabetes in England and Wales: a population-based, multilevel analysis

“…Rather, it is important to consider clinic variation as a share of the total variability in HbA 1c [24,25,34]. Rather, it is important to consider clinic variation as a share of the total variability in HbA 1c [24,25,34].…”

“…To better understand the scope for improvements in glycaemic outcomes, we need to consider variation between clinics relative to the total variation (i.e. between and within clinics) [24,25,34]. To explore this, we calculated the intraclass correlation coefficient (ICC) which represents the proportion of total variation in glycaemic control which occurs between clinics, i.e.…”

“…From a health policy perspective, focusing on absolute differences between clinic average values provides insufficient information. Rather, it is important to consider clinic variation as a share of the total variability in HbA 1c [24,25,34]. For example, it is possible to have quite large differences between clinics and still show a low ICC if the variation that occurs within clinics is sufficiently large.…”

“…However, one major obstacle to effective policy action regarding clinic variation, as explored in previous studies, is that it is conceptualized as absolute differences between clinic means. In addition to that, we need to consider the share of the total variation in the glycaemic control that exists between clinics [24,25]. This idea corresponds to the concept of clustering [26].…”

Acknowledging the role of patient heterogeneity in hospital outcome reporting: Mortality after acute myocardial infarction in five European countries

“…Either institutionally framed or developed under the umbrella of research projects, the wealth of administrative data offers the opportunity to access larger samples of patients, covering virtually all providers in a health plan, allowing cross-country comparisons and most importantly, enabling the systematic and continuous monitoring of providers' performance. Many institutional-based [1-7] and research-oriented examples [8][9][10][11][12][13][14][15] illustrate this enormous potential. On the other hand, as performance assessment is increasingly deemed to be the basis for different value-based initiatives (e.g.…”