Effect of Case-Mix and Random Variation on Breast Cancer Care Quality Indicators and Their Rankability

Abstract

Hospital comparisons to improve quality of care require valid and reliable quality indicators. We aimed to test the validity and reliability of 6 breast cancer indicators by quantifying the influence of case-mix and random variation. Methods:The nationwide population-based database included 79 690 patients with breast cancer from 91 Dutch hospitals between 2011 and 2016. The indicator-scores calculated were: (1) irradical breast-conserving surgery (BCS) for invasive disease, (2) irradical BCS for ductal carcinoma-in-situ, (3) breast contour-preserving treatment, (4) magnetic resonance imaging (MRI) before neo-adjuvant chemotherapy, (5) radiotherapy for locally advanced disease, and (6) surgery within 5 weeks from diagnosis. Case-mix and random variation adjustments were performed by multivariable fixed and random effect logistic regression models. Rankability quantified the between-hospital variation, representing unexplained differences that might be the result of the level of quality of care, as low (,50%), moderate (50%-75%), or high (.75%).Results: All of the indicators showed between-hospital variation with wide (interquartile) ranges. Case-mix adjustment reduced variation in indicators 1 and 3 to 5. Random variation adjustment (further) reduced the variation for all indicators. Case-mix and random variation adjustments influenced the indicator-scores of individual hospitals and their ranking. Rankability was poor for indicator 1, 2, and 5, and moderate for 3, 4, and 6. Conclusions:The 6 indicators lacked validity and/or reliability to a certain extent. Although measuring quality indicators may stimulate quality improvement in general, comparisons and judgments of individual hospital performance should be made with caution if based on indicators that have not been tested or adjusted for validity and reliability, especially in benchmarking.

Stability of hospital quality indicators over time: A multi-year observational study of German hospital data

“…From the analyzed quality indicators, indicator such as DECU and PNEU tend to be better suited as a basis for decision-making for outcome-based payment and contracting as they show a stability in their quintile ranking for the majority hospitals. Our findings support those of other studies that found that justification of such drastic interventions to single outcome indicators without very high stability over time is not recommended [ 18 , 34 ]. Thus, it is absolutely necessary to systematically evaluate the usefulness of every indicator with regards to sufficient stability over time, e.g.…”

Associations of hospital volume and hospital competition with short-term, middle-term and long-term patient outcomes after breast cancer surgery: a retrospective population-based study

“…From a methodological perspective, our study contributes to insight into the actionability of using patient outcomes as quality indicators. Although the conjoint use of three end points to assess hospital variation might be beneficial, the interpretation of the available patient outcomes in our study is accompanied by sensitivity problems and definition ambiguity 35 48 49. Hence, it emphasises the need for routinely collected outcome measures for high-volume tumours to adequately assess quality variation.…”

“…Alternatively, we calculated survival from the date of diagnosis, which may have resulted in a small overestimation of length of survival, as all patients underwent a surgical procedure and thus survived up and until the date of surgery. However, it is not likely that this has resulted in a large source of bias, as the vast majority of patients in the Netherlands has been operated within five weeks of diagnosis48. Fourth, although commonly used, our measure for hospital competition is rather crude and may therefore not accurately reflect all competitive pressures faced by hospitals.…”

Developing quality indicators for cancer hospitals in China: a national modified Delphi process

“…The indicator set developed in our study helps to measure the quality of care and conduct comparisons across hospitals or across different periods. When conducting a comparison, risk adjustment is a prerequisite, as otherwise underestimation of performance for these practices serving vulnerable patients is possible 51 52. Previous studies have also found that risk adjustment played a larger role for outcome indicators than for process indicators, indicating the importance of casemix adjustment for outcome indicators 53…”