The impact of varying patient populations on the in-control performance of the risk-adjusted CUSUM chart

Abstract

The simulation results imply that the control limits should vary based on the particular patient population of interest in order to control the in-control performance of the risk-adjusted Bernoulli CUSUM method.

Dynamic Probability Control Limits for Lower and Two‐Sided Risk‐Adjusted Bernoulli CUSUM Charts

“…The model was fitted using the first 2 years of data containing 2218 patients' records, which were treated as the in‐control group. The Parsonnet score sequences were randomly selected with replacement from each of the following five different Phase I patient populations represented by different Parsonnet score distributions defined by Tian et al : All: 2218 scores for all patients (mean = 8.9026). High risk: the highest 50% of the scores (mean = 16.4813). Low risk: the lowest 50% of the scores (mean = 2.0541). Surgeon 1: 565 scores for all of surgeon 1's patients (mean = 11.2513). Surgeon 6: 474 scores for all of surgeon 6's patients (mean = 5.5591). …”

“…Beiles et al , Harris et al , Sherlaw‐Johnson, Sherlaw‐Johnson et al , Novick et al , Moore et al , Sherlaw‐Johnson et al , Bottle and Aylin, Morton et al , Collins et al , and Chen et al used the risk‐adjusted CUSUM charts to assess or monitor clinical outcome performance for various applications. However, several researchers have brought up issues about the effect of different risk distributions on the performance of risk‐adjusted Bernoulli CUSUM charts (refer to Rogers et al ; Steiner et al ; Loke and Gan; Tian et al ). The in‐control average run lengths (ARLs) of risk‐adjusted CUSUM charts with the same risk adjustment model and constant control limits can vary by a factor of 10 for the highest‐risk and lowest‐risk patient populations.…”

Dynamic probability control limits for risk-adjusted Bernoulli CUSUM charts

“…The previous study of Tian et al found that the in‐control ARL of the Bernoulli risk‐adjusted CUSUM chart with fixed control limits varies considerably for different patient populations. Specifically, the in‐control ARLs decrease as the mean risk score of the patient population increases.…”

“…Because the Parsonnet score is the only explanatory variable in this logistic regression model to determine the probability of death, we can use the different Parsonnet score distributions to represent the different patient populations. We also use the same criteria as Tian et al to differentiate the Parsonnet score distributions. We randomly chose several sequences of 20,000 Parsonnet scores with replacement from each of the following five different Phase I risk distributions: All: 2218 scores for all patients (Mean=8.9026), High risk: the highest 50% of the scores (Mean=16.4813), Low risk: the lowest 50% of the scores (Mean=2.0541), Surgeon 1: 565 scores for all of surgeon 1's patients (Mean=11.2513), and Surgeon 6: 474 scores for all of surgeon 6's patients (Mean=5.5591). …”

Reduction of the Effect of Estimation Error on In-control Performance for Risk-adjusted Bernoulli CUSUM Chart with Dynamic Probability Control Limits

“…By maintaining the probability of a false alarm at a constant level conditional on no false alarm for previous observations, a concept initially proposed by Margavio et al , the charts with DPCLs give desirably consistent in‐control performance with approximately geometrically distributed in‐control run lengths. This method overcomes the disadvantage pointed out by Tian et al of using traditional constant control limits for RA‐CUSUM charts in that the effect of varying risk distributions of patients on the in‐control performance is significant . However, in practice, different Phase I data would provide different parameter estimates and therefore different risk‐adjustment models.…”