Review of methods for measuring and comparing center performance after organ transplantation

Abstract

The assessment of outcomes after transplantation is important for several reasons: it provides patients with data so that they can make informed decisions about the benefits of transplantation and the success of the transplant unit; it informs commissioners that resources are allocated properly; and it provides clinicians reassurance that results are acceptable or, if they are not, provides early warning so that problems can be identified, corrections can be instituted early, and all interested parties can be reassured that scarce resources are used fairly. The need for greater transparency in reporting outcomes after liver transplantation and for comparisons both between and within centers has led to a number of approaches being adopted for monitoring center performance. We review some of the commonly used methods, highlight their strengths and weaknesses, and concentrate on methods that incorporate risk adjustment. Measuring and comparing outcomes after transplantation is complex, and there is no single approach that gives a complete picture. All those using analyses of outcomes must understand the merits and limitations of individual methods. When used properly, such methods are invaluable in ensuring that a scarce resource is used effectively, any adverse trend in outcomes is identified promptly and remedied, and best performers are identified; they thus allow the sharing of best practices. However, when they are used inappropriately, such measurements may lead to inappropriate conclusions, encourage risk-averse behavior, and discourage innovation.

Liver transplantation in the United Kingdom

“…NHSBT is responsible for monitoring outcomes after transplantation and does this through cumulative sum (CUSUM) monitoring and use of Funnel plots . When a center signals, NHSBT will seek an explanation from the unit.…”

“…NHSBT is responsible for monitoring outcomes after transplantation and does this through cumulative sum (CUSUM) monitoring and use of Funnel plots. (7) When a center signals, NHSBT will seek an explanation from the unit. The response is assessed by the associate medical director in NHSBT, the Chair of the Advisory group (or deputy if there is a conflict of interest) and a representative, which forms the commissioners.…”

Pro: Patient and allograft survival remain the best metric to gauge successful liver transplantation

“…PSR use has been implicated in the observation of continually improved outcomes among OLT recipients but may have contributed to a perception of risk aversion in donor and recipient selection . In an attempt to increase donor utilization and reduce wait‐list mortality, various modifications in PSR analyses have been proposed to create a more comprehensive transplant performance metric (CTPM) …”

Advancing Transplantation

“…Outcome measurement, therefore, requires risk adjustment, but this is a blunt tool and a failure to include all relevant data can produce misleading conclusions. 211 Overreliance on outcomes in transplantation may encourage risk-averse behavior. 211,212 Surgeons concerned about adverse outcomes might be more likely to avoid circumstances that could reduce their patients' survival rates, including the avoidance of high-risk donors and recipients, not letting lessexperienced colleagues and trainees develop expertise, and even a resistance to innovation.…”

“…211 Overreliance on outcomes in transplantation may encourage risk-averse behavior. 211,212 Surgeons concerned about adverse outcomes might be more likely to avoid circumstances that could reduce their patients' survival rates, including the avoidance of high-risk donors and recipients, not letting lessexperienced colleagues and trainees develop expertise, and even a resistance to innovation. Avoiding high-risk donors will increase the risk of death without a transplant, and death while awaiting a transplant is just as lethal as death posttransplant.…”