Interobserver variability impairs radiologic grading of primary graft dysfunction after lung transplantation

Abstract

Objectives: The current score for primary graft dysfunction after lung transplantation relies heavily on chest radiographs, and radiologic judgment can make the difference between the lowest (primary graft dysfunction 0) and the highest (primary graft dysfunction 3) grade. This study aimed to evaluate interobserver variability of the scoring of postoperative chest radiographs and its impact on primary graft dysfunction grades in a large single-center cohort.Methods: We retrospectively analyzed 497 lung transplantations performed between January 2010 and July 2016 at the Medical University of Vienna. Five trained thoracic radiologists were asked to independently examine postoperative chest radiographs performed at 0 to 6 hours, 24 hours, 48 hours, and 72 hours after arrival at the intensive care unit. Interobserver variability was calculated using Fleiss' kappa (k) statistics.Results: A total of 1988 chest radiographs were evaluated. Consensus among all 5 radiologists was found in only 826 cases (43.0%). At 0 to 6 hours and 24 hours, only a moderate agreement was found among the 5 radiologists (k ¼ 0.456 and 0.456, respectively), and agreement was even worse at 48 and 72 hours (k ¼ 0.405 and k ¼ 0.409). On the basis of this high interobserver variability, best and worst case scenarios were calculated leading to primary graft dysfunction 3 rates of 8.4% versus 28.4% at 0 to 6 hours, 1.8% versus 4.8% at 24 hours, 2.0% versus 5.3% at 48 hours, and 0.2% versus 3.1% at 72 hours. A high recipient body mass index and size-reduced transplants were found to be factors associated with higher rates of interobserver variability. Conclusions:The substantial interobserver variability found in this retrospective analysis underlines the difficulty to adequately grade post-transplant organ function. Future revisions of the primary graft dysfunction grading should take this problem into consideration.

Predictors of long intensive care need after lung transplantation

“…Of these, PAH, female recipient, high mPAP, and oversized lung allograft have also been shown to be risk factors for PGD. 9,10 In our univariate model, female gender increases the risk for prolonged ICU LOS significantly (OR 2.27).…”

“…This is consistent with findings in other studies, where the effect of gender disappears when correcting for other variables. 10 This suggests that the risk is primarily caused by other factors, such as lung size.…”

“…Primary graft dysfunction (PGD) has become a standardized short‐term research endpoint since its definition by the International Society for Heart and Lung Transplantation (ISHLT) in 2005 7 . The diagnostic criteria of PGD are reduced PaO 2 /FiO 2 ratio combined with radiographic infiltrates under the exclusion of other known causes, 8,9 but interobserver agreement in diagnosing PGD might be limited in clinical practice 10 . For example, radiologists may interpret diffuse infiltrates differently and it might be challenging to rule out “other causes” that may induce infiltrates and poor oxygenation.…”

Commentary: Evaluation of primary graft dysfunction after lung transplantation—It is time to teach an old dog new tricks!

“…Patients with an increased body mass index and those with size-reduced transplants were associated with higher rates of interobserver variability. Schwarz and colleagues 3 conclude that substantial interobserver variability underlines the difficulty to grade posttransplant organ function adequately.…”

Commentary: INSPIRE results? A critical appraisal of study end points

“…We have previously shown that even trained thoracic radiologists only moderately agree in defining bilateral infiltrations in the modern era of lung transplantation, especially if radiographic changes are subtle. 5 In case of a PO 2 less than 200 mm Hg, a chest x-ray diagnosis can make a difference between PGD grade 0 and PGD grade 3. We and others believe that alternative measurements-such as the ratio of PO 2 to inspired oxygen fraction, compliance, length of mechanical ventilation or best forced expiratory volume in 1 second-might be more useful as markers of lung quality and need to be provided in future studies evaluating the effects of organ preservation strategies on primary organ function.…”