Interrater Reliability of Clinical Grading Measures for Cutaneous Chronic Graft-vs-Host Disease

Abstract

IMPORTANCE Cutaneous chronic graft-vs-host disease (cGVHD) is common after allogeneic hematopoietic stem cell transplant and is often associated with poor patient outcomes. A reliable and practical method for assessing disease severity and response to therapy among these patients is urgently needed. OBJECTIVE To evaluate the interrater agreement and reliability of skin-specific and range of motion (ROM) variables of the 2014 National Institutes of Health (NIH) response criteria for cGVHD and a skin sclerosis grading scale (SSG). DESIGN, SETTING, AND PARTICIPANTSIn this observational study performed at a single tertiary academic center, 6 academic blood and marrow transplant specialists and 4 medical dermatologists examined 8 patients with diagnosed cutaneous cGVHD on July 10, 2015. The patient cohort was enriched for patients with sclerotic features. Each patient was evaluated by using the skin-specific and ROM criteria of the 2014 NIH response criteria for cGVHD and an SSG ranging from 0 to 3. Each patient was also asked to complete quality-of-life scoring instruments. Interrater agreement and reliability were estimated by calculating the Krippendorff α and Cohen κ statistics. Data were analyzed from September 29, 2015, through November 22, 2018.MAIN OUTCOMES AND MEASURES Estimation of interrater agreement by interclass coefficient (Krippendorff α and Cohen κ statistics) for the skin-specific and ROM components of the 2014 NIH Response Criteria for Chronic GVHD and for the SSG. RESULTSThe median age of the patients evaluated was 54 years (range, 46-58 years). Patients were predominantly male (6 [75%]). Six of the 8 patients had a predominantly sclerotic cutaneous phenotype. Interrater agreement among our experts was acceptable for NIH skin feature score (0.68; 95% CI, 0.30-0.86) and good for NIH ROM scoring (0.80; 95% CI, 0.68-0.86). Dermatologists had acceptable agreement for NIH skin GVHD score (0.69; 95% CI, 0.25-0.82) and skin feature score (0.78; 95% CI, 0.17-0.98), good agreement in ROM grading (0.85; 95% CI, 0.69-0.90), and near perfect agreement in identifying sclerosis (0.82; 95% CI, 0.27-0.97). CONCLUSIONS AND RELEVANCEAlthough dermatologists had acceptable agreement in NIH skin GVHD score and skin features score, near perfect agreement in identifying cutaneous sclerosis, better agreement in grading severity of cutaneous cGVHD, especially in the intermediate grades, appears to be needed.

Decoding Skin Involvement in Chronic Graft-vs-Host Disease

“…Most importantly, specific details on the various types of skin involvement were not provided. Because most patients had skin sclerosis, the very poor agreement in scoring of the subtypes of epidermal skin involvement, seen in Table 2 of the study, may reflect the small sample size of certain epidermal features or overlapping clinical features in some patients. Furthermore, the clinical subtype of skin disease greatly influences the ability to accurately score skin GVHD by BSA.…”

“…In the study by Cardones and colleagues published in this issue of JAMA Dermatology , the authors take a step toward improving GVHD skin assessment by testing the interrater reliability of skin grading between transplant physicians and dermatologists at their institution. A panel of 6 blood and marrow transplant (BMT) specialists and 4 medical dermatologists evaluated a cohort of 8 patients (6 with skin sclerosis) according to the current NIH Consensus Response Criteria Working Group recommendations for grading of skin GVHD, as well as range of motion assessment for joint and fascial GVHD .…”

Patient-Reported Outcomes and Mortality in Cutaneous Chronic Graft-vs-Host Disease

“…The current reference-standard response measure used for therapy approval in clinical trials is the NIH Skin Score, which correlates with clinician-reported perception of change and survival outcomes . However, studies also demonstrate that the NIH Skin Score can be subject to poor inter-rater reliability and responsiveness to change . PROs are increasingly found to be discordant, and thus provide complementary information to clinical assessment measures .…”

Quantifying Skin Stiffness in Graft-Versus-Host Disease, Morphea, and Systemic Sclerosis Using Acoustic Radiation Force Impulse Imaging and Shear Wave Elastography

“…Clinical outcome measures for cutaneous fibrosing disorders have limitations in the detection of early disease, prediction of outcome, or evaluation of response to therapy (Aki et al, 2016;Baird et al, 2013;Kelsey and Torok, 2013). Inter-rater agreement can be poor, even among specialists (Cardones et al, 2019;Clements et al, 1995). Therefore, a great need .…”