Interpreting change from patient reported outcome (PRO) endpoints: patient global ratings of concept versus patient global ratings of change, a case study among osteoporosis patients

Abstract

BackgroundRegulatory guidance recommends anchor-based methods for interpretation of treatment effects measured by PRO endpoints. Methodological pros and cons of patient global ratings of change vs. patient global ratings of concept have been discussed but empirical evidence in support of either approach is lacking. This study evaluated the performance of patient global ratings of change and patient global ratings of concept for interpreting patient stability and patient improvement.MethodsPatient global ratings of change and patient global ratings of concept were included in a psychometric validation study of an osteoporosis-targeted PRO instrument (the OPAQ-PF) to assess its ability to detect change and to derive responder definitions. 144 female osteoporosis patients with (n = 37) or without (n = 107) a recent (within 6 weeks) fragility fracture completed the OPAQ-PF and global items at baseline, 2 weeks (no recent fracture), and 12 weeks (recent fracture) post-baseline.ResultsResults differed between the two methods. Recent fracture patients reported more improvement while patients without recent fracture reported more stability on ratings of change than ratings of concept. However, correlations with OPAQ-PF score change were stronger for ratings of concept than ratings of change (both groups). Effect sizes for OPAQ-PF score change increased consistently with level of change in ratings of concept but inconsistently with ratings of change, with the mean AUC for prediction of a one-point change being 0.72 vs. 0.56.ConclusionsThis study provides initial empirical support for methodological and regulatory recommendations to use patient global ratings of concept rather than ratings of change when interpreting change captured by PRO instruments in studies evaluating treatment effects. These findings warrant being confirmed in a purpose-designed larger scale analysis.Electronic supplementary materialThe online version of this article (doi:10.1186/s12955-016-0427-5) contains supplementary material, which is available to authorized users.

Psychometric evaluation of the NTDT-PRO questionnaire for assessing symptoms in patients with non-transfusion-dependent beta-thalassaemia

“…Although the NTDT-PRO T/W and SoB domains were shown to be responsive to changes over time on all the anchors examined in the responsiveness analysis, PGI-C scores had the weakest correlation (0.28) with change in T/W domain score at weeks 13–24 among the included anchors. The weaker correlation between the NTDT-PRO domain score changes and the PGI-C as compared with other potential anchors may be due to an issue with recall: it may have been difficult for patients to rate how much their overall thalassaemia symptoms—which can be many—had changed in the 24 weeks since the beginning of the study 27 28…”

The minimal detectable difference for endurance shuttle walk test performance in people with COPD on completion of a program of high-intensity ground-based walking

“…This resulted in few people reporting hardly any change in their walking ability on completion of the training program. Further, difficulty with using the GRC scale may result from the long time between measurements taken before and after an intervention such as exercise training, and day to day variability in symptoms [23]. That is, accurate use of the GRC scale constitutes a somewhat challenging cognitive process for participants, and assumes that people can truly recall their prior health status [12].…”

“…That is, accurate use of the GRC scale constitutes a somewhat challenging cognitive process for participants, and assumes that people can truly recall their prior health status [12]. To overcome some of these limitations, recent work has suggested that rather than using a GRC scale, the use of a global rating of concept scale improves the capacity to detect change [23]. The method requires people to rate their current state before and after an intervention, and the change in these global concept ratings are used as the anchor for analysis [23].…”

“…To overcome some of these limitations, recent work has suggested that rather than using a GRC scale, the use of a global rating of concept scale improves the capacity to detect change [23]. The method requires people to rate their current state before and after an intervention, and the change in these global concept ratings are used as the anchor for analysis [23]. This method reduces the impact recall bias may have on the estimate of the MDD, and should be considered in future studies in which people are asked to perceive changes in an outcome measure following an intervention which lasts for several weeks, such as exercise training programs.…”

German version of the Chedoke McMaster arm and hand activity inventory (CAHAI-G): intra-rater reliability and responsiveness

“…The patients hereby estimated the activity of the affected upper extremities in the last 7 days. For this purpose, following Nixon et al [ 18 ], it was asked: “In the last seven days, how hard has it been for you to cope with everyday tasks (such as dressing, grooming)?” This was answered based on the following 5 point scale of responses: (5) not difficult at all, (4) somewhat difficult, (3) rather difficult, (2) very difficult, (1) not possible at all. The possible responses were taken from the subscale for the manual function of the Stroke Impact Scale [ 19 ].…”