Operational characteristics of generalized pairwise comparisons for hierarchically ordered endpoints

Abstract

The method of generalized pairwise comparisons (GPC) is a multivariate extension of the well-known non-parametric Wilcoxon-Mann-Whitney test. It allows comparing two groups of observations based on multiple hierarchically ordered endpoints, regardless of the number or type of the latter. The summary measure, "net benefit," quantifies the difference between the probabilities that a random observation from one group is doing better than an observation from the opposite group. The method takes into account the correlations between the endpoints. We have performed a simulation study for the case of two hierarchical endpoints to evaluate the impact of their correlation on the type-I error probability and power of the test based on GPC. The simulations show that the power of the GPC test for the primary endpoint is modified if the secondary endpoint is included in the hierarchical GPC analysis. The change in power depends on the correlation between the endpoints. Interestingly, a decrease in power can occur, regardless of whether there is any marginal treatment effect on the secondary endpoint. It appears that the overall power of the hierarchical GPC procedure depends, in a complex manner, on the entire variance-covariance structure of the set of outcomes. Any additional factors (such as thresholds of clinical relevance, drop out, or censoring scheme) will also affect the power and will have to be taken into account when designing a trial based on the hierarchical GPC procedure.

Impact of correlations between prioritized outcomes on the net benefit and its estimate by generalized pairwise comparisons

“…When analyzed simultaneously, several authors have claimed that the correlation between efficacy and safety should be taken into account 4,6,7 . Note that in this paper, positive correlations refer to the tendency of two favorable outcomes to coexist unless otherwise specified.…”

“…The net benefit can be described as the probability that a subject in the experimental arm has a more favorable overall outcome than one in the control arm minus the probability of the opposite occurring. Previous studies have provided examples of different correlation settings leading to different values of the net benefit and estimates from generalized pairwise comparisons 6‐8 . However, the direction and magnitude of this impact and their theoretical foundations are not well understood, especially with different correlation values among treatment arms.…”

“…[3][4][5] When analyzed simultaneously, several authors have claimed that the correlation between efficacy and safety should be taken into account. 4,6,7 Note that in this paper, positive correlations refer to the tendency of two favorable outcomes to coexist unless otherwise specified. If efficacy and safety are positively correlated, patients who experience efficacy, compared to those who do not, are less likely to suffer from toxicity.…”

“…Previous studies have provided examples of different correlation settings leading to different values of the net benefit and estimates from generalized pairwise comparisons. [6][7][8] However, the direction and magnitude of this impact and their theoretical foundations are not well understood, especially with different correlation values among treatment arms. In addition, despite the possible bias with censored survival data, the impact on the estimated values has not been fully differentiated from the impact on the true values.…”

Generalized pairwise comparisons of prioritized outcomes are a powerful and patient-centric analysis of multi-domain scores

“…While in our simulations the prioritized approach was generally superior to the non-prioritized GPC, caution should be exercised when generalizing our results: the power of the prioritized GPC will not always be higher than that of the nonprioritized approach. The power of the prioritized GPC can be further in uenced not only by marginal treatment effects, but also by correlations between the individual outcomes [7]. Our simulations also showed that the power of the non-prioritized approach becomes more similar to the power of the prioritized approach for high threshold values.…”