Learning and confirming a class of treatment responders in clinical trials

Abstract

Clinical trials require substantial effort and time to complete, and regulatory agencies may require two successful efficacy trials before approving a new drug. One way to improve the chance of follow‐up success is to identify a subpopulation among whom treatment effects are estimated to be beneficial, and enrolling future studies from this subpopulation. In this article we study confirmable responder class (CRC) learning, where the objective is to learn in a random half of the dataset (training set) a subpopulation among whom the predicted conditional ATE (CATE) suggests clinically meaningful benefit, with maximum power of being confirmed via hypothesis test in the other half (test set). We studied a set of CRC learners across simulated datasets in which either all patients benefited, or only some did. Performance metrics included the rate of confirmation in the test set, and the classification accuracy of the CRC compared with the group with true CATE>0. In trials where all patients benefitted, only two learners were able to consistently identify most of the population as the CRC. One of these also performed especially well when only some patients benefitted, having relatively high confirmation rates, and showing robustness to the dimension of the covariate vector and population characteristics. This learner is based on cross‐validation, and is a possible avenue for further development of model selection criteria for CRC learning. We also show that the performance of all methods can be improved by using both halves of the original dataset as training and test sets in turn.

Finding the best subgroup with differential treatment effect with multiple outcomes

“…The use of K = 2 has been recommended in the literature. 27 Figure 2 summarizes the %U in the estimated best subgroup from subgroup identification methods across multiple-outcome settings. Model 0 is not included as %U under the null hypothesis is not informative.…”

“…Maximizing this utility is equivalent to maximizing the power to test for the CATE due to the exact correspondence between power and test statistics curves for subgroups defined by the minimum CATE threshold. 6,7,27 We extend this definition to the case of multiple outcomes. For the multiple-outcome setting, we define the best subgroup S true and the optimal weight vector w true as the pair that jointly maximizes the utility U(S, w) = w 1 U 1 (S) + … + w J U J (S) over all possible subgroups in the subgroup space and all possible weight vectors w = (w 1 , … , w J ) such that ∑ J j=1 w 2 j = 1.…”

Inference on subgroups identified based on a heterogeneous treatment effect in a post hoc analysis of a clinical trial

“…2,4,5 The selection criteria vary across the methods, depending on the predetermined targets-maximizing CATE, a utility function that takes into account the ''treatment burden,'' predictive power of future trials, or the value function of the optimal treatment assignment rule. 1,2,[4][5][6][7][8][9][10][11][12][13][14] In this article, we assume that a subgroup identification method has been previously selected for analysis and focus on the confirmatory phase. The goal of the subgroup confirmation phase is to obtain unbiased estimates and reliable inference of CATEs in the identified subgroups, also known as ''honest'' estimates.…”

Flexible machine learning estimation of conditional average treatment effects: a blessing and a curse

“…Whether the CATE is an appropriate proxy for the ITE depends on the remaining variability of causal effects given the considered modifiers, e.g. a CATE ≥ 0 given X = x (Talisa and Chang, 2021) does not imply that all ITEs ≥ 0 for those individuals (Hand, 1992). In this work, we investigate whether we can use a causal random forest (CRF) to estimate characteristics of the marginal ITE distribution.…”