Super-Learning of an Optimal Dynamic Treatment Rule

Abstract

We consider the estimation of an optimal dynamic two time-point treatment rule defined as the rule that maximizes the mean outcome under the dynamic treatment, where the candidate rules are restricted to depend only on a user-supplied subset of the baseline and intermediate covariates. This estimation problem is addressed in a statistical model for the data distribution that is nonparametric, beyond possible knowledge about the treatment and censoring mechanisms. We propose data adaptive estimators of this optimal dynamic regime which are defined by sequential loss-based learning under both the blip function and weighted classification frameworks. Rather than a priori selecting an estimation framework and algorithm, we propose combining estimators from both frameworks using a super-learning based cross-validation selector that seeks to minimize an appropriate cross-validated risk. The resulting selector is guaranteed to asymptotically perform as well as the best convex combination of candidate algorithms in terms of loss-based dissimilarity under conditions. We offer simulation results to support our theoretical findings.

Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy

“…Before estimating the optimal value, one typically estimates the optimal rule. Recently, researchers have suggested applying machine learning algorithms to estimate the optimal rules from large classes which cannot be described by a finite dimensional parameter [see, e.g., Zhang et al (2012b), Zhao et al (2012), Luedtke and van der Laan (2014)].…”

Targeted sequential design for targeted learning inference of the optimal treatment rule and its mean reward

“…Another reason is that the mean reward under the optimal TR seen as a functional, Ψ, is pathwise differentiable at Q 0 if and only if, Q 0 -almost surely, either | q Y, 0 ( W )| > 0 or the conditional distributions of Y given ( A = 1 ,W ) and ( A = 0 ,W ) under Q 0 are degenerated [19, Theorem 1]. This explains why it is also assumed that the true law is not exceptional in [34, 18, 20]. Other approaches have been considered to circumvent the need to make this assumption: relying on m -out-of- n bootstrap [4] (at the cost of a $\sqrt{m}=o(\sqrt{n})$-rate of convergence and need to fine-tune m ), or changing the parameter of interest by focusing on the mean reward under the optimal TR conditional on patients for whom the best treatment has a clinically meaningful effect (truncation) [12, 16, 17].…”

“…The estimation of the optimal TR from i.i.d. observations has been studied extensively, with a recent interest in the use of machine learning algorithms to reach this goal [24, 36, 37, 34, 35, 28, 20]. In contrast, we estimate the optimal TR (and its mean reward) based on sequentially sampled dependent observations by empirical risk minimization over sample-size-dependent classes of candidate estimates with a complexity controlled in terms of uniform entropy integral.…”

Discussion of Identification, Estimation and Approximation of Risk under Interventions that Depend on the Natural Value of Treatment Using Observational Data, by Jessica Young, Miguel Hernán, and James Robins

“…In van der Laan and Luedtke (2014b), our estimator d n is based on a highly data adaptive super-learner of d 0 developed in Luedtke and van der Laan (2014), so that one might be concerned that the Donsker class condition on d n might be violated theoretically or negatively affect the finite sample coverage of the confidence interval for E 0 Y d n . To deal with this challenge, in van der Laan et al (2013) and van der Laan (2013), van der Laan and Luedtke (2014b) we started a general theory for estimation and inference for data adaptive parameters, such as theorem 2 in van der Laan et al (2013) that avoids any conditions on the estimator ${\hat{g}}^{\ast }$, beyond convergence to some fixed g * .…”