Estimating the quality of optimal treatment regimes

Sies, Aniek; Mechelen, Iven Van

doi:10.1002/sim.8342

Cited by 4 publications

(4 citation statements)

References 43 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This issue will go with questions on the quality of the estimator that may be picked for this purpose. Otherwise, further advancing this issue, also making correct inferences about the quality on the level of the population of an estimated OTR (Sies & Van Mechelen, 2019) (as well as about the difference in such quality between different estimated OTR's) is a critical inferential issue, which, in part, is an open problem that may be considered a challenge for further statistical research.…”

Section: Discussionmentioning

confidence: 99%

Reflections on the concept of optimality of single decision point treatment regimes

Tran,

Abad,

Verbeke

et al. 2023

Biometrical J

Self Cite

View full text Add to dashboard Cite

In many areas, applied researchers as well as practitioners have to choose between different solutions for a problem at hand; this calls for optimal decision rules to settle the choices involved. As a key example, one may think of the search for optimal treatment regimes (OTRs) in clinical research, that specify which treatment alternative should be administered to each patient under study. Motivated by the fact that the concept of optimality of decision rules in general and treatment regimes in particular has received so far relatively little attention and discussion, we will present a number of reflections on it, starting from the basics of any optimization problem. Specifically, we will analyze the search space and the to be optimized criterion function underlying the search of single decision point OTRs, along with the many choice aspects that show up in their specification. Special attention is paid to formal characteristics and properties as well as to substantive concerns and hypotheses that may guide these choices. We illustrate with a few empirical examples taken from the literature. Finally, we discuss how the presented reflections may help sharpen statistical thinking about optimality of decision rules for treatment assignment and to facilitate the dialogue between the statistical consultant and the applied researcher in search of an OTR.

show abstract

Section: Discussionmentioning

confidence: 99%

Reflections on the concept of optimality of single decision point treatment regimes

Tran,

Abad,

Verbeke

et al. 2023

Biometrical J

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, for the purpose of building a decision rule for allocating patients to their optimal treatment, it is important to evaluate the benefit of such an allocation (e.g., compared with a random allocation). To this end, several estimators have been developed (Sies & Mechelen, 2019) and call for evaluation within the frame of psychotherapy research.…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

Covariate Selection for Estimating Individual Treatment Effects in Psychotherapy Research: A Simulation Study and Empirical Example

Wester

Rubel

Mayer

2022

Clinical Psychological Science

View full text Add to dashboard Cite

Estimating individual treatment effects (ITEs) is crucial to personalized psychotherapy. It depends on identifying all covariates that interact with treatment, a challenging task considering the many patient characteristics hypothesized to influence treatment outcome. The goal of this study was to compare different covariate-selection strategies and their consequences on estimating ITEs. A Monte Carlo simulation was conducted to compare stepwise regression with and without cross-validation and shrinkage methods. The study was designed to mimic the setting of psychotherapy studies. No single covariate-selection strategy dominated all others across all factor-level combinations and on all performance measures. The least absolute shrinkage and selection operator showed the most accurate out-of-sample predictions, identified the highest number of true treatment-covariate interactions, and estimated ITEs with the highest precision across the most conditions. Domain backward stepwise regression and backward stepwise regression using Bayesian information criterion were least biased in estimating variance of ITEs across the most conditions.

show abstract

“…We refer the reader to [14] and [16] for a discussion of considerations and best practices when implementing the ODTR SuperLearner. Obtaining inference for the mean outcome under the ODTR has been shown to be difficult due to its lack of smoothness [6,9,29]; however, several methods have been proposed for constructing valid confidence intervals for this parameter, such as re-sampling techniques [6,30,31]. One approach to inference is to rely on parametric models; however, misspecification of these models can bias results.…”

Section: Introductionmentioning

confidence: 99%

“…Both [27] and [17] found that, indeed, there exists a positive finite sample bias when using TMLE versus CV-TMLE when estimating the value of the ODTR; in other words, with the rule learned and evaluated on the same data, estimates of the value of the rule may be optimistic, and CV-TMLE corrects this bias. Additionally, recently, [34] showed that cross-validation techniques for estimating the value of the rule, and in particular CV-TMLE, yielded a smaller difference between the true expected value under the true rule and its estimate, versus, for example, bootstrap techniques for evaluating a rule.…”

Section: Introductionmentioning

confidence: 99%

Performance and Application of Estimators for the Value of an Optimal Dynamic Treatment Rule

Montoya¹,

Skeem²,

Laan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Given an (optimal) dynamic treatment rule, it may be of interest to evaluate that rule -that is, to ask the causal question: what is the expected outcome had every subject received treatment according to that rule? In this paper, we study the performance of estimators that approximate the true value of: 1) an a priori known dynamic treatment rule 2) the true, unknown optimal dynamic treatment rule (ODTR); 3) an estimated ODTR, a so-called "data-adaptive parameter," whose true value depends on the sample. Using simulations of point-treatment data, we specifically investigate: 1) the impact of increasingly data-adaptive estimation of nuisance parameters and/or of the ODTR on performance; 2) the potential for improved efficiency and bias reduction through the use of semiparametric efficient estimators; and, 3) the importance of sample splitting based on CV-TMLE for accurate inference. In the simulations considered, there was very little cost and many benefits to using the cross-validated targeted maximum likelihood estimator (CV-TMLE) to estimate the value of the true and estimated ODTR; importantly, and in contrast to non cross-validated estimators, the performance of CV-TMLE was maintained even when highly data-adaptive algorithms were used to estimate both nuisance parameters and the ODTR. In addition, we apply these estimators for the value of the rule to the "Interventions" Study, an ongoing randomized controlled trial, to identify whether assigning cognitive behavioral therapy (CBT) to criminal justice-involved adults with mental illness using an ODTR significantly reduces the probability of recidivism, compared to assigning CBT in a non-individualized way.

show abstract

Estimating the quality of optimal treatment regimes

Cited by 4 publications

References 43 publications

Reflections on the concept of optimality of single decision point treatment regimes

Reflections on the concept of optimality of single decision point treatment regimes

Covariate Selection for Estimating Individual Treatment Effects in Psychotherapy Research: A Simulation Study and Empirical Example

Performance and Application of Estimators for the Value of an Optimal Dynamic Treatment Rule

Contact Info

Product

Resources

About