Machine Learning for Causal Inference: On the Use of Cross-fit Estimators

Zivich, Paul N; Breskin, Alexander

doi:10.1097/ede.0000000000001332

Cited by 51 publications

(37 citation statements)

References 45 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, even if TMLE is less prone to errors due to misspecification than alternative methods (eg, inverse probability weighting) there is some question regarding the validity of the robustness of inference produced by TMLE in nonparametric settings 49 . This is an area of ongoing work (ie, double/debiased machine learning, cross‐validated TMLE and cross‐fit estimators) 44,50,51 . Furthermore, TMLE and the SuperLearner were originally developed in R 35,46 .…”

Section: Discussionmentioning

confidence: 99%

“…Using these approaches may require larger sample sizes to avoid finite‐sample bias 16,54 . However, recent developments support the use of cross‐fit double‐robust estimators for data adaptive estimation 44,50 . Tutorials introducing the use and derivation of the functional delta method and influence curve for applied researchers are needed.…”

Section: Discussionmentioning

confidence: 99%

“…However, using the IF for statistical inference may require larger sample sizes to avoid finite‐sample issues. Recent research and theoretical developments support the use of double‐robust cross‐fit estimators to retain valid statistical inference when using machine learning algorithms that are non‐Donsker 44 . The computation of the IF is provided in Box 25 (Step 6: rows 38‐53).…”

Section: Data‐adaptive Estimation: Ensemble Learning Targeted Maximum Likelihood Estimationmentioning

confidence: 99%

See 2 more Smart Citations

Introduction to computational causal inference using reproducible Stata, R, and Python code: A tutorial

Smith

Mansournia

Maringe

et al. 2021

Statistics in Medicine

Self Cite

View full text Add to dashboard Cite

The main purpose of many medical studies is to estimate the effects of a treatment or exposure on an outcome. However, it is not always possible to randomize the study participants to a particular treatment, therefore observational study designs may be used. There are major challenges with observational studies; one of which is confounding. Controlling for confounding is commonly performed by direct adjustment of measured confounders; although, sometimes this approach is suboptimal due to modeling assumptions and misspecification. Recent advances in the field of causal inference have dealt with confounding by building on classical standardization methods. However, these recent advances have progressed quickly with a relative paucity of computational‐oriented applied tutorials contributing to some confusion in the use of these methods among applied researchers. In this tutorial, we show the computational implementation of different causal inference estimators from a historical perspective where new estimators were developed to overcome the limitations of the previous estimators (ie, nonparametric and parametric g‐formula, inverse probability weighting, double‐robust, and data‐adaptive estimators). We illustrate the implementation of different methods using an empirical example from the Connors study based on intensive care medicine, and most importantly, we provide reproducible and commented code in Stata, R, and Python for researchers to adapt in their own observational study. The code can be accessed at https://github.com/migariane/Tutorial_Computational_Causal_Inference_Estimators.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Data‐adaptive Estimation: Ensemble Learning Targeted Maximum Likelihood Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Introduction to computational causal inference using reproducible Stata, R, and Python code: A tutorial

Smith

Mansournia

Maringe

et al. 2021

Statistics in Medicine

Self Cite

View full text Add to dashboard Cite

show abstract

“…Curth and van der Schaar (2021) focus directly on meta-learning algorithms for estimation of heterogeneous treatment effects, but refrain from studying sample-splitting and cross-fitting procedures and rely fully on the full-sample estimation. In this regard, Zivich and Breskin (2021) study the performance of treatment effect estimators based on cross-fitting, including some meta-learners as well. Similarly to Knaus et al (2021) they find the DR-learner with an ensemble machine learning base learners together with cross-fitting to perform the best among all considered estimators, both in comparison to cases without cross-fitting and to parametric base learners.…”

Section: Related Workmentioning

confidence: 99%

“…Similarly to Knaus et al (2021) they find the DR-learner with an ensemble machine learning base learners together with cross-fitting to perform the best among all considered estimators, both in comparison to cases without cross-fitting and to parametric base learners. However, Zivich and Breskin (2021) study exclusively the estimation of average effects without examining convergence performance of the estimators, considering only a single sample size of 3 000 observations. 3 Recently, Jacob (2020) focuses on the estimation of heterogeneous treatment effects under various cross-fitting schemes for selected meta-learning algorithms.…”

Section: Related Workmentioning

confidence: 99%

Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance

Okasa¹

2022

Preprint

View full text Add to dashboard Cite

Estimation of causal effects using machine learning methods has become an active research field in econometrics. In this paper, we study the finite sample performance of meta-learners for estimation of heterogeneous treatment effects under the usage of sample-splitting and cross-fitting to reduce the overfitting bias. In both synthetic and semi-synthetic simulations we find that the performance of the meta-learners in finite samples greatly depends on the estimation procedure. The results imply that sample-splitting and cross-fitting are beneficial in large samples for bias reduction and efficiency of the meta-learners, respectively, whereas full-sample estimation is preferable in small samples. Furthermore, we derive practical recommendations for application of specific meta-learners in empirical studies depending on particular data characteristics such as treatment shares and sample size.

show abstract

Use of Machine Learning to Estimate the Per-Protocol Effect of Low-Dose Aspirin on Pregnancy Outcomes

et al. 2022

View full text Add to dashboard Cite

IMPORTANCEIn randomized clinical trials (RCTs), per-protocol effects may be of interest in the presence of nonadherence with the randomized treatment protocol. Using machine learning in per-protocol effect estimation can help avoid model misspecification owing to strong parametric assumptions, as is common with standard methods (eg, logistic regression).OBJECTIVES To demonstrate the use of ensemble machine learning with augmented inverse probability weighting (AIPW) for per-protocol effect estimation in RCTs and to evaluate the per-protocol effect size of aspirin on pregnancy. DESIGN, SETTING, AND PARTICIPANTSThis secondary analysis used data from 1227 women in the Effects of Aspirin in Gestation and Reproduction (EAGeR) trial, a multicenter, block-randomized, double-blind, placebo-controlled clinical trial of the effect of daily low-dose aspirin on pregnancy outcomes in women at high risk of pregnancy loss. Participants were recruited at 4 university medical centers in the US from June 15, 2007, to July 15, 2012. Women were followed up for 6 menstrual cycles for attempted pregnancy and 36 weeks of gestation if pregnancy occurred. Follow-up was completed on August 17, 2012. Data analyses were performed on July 9, 2021.EXPOSURES Daily low-dose (81 mg) aspirin taken at least 5 of 7 days per week for at least 80% of follow-up time relative to placebo. MAIN OUTCOMES AND MEASURESPregnancy detected using human chorionic gonadotropin (hCG) levels. RESULTS Among the 1227 women included in the analysis (mean SD age, 28.74 [4.80] years), 1161 (94.6%) were non-Hispanic White and 858 (69.9%) adhered to the protocol. Five machine learning models were combined into 1 meta-algorithm, which was used to construct an AIPW estimator for the per-protocol effect. Compared with adhering to placebo, adherence to the daily low-dose aspirin protocol for at least 5 of 7 days per week was associated with an increase in the probability of hCG-detected pregnancy of 8.0 (95% CI, 2.5-13.6) more hCG-detected pregnancies per 100 women in the sample, which is substantially larger than the estimated intention-to-treat estimate of 4.3 (95% CI, −1.1 to 9.6) more hCG-detected pregnancies per 100 women in the sample. CONCLUSIONS AND RELEVANCEThese findings suggest that a low-dose aspirin protocol is associated with increased hCG-detected pregnancy in women who adhere to treatment for at least 5 days per week. With the presence of nonadherence, per-protocol treatment effect estimates differ from intention-to-treat estimates in the EAGeR trial. The results of this secondary analysis of clinical (continued) Key Points Question How can machine learning be used to estimate per-protocol effects in randomized clinical trials? Findings In a cohort of 1227 women derived from secondary analysis of a randomized clinical trial, ensemble machine learning with augmented inverse probability weighting was used to estimate the per-protocol effect of daily low-dose aspirin on pregnancy detected using human chorionic gonadotropin (hCG) levels. Relative to placebo, adher...

show abstract

Machine Learning for Causal Inference: On the Use of Cross-fit Estimators

Cited by 51 publications

References 45 publications

Introduction to computational causal inference using reproducible Stata, R, and Python code: A tutorial

Introduction to computational causal inference using reproducible Stata, R, and Python code: A tutorial

Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance

Use of Machine Learning to Estimate the Per-Protocol Effect of Low-Dose Aspirin on Pregnancy Outcomes

Contact Info

Product

Resources

About