Demystifying statistical learning based on efficient influence functions

Hines, Oliver; Dukes, Oliver; Diaz-Ordaz, Karla; Vansteelandt, Stijn

doi:10.48550/arxiv.2107.00681

Cited by 2 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semiparametric theory and, in particular, the concept of Influence Functions (IFs), are known to be challenging to assimilate [18,28,40]. Here we attempt to provide a brief, top-level intuition, but a detailed exposition lies beyond the scope of this paper (more details are included in the supplementary).…”

Section: Influence Functionsmentioning

confidence: 99%

“…Here we attempt to provide a brief, top-level intuition, but a detailed exposition lies beyond the scope of this paper (more details are included in the supplementary). Interested readers are encouraged to consider introductions by [18,28,35,62].…”

Section: Influence Functionsmentioning

confidence: 99%

“…To better illustrate these approaches, we consider the ATE to be our chosen target estimand, the IF for which is derived in [28] as well as supplementary and is given below. For this derivation, we assume t = t * ∈ {0, 1} such that the target estimand becomes the potential outcome under intervention t = t * :…”

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

“…In the case of the ATE, the IF which is given in Eq. ( 6), this yields the augmented inverse propensity weighted (AIPW) estimator [28,39,44].…”

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

“…Furthermore, in cases where both models are well-specified, we achieve efficient estimation. For more technical details on the double robustness property see [28,39,65].…”

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

See 4 more Smart Citations

A Free Lunch with Influence Functions? Improving Neural Network Estimates with Concepts from Semiparametric Statistics

Vowels¹,

Akbari²,

Etesami³

et al. 2022

Preprint

View full text Add to dashboard Cite

Parameter estimation in the empirical fields is usually undertaken using parametric models, and such models are convenient because they readily facilitate statistical inference. Unfortunately, they are unlikely to have a sufficiently flexible functional form to be able to adequately model realworld phenomena, and their usage may therefore result in biased estimates and invalid inference. Unfortunately, whilst non-parametric machine learning models may provide the needed flexibility to adapt to the complexity of realworld phenomena, they do not readily facilitate statistical inference, and may still exhibit residual bias. We explore the potential for semiparametric theory (in particular, the Influence Function) to be used to improve neural networks and machine learning algorithms in terms of (a) improving initial estimates without needing more data (b) increasing the robustness of our models, and (c) yielding confidence intervals for statistical inference. We propose a new neural network method 'MultiNet', which seeks the flexibility and diversity of an ensemble using a single architecture. Results on causal inference tasks indicate that MultiNet yields better performance than other approaches, and that all considered methods are amenable to improvement from semiparametric techniques under certain conditions. In other words, with these techniques we show that we can improve existing neural networks for 'free', without needing more data, and without needing to retrain them. Finally, we provide the expression for deriving influence functions for estimands from a general graph, and the code to do so automatically.

show abstract

Section: Influence Functionsmentioning

confidence: 99%

Section: Influence Functionsmentioning

confidence: 99%

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

“…In the case of the ATE, the IF which is given in Eq. ( 6), this yields the augmented inverse propensity weighted (AIPW) estimator [28,39,44].…”

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

“…Furthermore, in cases where both models are well-specified, we achieve efficient estimation. For more technical details on the double robustness property see [28,39,65].…”

Section: Updating/debiasing Our Estimators With Ifsmentioning

confidence: 99%

See 3 more Smart Citations

A Free Lunch with Influence Functions? Improving Neural Network Estimates with Concepts from Semiparametric Statistics

Vowels¹,

Akbari²,

Etesami³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

A Causal Research Pipeline and Tutorial for Psychologists and Social Scientists

Vowels¹

2022

Preprint

View full text Add to dashboard Cite

Causality is a fundamental part of the scientific endeavour to understand the world. Unfortunately, causality is still taboo in much of psychology and social science. Motivated by a growing number of recommendations for the importance of adopting causal approaches to research, we reformulate the typical approach to research in psychology to harmonize inevitably causal theories with the rest of the research pipeline. We present a new process which begins with the incorporation of techniques from the confluence of causal discovery and machine learning for the development, validation, and transparent formal specification of theories. We then present methods for reducing the complexity of the fully specified theoretical model into the fundamental submodel relevant to a given target hypothesis. From here, we establish whether or not the quantity of interest is estimable from the data, and if so, propose the use of semi-parametric machine learning methods for the estimation of causal effects. The overall goal is the presentation of a new research pipeline which can (a) facilitate scientific inquiry compatible with the desire to test causal theories (b) encourage transparent representation of our theories as unambiguous mathematical objects, (c) to tie our statistical models to specific attributes of the theory, thus reducing under-specification problems frequently resulting from the theory-to-model gap, and (d) to yield results and estimates which are causally meaningful and reproducible. The process is demonstrated through didactic examples with real-world data, and we conclude with a summary and discussion of limitations.

show abstract

Demystifying statistical learning based on efficient influence functions

Cited by 2 publications

References 24 publications

A Free Lunch with Influence Functions? Improving Neural Network Estimates with Concepts from Semiparametric Statistics

A Free Lunch with Influence Functions? Improving Neural Network Estimates with Concepts from Semiparametric Statistics

A Causal Research Pipeline and Tutorial for Psychologists and Social Scientists

Contact Info

Product

Resources

About