Visually Communicating and Teaching Intuition for Influence Functions

Abstract

Estimators based on influence functions (IFs) have been shown to be effective in many settings, especially when combined with machine learning techniques. By focusing on estimating a specific target of interest (e.g., the average effect of a treatment), rather than on estimating the full underlying data generating distribution, IF-based estimators are often able to achieve asymptotically optimal mean-squared error. Still, many researchers find IF-based estimators to be opaque or overly technical, which makes their use less prevalent and their benefits less available. To help foster understanding and trust in IF-based estimators, we present tangible, visual illustrations of when and how IF-based estimators can outperform standard "plug-in" estimators. The figures we show are based on connections between IFs, gradients, linear approximations, and Newton-Raphson.

Parameterising the effect of a continuous exposure using average derivative effects

“…For our discussion of efficiency and estimation we restrict ourselves to the case where g(x, z) = m(x, z) is the conditional response function. We also draw heavily on nonparametric inference methods using influence curves (ICs), and recommend two recent tutorial papers for an introduction to these ideas (Hines et al, 2021;Fisher and Kennedy, 2020). In the nonparametric setting, an IC is a model-free, mean zero, functional of the true data distribution, which characterizes the sensitivity of an estimand to small changes in the data distribution.…”

Statistical Modelling in the Age of Data Science

“…In times where software is abundant and accessible, the focus of introductory statistics courses should be primarily on statistical reasoning, as made most clear by computational, simulation-based methods such as bootstrap, permutation tests, simulation-based sample size calculation, ..., on concepts of bias (e.g., selection bias, confounding bias) and imprecision, on the translation of scientific questions into statistical estimands, on key assumptions linked to study design (e.g., independence assumptions), on flexible (statistical or machine) learning methods for prediction, which form a cornerstone of the methods of Section 3, ... For the smaller minority of students who are mathematically-minded and/or wish to engage in methods development, a core training in asymptotic statistics obviously remains indispensable. I believe that a central focus of such training should lie on nonparametric estimation and efficiency theory (Bickel et al, 1993;Kennedy, 2016;Fisher and Kennedy, 2020).…”

Demystifying statistical learning based on efficient influence functions

“…This formalisation will be practically useful, as it will provide insight what the so-called efficient influence function (also referred to as canonical gradient, or influence curve) is, how it can be calculated, and why it is useful. As in Fisher and Kennedy (2020), we develop some intuition by first considering the special case of discrete data O with support {o 1 , ..., o k }. Then…”

“…is limiting (by being focussed on discrete data) and ignores that the probabilities P t (o 1 ), ..., P t (o k ) are not variation-independent (i.e., they sum to 1 and thus cannot be changed in arbitrary ways) (Fisher and Kennedy, 2020). We therefore appeal to Riesz's representation theorem, according to which this derivative, when it exists, can be obtained via integration of a unique 'representer' φ(O, P) with finite variance under P, w.r.t.…”