Causal Forecasting:Generalization Bounds for Autoregressive Models

Vankadara, Leena Chennuru; M., Faller, Philipp; Hardt, Michaela; Minorics, Lenon; Ghoshdastidar, Debarghya; Janzing, Dominik

doi:10.48550/arxiv.2111.09831

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is also empirical evidence that suggests that augmenting such techniques with additional explicit norm-based regularization may help in learning distributionally robust models in the overparameterized regime (Sagawa et al, 2020;Donhauser et al, 2021). In the context of causal learning, Janzing (2019); Vankadara et al (2021) suggest that explicit regularization may help improve causal generalization. Furthermore, Janzing ( 2019) conjectures generally that one may need to regularize more strongly for causal learning than for statistical learning.…”

Section: Motivation and Related Workmentioning

confidence: 99%

Interpolation and Regularization for Causal Learning

Vankadara¹,

Rendsburg²,

Luxburg³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of learning causal models from observational data through the lens of interpolation and its counterpart-regularization. A large volume of recent theoretical as well as empirical work suggests that, in highly complex model classes, interpolating estimators can have good statistical generalization properties and can even be optimal for statistical learning. Motivated by an analogy between statistical and causal learning recently highlighted by Janzing (2019), we investigate whether interpolating estimators can also learn good causal models. To this end, we consider a simple linearly confounded model and derive precise asymptotics for the causal risk of the min-norm interpolator and ridge-regularized regressors in the high-dimensional regime. Under the principle of independent causal mechanisms, a standard assumption in causal learning, we find that interpolators cannot be optimal and causal learning requires stronger regularization than statistical learning. This resolves a recent conjecture in Janzing (2019). Beyond this assumption, we find a larger range of behavior that can be precisely characterized with a new measure of confounding strength. If the confounding strength is negative, causal learning requires weaker regularization than statistical learning, interpolators can be optimal, and the optimal regularization can even be negative. If the confounding strength is large, the optimal regularization is infinite and learning from observational data is actively harmful.

show abstract

Section: Motivation and Related Workmentioning

confidence: 99%