Learning Human Driving Behaviors with Sequential Causal Imitation Learning

Abstract

Learning human driving behaviors is an efficient approach for self-driving vehicles. Traditional Imitation Learning (IL) methods assume that the expert demonstrations follow Markov Decision Processes (MDPs). However, in reality, this assumption does not always hold true. Spurious correlation may exist through the paths of historical variables because of the existence of unobserved confounders. Accounting for the latent causal relationships from unobserved variables to outcomes, this paper proposes Sequential Causal Imitation Learning (SeqCIL) for imitating driver behaviors. We develop a sequential causal template that generalizes the default MDP settings to one with Unobserved Confounders (MDPUC-HD). Then we develop a sufficient graphical criterion to determine when ignoring causality leads to poor performances in MDPUC-HD. Through the framework of Adversarial Imitation Learning, we develop a procedure to imitate the expert policy by blocking π-backdoor paths at each time step. Our methods are evaluated on a synthetic dataset and a real-world highway driving dataset, both demonstrating that the proposed procedure significantly outperforms non-causal imitation learning methods.

A Self-Labeling Method for Adaptive Machine Learning by Interactive Causality

“…Progress in statistical causality, such as Granger Causality (GC) and Structural Causal Models (SCM), have formalized causality testing, representation, and analysis with mathematical tools. Recently, ML algorithms have been used in conjunction with statistical causal representations for the causal analysis, such as in multi-domain causal structural learning [38], causal imitation learning [39], causal discovery [40], and causal inference by graphical models [41]- [43]. Schölkopf [10] looked conversely into how causality can be used in ML, especially semi-supervised learning, to enhance robustness by leveraging cross-domain invariant causal mechanisms.…”

Data-Driven Policy Learning Methods from Biological Behavior: A Systematic Review

“…The traditional CPL framework for SCM [69][70][71][72][73] first defines the SCM M as {U, V, F, P(u)} to capture the causal relationship between the variables of interest. It divides the endogenous variable V into observed variables O and latent variables L, where O ⊆ V and L = V \ O.…”

“…CPL performed well on several synthetic datasets, including highway-driving vehicle trajectories (Figure 3), MNIST digits [69,73], and visual navigation tasks [76]. They also exhibit promising applications in autonomous driving and industrial automation.…”

“…IRL and GAIL model drivers' driving preferences from driver data, which enables optimal driving strategies in different driving situations (e.g., car following, lane changing, and overtaking) [2,35,43,44]. In addition, safety is also an important factor, as CPL improves the safety of autonomous driving strategies by modeling the causal relationship between variables such as speed and distance to the vehicle in front [69,73]. (2) Robot control: Robot control applications, including robot manipulation, motion control, and navigation, are discussed.…”

Intend-Wait-Perceive-Cross: Exploring the Effects of Perceptual Limitations on Pedestrian Decision-Making

“…Top-down (or bird's eye) views common in the autonomous driving datasets provide information on the dynamics of the agents but not their awareness or decisionmaking process. The latter may be captured implicitly, however, such unobserved variables cannot be effectively learned as recent findings indicate [6]. Likewise, training on synthetic data with similar characteristics would increase sim-real gap, therefore simulations would greatly benefit from integration of more psychologically-plausible elements [7].…”