On complementing end-to-end human behavior predictors with planning

Abstract

High capacity end-to-end approaches for human motion prediction have the ability to represent subtle nuances in human behavior, but struggle with robustness to out of distribution inputs and tail events. Planning-based prediction, on the other hand, can reliably output decentbut-not-great predictions: it is much more stable in the face of distribution shift, but it has high inductive bias, missing important aspects that drive human decisions, and ignoring cognitive biases that make human behavior suboptimal.In this work, we analyze one family of approaches that strive to get the best of both worlds: use the end-to-end predictor on common cases, but do not rely on it for tail events / out-of-distribution inputs -switch to the planningbased predictor there. We contribute an analysis of different approaches for detecting when to make this switch, using an autonomous driving domain. We find that promising approaches based on ensembling or generative modeling of the training distribution might not be reliable, but that there very simple methods which can perform surprisingly wellincluding training a classifier to pick up on tell-tale issues in predicted trajectories.

Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Prediction

“…However, it is prohibitive in general to train such a perfect model with offline datasets. Instead, a practical solution is to equip the prediction model with a module detecting outof-distribution (OOD) inputs of planned trajectories [24], [25], which can be utilized to prevent the planning module from exploiting the prediction model with those OOD inputs. Therefore, it is necessary to require such an OOD module for an IBP model and include the evaluation of OOD detection as a part of an IBP benchmark.…”

Efficient Game-Theoretic Planning with Prediction Heuristic for Socially-Compliant Autonomous Driving

“…Let Ŷi N = {ŷ i 1 , ..., ŷi N } and p i denote the i th predicted trajectory and its probability of being executed respectively. Therefore, the state at time step t can be modeled as a random distribution with mean ŷi t and variances Σ i t , where the variances Σ i t in the predicted trajectory can be calculated by methods discussed in [22], [23]. Thus, each predicted trajectory introduces a confidence range G i t at each time step by Mahalanobis distance, which is given as…”