Accurate and Diverse Sampling of Sequences Based on a "Best of Many" Sample Objective

Abstract

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence. This problem has been formalized as a sequence extrapolation problem, where a number of observations are used to predict the sequence into the future. Real-world scenarios demand a model of uncertainty of such predictions, as predictions become increasingly uncertainin particular on long time horizons. While impressive results have been shown on point estimates, scenarios that induce multi-modal distributions over future sequences remain challenging. Our work addresses these challenges in a Gaussian Latent Variable model for sequence prediction. Our core contribution is a "Best of Many" sample objective that leads to more accurate and more diverse predictions that better capture the true variations in real-world sequence data. Beyond our analysis of improved model fit, our models also empirically outperform prior work on three diverse tasks ranging from traffic scenes to weather data.

Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

“…They showed that minimizing the RWTA loss is able to capture the possible futures for a car approaching a road crossing, i.e., going straight, turning left, and turning right. Bhattacharyya et al [3] set up this optimization within an LSTM network for future location prediction. Despite capturing the future locations, these works do not provide the whole distribution over the possible locations.…”

“…Also statistical information on how bicyclists moved in the past in this round- Figure 1: Given the past images, the past positions of an object (red boxes), and the experience from the training data, the approach predicts a multimodal distribution over future states of that object (visualized by the overlaid heatmap). The bicyclist is most likely to move straight (1), but could also continue on the roundabout (2) or turn right (3). about and potentially subtle cues like the orientation of the bicycle and its speed can indicate where a bicyclist is more likely to go.…”

Introducing Probabilistic Bézier Curves for N-Step Sequence Prediction

“…Recent approaches actually targeting multi-modal prediction are given by e.g. (Bhattacharyya, Schiele, and Fritz 2018) and (Hug et al 2018).…”

Multi-modal Probabilistic Prediction of Interactive Behavior via an Interpretable Model

“…[15] applied the Hidden Markov Model (HMM) to predict the trajectories for individual driver. [16] combined CNN and LSTM to predict multi-modal trajectories for an agent on a bird-view image. The main limitation of these works, however, is that they only predict motions for one selected agent without considering the influence of other agents with potential interactions.…”