Imitative Non-Autoregressive Modeling for Trajectory Forecasting and Imputation

Abstract

Trajectory forecasting and imputation are pivotal steps towards understanding the movement of human and objects, which are quite challenging since the future trajectories and missing values in a temporal sequence are full of uncertainties, and the spatial-temporally contextual correlation is hard to model. Yet, the relevance between sequence prediction and imputation is disregarded by existing approaches. To this end, we propose a novel imitative nonautoregressive modeling method to simultaneously handle the trajectory prediction task and the missing value imputation task. Specifically, our framework adopts an imitation learning paradigm, which contains a recurrent conditional variational autoencoder (RC-VAE) as a demonstrator, and a non-autoregressive transformation model (NART) as a learner. By jointly optimizing the two models, RC-VAE can predict the future trajectory and capture the temporal relationship in the sequence to supervise the NART learner. As a result, NART learns from the demonstrator and imputes the missing value in a non-autoregressive strategy. We conduct extensive experiments on three popular datasets, and the results show that our model achieves state-of-the-art performance across all the datasets.

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

“…For instance, some deep autoregressive methods based on RNNs [8,10,35,75] are proposed to impute the sequential data. Some other methods [18,37,42,47,58,74] have been proposed to leverage GANs or VAEs to generate reconstructed sequences.…”

“…However, these two methods only focus on the trajectory imputation task and fail to investigate the prediction task. In work INAM [47], an imitation learning paradigm is proposed to handle the imputation and prediction in an asynchronous mode. While our model handles these two tasks simultaneously and is trained in an end-to-end fashion.…”

Trajectory Prediction with Latent Belief Energy-Based Model

“…The trajectory generator or policy network is an autoregressive model in most prior works [1,15,25,50]. Some recent works explored the use of a non-autoregressive model [32,45]. We choose to use a non-autoregressive model (MLP) considering its efficiency and the avoidance of exposure bias inherent in autoregressive models.…”

Multi-Modal Hybrid Architecture for Pedestrian Action Prediction

“…The topic of human behavior prediction is of great interest to robotics and computer vision communities. Behavior prediction has been used in many applications such as humanobject [11], [12] and human-human interaction [13], [14], risk assessment [15], [16], anomaly detection [17], surveillance [18], [7], sports forecasting [19], [20] and intelligent driving systems [21], [5].…”