Time-series Imputation of Temporally-occluded Multiagent Trajectories

Abstract

In multiagent environments, several decision-making individuals interact while adhering to the dynamics constraints imposed by the environment. These interactions, combined with the potential stochasticity of the agents' decision-making processes, make such systems complex and interesting to study from a dynamical perspective. Significant research has been conducted on learning models for forward-direction estimation of agent behaviors, for example, pedestrian predictions used for collision-avoidance in self-driving cars. However, in many settings, only sporadic observations of agents may be available in a given trajectory sequence. For instance, in football, subsets of players may come in and out of view of broadcast video footage, while unobserved players continue to interact off-screen. In this paper, we study the problem of multiagent time-series imputation, where available past and future observations of subsets of agents are used to estimate missing observations for other agents. Our approach, called the Graph Imputer, uses forward-and backward-information in combination with graph networks and variational autoencoders to enable learning of a distribution of imputed trajectories. We evaluate our approach on a dataset of football matches, using a projective camera module to train and evaluate our model for the off-screen player state estimation setting. We illustrate that our method outperforms several state-of-the-art approaches, including those hand-crafted for football.Preprint. Under review.

Layout Sequence Prediction From Noisy Mobile Modality

“…However, less attention pays to the mobile modality that has no out-of-sight problem to estimate the location and motion status of objects without concerns about out-of-view problems. With the development of mobile computing, sensors are now ubiquitous, including IMU mobile sensors [4,12,22,27], wireless signals [26,35], GPS [1,10], satellite data, and user reports to provide out-of-view [5,28] and blind area [7,16] trajectory information. These sensors have no out-of-sight problem and can be received through a long range without showing in the camera, which is much more ideal than the camera data in the visual modality.…”