Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

Abstract

In this work, we explore the correlation between people trajectories and their head orientations. We argue that people trajectory and head pose forecasting can be modelled as a joint problem. Recent approaches on trajectory forecasting leverage short-term trajectories (aka tracklets) of pedestrians to predict their future paths. In addition, sociological cues, such as expected destination or pedestrian interaction, are often combined with tracklets. In this paper, we propose MiXing-LSTM (MX-LSTM) to capture the interplay between positions and head orientations (vislets) thanks to a joint unconstrained optimization of full covariance matrices during the LSTM backpropagation. We additionally exploit the head orientations as a proxy for the visual attention, when modeling social interactions. MX-LSTM predicts future pedestrians location and head pose, increasing the standard capabilities of the current approaches on long-term trajectory forecasting. Compared to the state-of-the-art, our approach shows better performances on an extensive set of public benchmarks. MX-LSTM is particularly effective when people move slowly, i.e. the most challenging scenario for all other models. The proposed approach also allows for accurate predictions on a longer time horizon.

Multiple Trajectory Prediction with Deep Temporal and Spatial Convolutional Neural Networks

“…Our approach delivers stateof-the-art results as it won the first place at the Argoverse Motion Forecasting Challenge, as presented on the NeurIPS 2019 workshop on "Machine Learning for Autonomous Driving". In our future work, we want to explore multiple trajectory prediction for different domains such as human motion prediction, and extend existing approaches such as [34]. Also, we want to evaluate the potential of robust regression methods as proposed in [35] for the prediction.…”

Human Trajectory Forecasting in Crowds: A Deep Learning Perspective

“…Bisagno et al [51] proposed to consider only pedestrians not belonging to the same group during social pooling. While modelling social interactions, Hasan et al [59], [60] based on domain knowledge, only consider the pedestrians in the visual frustum of attention [62]. Gupta et al [52] propose to encode neighbourhood information through the use of a permutation-invariant (symmetric) max-pooling function.…”

Efficient Prediction of Human Motion for Real-Time Robotics Applications With Physics-Inspired Neural Networks

“…Neural Networks (NN) hold the promise to master the complexity of predicting the intention of humans in a relatively simple way. Similarly to out work, [13] modelled an area of visual attention and social interaction for the pedestrian, jointed to the head orientation, and used to strengthen the training of a Long Short Term Memory network (LSTM). A deep learning-based classifier is used in [14] to learn behaviour patterns from visual cues is mixed with a game theory model encoding the SFM to forecast the interaction between multiple pedestrians.…”