Dynamic and Static Context-Aware LSTM for Multi-agent Motion Prediction

Abstract

Multi-agent motion prediction is challenging because it aims to foresee the future trajectories of multiple agents (e.g. pedestrians) simultaneously in a complicated scene. Existing work addressed this challenge by either learning social spatial interactions represented by the positions of a group of pedestrians, while ignoring their temporal coherence (i.e. dependencies between different long trajectories), or by understanding the complicated scene layout (e.g. scene segmentation) to ensure safe navigation. However, unlike previous work that isolated the spatial interaction, temporal coherence, and scene layout, this paper designs a new mechanism, i.e., Dynamic and Static Context-aware Motion Predictor (DSCMP), to integrates these rich information into the longshort-term-memory (LSTM). It has three appealing benefits. (1) DSCMP models the dynamic interactions between agents by learning both their spatial positions and temporal coherence, as well as understanding the contextual scene layout. (2) Different from previous LSTM models that predict motions by propagating hidden features frame by frame, limiting the capacity to learn correlations between long trajectories, we carefully design a differentiable queue mechanism in DSCMP, which is able to explicitly memorize and learn the correlations between long trajectories.(3) DSCMP captures the context of scene by inferring latent variable, which enables multimodal predictions with meaningful semantic scene layout. Extensive experiments show that DSCMP outperforms state-ofthe-art methods by large margins, such as 9.05% and 7.62% relative improvements on the ETH-UCY and SDD datasets respectively.

Human Trajectory Forecasting in Crowds: A Deep Learning Perspective

“…[57], [58] propose to input the relative position and relative velocity of k nearest neighbours directly to a Multi-Layer Perceptron (MLP) to obtain the interaction vector. Many works [63]- [77] propose interaction module designs based on attention mechanisms [78], [79] to identify the neighbours which affect the trajectory of the person of interest. The attention weights are either learned or handcrafted based on domain knowledge (e.g., euclidean distance).…”

Blind Travel Prediction Based on Obstacle Avoidance in Indoor Scene

“…It uses a special Structural RNN (S-RNN) to calculate the weight of the spatial-temporal graph data. It takes the problem content as the node and the time series data as the edge value (iv) DSCMP [28]. It designs a queue mechanism to explicitly memorize and learn the correlation among long trajectories.…”

Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches

“…Other LSTM networks focus on attention mechanisms in the interaction module to capture the relative importance of each person in the scene. In some of these works, the attention weights are learned by data or by added handcrafted based on domain knowledge [176], [190], [191], [192]. Another relevant work is STGAT by Huang et al [191] that uses graph neural networks (GNN) instead of a pooling module.…”