HOME: Heatmap Output for future Motion Estimation

Gilles, Thomas; Sabatini, Stefano; Tsishkou, Dzmitry; Stanciulescu, Bogdan; Moutarde, Fabien

doi:10.1109/itsc48978.2021.9564944

Cited by 146 publications

(101 citation statements)

References 9 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…In our experimental results, we explore the effectiveness of states and kinematic controls, with and without an underlying polynomial basis. Notably unique are (1) HOME [22] and GOHOME [21] which first predict a heatmap, and then decode trajectories after sampling, and (2) MP3 [11] and NMP [52] which learn a cost function evaluator of trajectories, and the trajectories are enumerated heuristically rather than generated by a learned model.…”

Section: Related Workmentioning

confidence: 99%

MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

Varadarajan¹,

Hefny²,

Srivastava³

et al. 2021

Preprint

View full text Add to dashboard Cite

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multimodal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture [45] by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined, static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques-common in other ML domains-and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition [12] and the Waymo Open Dataset Motion Prediction Challenge [18].

show abstract

Section: Related Workmentioning

confidence: 99%

MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

Varadarajan¹,

Hefny²,

Srivastava³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…3, the relation predictor model consists of a context encoder that extracts the context information, including the observed states of the interacting agents and nearby agents and map coordinates, into a hidden vector, as well as a relation prediction head that outputs the probability over each relation type. There is a rich set of literature on learning context information from a traffic scene, such as [7,12,14,25]. Our model could utilize any existing context encoder thanks to its modularized design, and we defer a detailed explanation of our choice in Sec.…”

Section: Relation Predictormentioning

confidence: 99%

“…We propose a marginal trajectory predictor for the influencer based on an encoder-decoder structure, as shown in Fig. 3, which is widely adopted in the trajectory prediction literature [10,14,42]. The predictor utilizes the same context encoder as in Sec.…”

Section: Marginal Trajectory Predictormentioning

confidence: 99%

“…Rasterized representation draws traffic context on a single image with multiple channels and excels at capturing geometrical information. Both representations have achieved top performance in trajectory prediction benchmarks such as Argoverse and WOMD [6,10,12,14,15].…”

Section: Context Encodermentioning

confidence: 99%

“…In addition to encoding the vectorized feature, we utilize a second encoder to learn features from a rasterized representation. Following [14], We first rasterize the input states into an image with 60 channels, including the position of the agents at each past time frame with the map information. The size of the image is 224×224 and each pixel represents an area of 1m × 1m.…”

Section: Context Encodermentioning

confidence: 99%

See 2 more Smart Citations

M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction

Sun¹,

Huang²,

Gu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Predicting future motions of road participants is an important task for driving autonomously in urban scenes. Existing models excel at predicting marginal trajectories for single agents, yet it remains an open question to jointly predict scene compliant trajectories over multiple agents. The challenge is due to exponentially increasing prediction space as a function of the number of agents. In this work, we exploit the underlying relations between interacting agents and decouple the joint prediction problem into marginal prediction problems. Our proposed approach M2I first classifies interacting agents as pairs of influencers and reactors, and then leverages a marginal prediction model and a conditional prediction model to predict trajectories for the influencers and reactors, respectively. The predictions from interacting agents are combined and selected according to their joint likelihoods. Experiments show that our simple but effective approach achieves state-of-the-art performance on the Waymo Open Motion Dataset interactive prediction benchmark.

show abstract