PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction

Abstract

Predicting pedestrian behavior is one of the main challenges for intelligent driving systems. In this paper, we present a new paradigm for evaluating egocentric pedestrian trajectory prediction algorithms. Based on various contextual information, we extract driving scenarios for a meaningful and systematic approach to identifying challenges for prediction models. In this regard, we also propose a new metric for more effective ranking within the scenario-based evaluation. We conduct extensive empirical studies of existing models on these scenarios to expose shortcomings and strengths of different approaches. The scenario-based analysis highlights the importance of using multimodal sources of information and challenges caused by inadequate modeling of ego-motion and scale of pedestrians. To this end, we propose a novel egocentric trajectory prediction model that benefits from multimodal sources of data fused in an effective and efficient step-wise hierarchical fashion and two auxiliary tasks designed to learn more robust representation of scene dynamics. We show that our approach achieves significant improvement by up to 40% in challenging scenarios compared to the past arts via empirical evaluation on common benchmark datasets. Pedestrian Scale (pixels) Pedestrian State 0-50 50-80 80-100 100-150 150-200 200-300 300+ Walking Standing 5.

CAPformer: Pedestrian Crossing Action Prediction Using Transformer

“…In [ 26 ], the creators of JAAD released a new dataset, called PIE (details in Table 1 ), recorded continuously in a single day in Toronto, with clear weather. In addition to label more relevant objects than in JAAD, it includes ego-motion information of the car.…”

“…This new information is used in combination with estimated pose keypoints, bounding box coordinates, and bounding boxes image crops to train a stacked GRU model in [ 5 ]. In the same release article of PIE [ 26 ], a multi-task and multi-branch model is proposed. It combines the use of LSTM layers for non-image information (bounding box coordinates and ego-motion information through ego-vehicle OBD measures) and convolutional LSTM layers for image information (bounding boxes crops).…”

Algorithm for Predicting the Trajectory of Road Users to Automate Control of an Autonomous Vehicle

“…Most often, recurrent layers are also used for this purpose. [15], [16], [17]. Then, having the calibration parameters of the camera, we can understand the position of the object in space [18].…”

Roadside pedestrian motion prediction using Bayesian methods and particle filter

“…Unsupervised learning was combined with the Gaussian process to quickly detect the changes in pedestrian intention. Rasouli et al [43] proposed an RNN encoder-decoder architecture for pedestrian trajectory prediction. Pedestrian identified latent intention and the predicted vehicle speed were combined to the trajectory prediction module by the decoder.…”

“…Rasouli et al. [43] proposed an RNN encoder‐decoder architecture for pedestrian trajectory prediction. Pedestrian identified latent intention and the predicted vehicle speed were combined to the trajectory prediction module by the decoder.…”