Benchmark for Evaluating Pedestrian Action Prediction

Kotseruba, Iuliia; Rasouli, Amir; Tsotsos, John K.

doi:10.1109/wacv48630.2021.00130

Cited by 76 publications

(96 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It integrates four feature sources: semantic map, pedestrian trajectory, grid position and vehicle speed. Kotseruba [13] considered four feature sources: local environment, pedestrian posture, pedestrian bounding box and vehicle speed. A threedimensional volume integral branch is used to encode visual information and a single RNN branch is used to process other information in parallel.…”

Section: Related Workmentioning

confidence: 99%

“…For j = 0 (the bottom level of the stack), x t 0 = c t p and for j > 0, x t j = h t−1 j + c t p . Meanwhile, inspired by [3,13], we introduced the attention mechanism [18] into GRU to form At-GRU (attention-GRU). The attention module can selectively focus on some features, so as to better deal with key objects.…”

Section: Model Constructionmentioning

confidence: 99%

“…According to the summary in [13], in the current research on the prediction of pedestrian crossing behavior, most of the observation time is about 0.5 s. The TTP is mostly in the range of 1 s to 2 s. The experimental data of some studies are taken from the whole process of crossing the street, while others are taken from the part before crossing the street. Since the prediction of the time when pedestrians begin to cross the street is the focus and difficulty in the research.…”

Section: Quantitative Experimentsmentioning

confidence: 99%

“…Later studies considered both spatial and temporal features, using recurrent neural networks (RNN) and three-dimensional convolution neural network (3DCNN) to extract spatio-temporal information [5][6][7][8][9]. At the same time, different methods are used to fuse a variety of features to predict pedestrians' crossing behaviors, such as pedestrian bounding box, posture, vehicle speed and surrounding environment information [10][11][12][13][14].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Prediction of Pedestrian Crossing Behavior Based on Surveillance Video

Zhou

Ren

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

Prediction of pedestrian crossing behavior is an important issue faced by the realization of autonomous driving. The current research on pedestrian crossing behavior prediction is mainly based on vehicle camera. However, the sight line of vehicle camera may be blocked by other vehicles or the road environment, making it difficult to obtain key information in the scene. Pedestrian crossing behavior prediction based on surveillance video can be used in key road sections or accident-prone areas to provide supplementary information for vehicle decision-making, thereby reducing the risk of accidents. To this end, we propose a pedestrian crossing behavior prediction network for surveillance video. The network integrates pedestrian posture, local context and global context features through a new cross-stacked gated recurrence unit (GRU) structure to achieve accurate prediction of pedestrian crossing behavior. Applied onto the surveillance video dataset from the University of California, Berkeley to predict the pedestrian crossing behavior, our model achieves the best results regarding accuracy, F1 parameter, etc. In addition, we conducted experiments to study the effects of time to prediction and pedestrian speed on the prediction accuracy. This paper proves the feasibility of pedestrian crossing behavior prediction based on surveillance video. It provides a reference for the application of edge computing in the safety guarantee of automatic driving.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Model Constructionmentioning

confidence: 99%

Section: Quantitative Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Prediction of Pedestrian Crossing Behavior Based on Surveillance Video

Zhou

Ren

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Also, we can formulate it as a higher-level semantic prediction such as the early anticipation of the future action of the pedestrian, for example, walking, running, performing hand gestures, or most importantly crossing or not crossing the street in front of the AV. Recently, trajectory and action prediction solutions have been proposed based on sequential reasoning that mainly use algorithms built on recurrent neural networks (i.e., RNN, LSTM) [4], [5], [6], [7]. However, it has recently become clear that LSTM lacks many capabilities to model sequential data.…”

Section: Introductionmentioning

confidence: 99%

Is attention to bounding boxes all you need for pedestrian action prediction?

Achaji¹,

Moreau²,

Fouqueray³

et al. 2021

Preprint

View full text Add to dashboard Cite

The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AV in urban places underpins essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict their future behavior. In this paper, we present a framework based on multiple variations of the Transformer models to reason attentively about the dynamic evolution of the pedestrians' past trajectory and predict its future actions of crossing or not crossing the street. We proved that using only bounding-boxes as input to our model can outperform the previous state-of-the-art models and reach a prediction accuracy of 91% and an F1-score of 0.83 on the PIE dataset up to two seconds ahead in the future. In addition, we introduced a large-size simulated dataset (CP2A) using CARLA for action prediction. Our model has similarly reached high accuracy (91 %) and F1-score (0.91) on this dataset. Interestingly, we showed that pre-training our Transformer model on the simulated dataset and then fine-tuning it on the real dataset can be very effective for the action prediction task.

show abstract