Pedestrian Behavior Prediction Using Deep Learning Methods for Urban Scenarios: A Review

Zhang, Chi; Berger, Christian

doi:10.1109/tits.2023.3281393

Cited by 21 publications

(7 citation statements)

References 130 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Contributions: (1) We propose BA-PTP, a novel method for pedestrian trajectory prediction in the image plane, which incorporates behavioral features of pedestrians using independent encoding streams. (2) We introduce STEMM, a novel approach for predicting the future ego-motion of the ego-vehicle using spatial goal points, which is integrated into BA-PTP.…”

Section: Figurementioning

confidence: 99%

“…One of the essential challenges for automated driving in urban traffic conditions is the task of pedestrian behavior prediction. Therefore, understanding the underlying intentions of pedestrians is crucial for automated vehicles to better understand their surroundings in order to make better and safer driving decisions and mitigate potential risks and hazardous scenarios [1]. Due to the inherently variable nature of pedestrians' behavior, solving the task of predicting this behavior involves numerous challenges.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

Czech

Braun

Kreßel

et al. 2023

MAKE

View full text Add to dashboard Cite

With the ongoing development of automated driving systems, the crucial task of predicting pedestrian behavior is attracting growing attention. The prediction of future pedestrian trajectories from the ego-vehicle camera perspective is particularly challenging due to the dynamically changing scene. Therefore, we present Behavior-Aware Pedestrian Trajectory Prediction (BA-PTP), a novel approach to pedestrian trajectory prediction for ego-centric camera views. It incorporates behavioral features extracted from real-world traffic scene observations such as the body and head orientation of pedestrians, as well as their pose, in addition to positional information from body and head bounding boxes. For each input modality, we employed independent encoding streams that are combined through a modality attention mechanism. To account for the ego-motion of the camera in an ego-centric view, we introduced Spatio-Temporal Ego-Motion Module (STEMM), a novel approach to ego-motion prediction. Compared to the related works, it utilizes spatial goal points of the ego-vehicle that are sampled from its intended route. We experimentally validated the effectiveness of our approach using two datasets for pedestrian behavior prediction in urban traffic scenes. Based on ablation studies, we show the advantages of incorporating different behavioral features for pedestrian trajectory prediction in the image plane. Moreover, we demonstrate the benefit of integrating STEMM into our pedestrian trajectory prediction method, BA-PTP. BA-PTP achieves state-of-the-art performance on the PIE dataset, outperforming prior work by 7% in MSE-1.5 s and CMSE as well as 9% in CFMSE.

show abstract

Section: Figurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

Czech

Braun

Kreßel

et al. 2023

MAKE

View full text Add to dashboard Cite

show abstract

“…Given the breakthroughs of Transformers [15] in natural language processing and their wide-spread adoption for predicting agent behavior due to their long-term predictive capabilities [16,17], they efficiently address the memory problem of handling long sequences with attention mechanisms. The model can directly associate the entirety of input data sequences and context vectors, rather than being limited to the association with the last hidden states [18]. Syed et al [19] introduced the Spatiotemporal Graph Transformer (STGT) model, which uses CNN models for processing environmental image features and employs Transformers for sequence prediction.…”

Section: Introductionmentioning

confidence: 99%

Goal-Guided Graph Attention Network with Interactive State Refinement for Multi-Agent Trajectory Prediction

Wu,

Qiao,

et al. 2024

Sensors

View full text Add to dashboard Cite

The accurate prediction of the future trajectories of traffic participants is crucial for enhancing the safety and decision-making capabilities of autonomous vehicles. Modeling social interactions among agents and revealing the inherent relationships is crucial for accurate trajectory prediction. In this context, we propose a goal-guided and interaction-aware state refinement graph attention network (SRGAT) for multi-agent trajectory prediction. This model effectively integrates high-precision map data and dynamic traffic states and captures long-term temporal dependencies through the Transformer network. Based on these dependencies, it generates multiple potential goals and Points of Interest (POIs). Through its dual-branch, multimodal prediction approach, the model not only proposes various plausible future trajectories associated with these POIs, but also rigorously assesses the confidence levels of each trajectory. This goal-oriented strategy enables SRGAT to accurately predict the future movement trajectories of other vehicles in complex traffic scenarios. Tested on the Argoverse and nuScenes datasets, SRGAT surpasses existing algorithms in key performance metrics by adeptly integrating past trajectories and current context. This goal-guided approach not only enhances long-term prediction accuracy, but also ensures its reliability, demonstrating a significant advancement in trajectory forecasting.

show abstract

“…(Rasouli and Tsotsos 2020). As high-level information of be-havior is not directly observable and cannot be estimated by simply using the pedestrian's trajectories, pedestrian crossing intention prediction requires a holistic comprehension of the context, scene, pedestrian behavioral attributes, and meticulous inference from past actions (Sharma, Dhiman, and Indu 2022;Zhang and Berger 2023).…”

Section: Introductionmentioning

confidence: 99%

GPT-4V Takes the Wheel: Promises and Challenges for Pedestrian Behavior Prediction

Huang,

Jiang,

Gautam

et al. 2024

AAAI-SS

View full text Add to dashboard Cite

Predicting pedestrian behavior is the key to ensure safety and reliability of autonomous vehicles. While deep learning methods have been promising by learning from annotated video frame sequences, they often fail to fully grasp the dynamic interactions between pedestrians and traffic, crucial for accurate predictions. These models also lack nuanced common sense reasoning. Moreover, the manual annotation of datasets for these models is expensive and challenging to adapt to new situations. The advent of Vision Language Models (VLMs) introduces promising alternatives to these issues, thanks to their advanced visual and causal reasoning skills. To our knowledge, this research is the first to conduct both quantitative and qualitative evaluations of VLMs in the context of pedestrian behavior prediction for autonomous driving. We evaluate GPT-4V(ision) on publicly available pedestrian datasets: JAAD and WiDEVIEW. Our quantitative analysis focuses on GPT-4V's ability to predict pedestrian behavior in current and future frames. The model achieves a 57% accuracy in a zero-shot manner, which, while impressive, is still behind the state-of-the-art domain-specific models (70%) in predicting pedestrian crossing actions. Qualitatively, GPT-4V shows an impressive ability to process and interpret complex traffic scenarios, differentiate between various pedestrian behaviors, and detect and analyze groups. However, it faces challenges, such as difficulty in detecting smaller pedestrians and assessing the relative motion between pedestrians and the ego vehicle.

show abstract

Pedestrian Behavior Prediction Using Deep Learning Methods for Urban Scenarios: A Review

Cited by 21 publications

References 130 publications

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

Goal-Guided Graph Attention Network with Interactive State Refinement for Multi-Agent Trajectory Prediction

GPT-4V Takes the Wheel: Promises and Challenges for Pedestrian Behavior Prediction

Contact Info

Product

Resources

About