Context based pedestrian intention prediction using factored latent dynamic conditional random fields

Abstract

Smooth handling of pedestrian interactions is a key requirement for Autonomous Vehicles (AV) and Advanced Driver Assistance Systems (ADAS). Such systems call for early and accurate prediction of a pedestrian's crossing/not-crossing behaviour in front of the vehicle. Existing approaches to pedestrian behaviour prediction make use of pedestrian motion, his/her location in a scene and static context variables such as traffic lights, zebra crossings etc. We stress on the necessity of early prediction for smooth operation of such systems. We introduce the influence of vehicle interactions on pedestrian intention for this purpose. In this paper, we show a discernible advance in prediction time aided by the inclusion of such vehicle interaction context. We apply our methods to two different datasets, one in-house collected -NTU dataset and another public real-life benchmark -JAAD dataset. We also propose a generic graphical model Factored Latent-Dynamic Conditional Random Fields (FLDCRF) for single and multilabel sequence prediction as well as joint interaction modeling tasks. FLDCRF outperforms Long Short-Term Memory (LSTM) networks across the datasets (∼100 sequences per dataset) over identical time-series features. While the existing best system predicts pedestrian stopping behaviour with 70% accuracy 0.38 seconds before the actual events, our system achieves such accuracy at least 0.9 seconds on an average before the actual events across datasets.

Set-Based Prediction of Pedestrians in Urban Environments Considering Formalized Traffic Rules

“…In order to apply such predictions, we require the current state of pedestrians from sensor data, which can be obtained as described in [5]- [7]; however, this process itself is beyond the scope of this work. a) Single behavior: The probability of whether pedestrians intend to cross the roadway is computed in [8]- [12] using one or more of the following sources: motion information (previous path and current position), situation awareness (e.g., head pose), and contextual information (e.g., proximity to curb or intersection). Based on the predicted intention, the most likely behavior can be inferred, while other works directly compute a single trajectory [13], [14] or the time until the pedestrian will most likely cross [15].…”

Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences

“…Existing approaches consider particular situations like crossing at an intersection, or at a marked crossing zebra, or more generically, in situations when the street is not marked at all. Several categories of prediction models are very popular [39]: (1) pedestrian related approaches (2) context based approaches (3) path prediction approaches. These approaches are applied on color images and few methods have been proposed for infrared images.…”

Pedestrian Intention Prediction: A Multi-task Perspective

“…There are model-based approaches using social behavior analysis [2], social force modeling [6] and conditional random fields [3] for pedestrian dynamic prediction. These models are scenario-specific and have low performance in approximating complex functions.…”

“…To guarantee the safety of pedestrians, a self-driving car should not only predict their intention but also predict it sufficiently in advance in order to react accordingly. In most papers of this category [2,3,4,5], either they do intention estimation, or the prediction is performed only for a short horizon as they are considering the current intention. However, we tackle this problem by providing a sequence of predictions of the intention for the next few frames.…”