Composable Q- Functions for Pedestrian Car Interactions

Abstract

We propose a novel algorithm that predicts the interaction of pedestrians with cars within a Markov Decision Process framework. It leverages the fact that Q-functions may be composed in the maximum-entropy framework, thus the solutions of two sub-tasks may be combined to approximate the full interaction problem. Sub-task one is the interactionfree navigation of a pedestrian in an urban environment and sub-task two is the interaction with an approaching car (deceleration, waiting etc.) without accounting for the environmental context (e.g. street layout). We propose a regularization scheme motivated by the soft-Bellman-equations and illustrate its necessity. We then analyze the properties of the algorithm in detail with a toy model. We find that as long as the interactionfree sub-task is modelled well with a Q-function, we can learn a representation of the interaction between a pedestrian and a car.

Human motion trajectory prediction: a survey

“…This scenario is safety-critical and crucial for autonomous vehicles to solve with high confidence. Pose and high-level contextual cues of the target agent (Kooij et al, 2019), and the scene context modeling (e.g., location and type of the obstacles (Muench and Gavrila, 2019; Völz et al, 2016), state of the traffic lights (Karasev et al, 2016)) are helpful to improve the crossing trajectory prediction.…”

“…Most planning-based methods discussed so far do not consider interactions between agents in the scene. To account for the presence of other agents, several authors proposed to modify individual optimal policies locally with physics-based methods (Rudenko et al, 2018a; van Den Berg et al, 2008; Wu et al, 2018) or imitation learning Muench and Gavrila (2019). A crowd simulation approach that combines global planning and local collision avoidance was presented by van Den Berg et al (2008).…”

“…The motion of pedestrians is predicted jointly with other traffic participants by risk checking of future states based on gap acceptance model (Brewer et al, 2006). Instead of using a physics-based approach (e.g., social forces) for augmenting the MDP-based predictor, Muench and Gavrila (2019) proposed to learn an additional interaction-aware Q-function with imitation learning.…”

“…Wu et al (2018) and Zechel et al (2019) discounted predicted transition probabilities to states in collision with other agents. Muench and Gavrila (2019) decomposed the interactive planning problem into two policies with the corresponding Q-functions: one for prediction in static environment, and another for interaction prediction in an obstacle-free environment. Many deep learning methods consider interactions between participants: explicitly modeling interacting entities (Alahi et al, 2016; Amirian et al, 2019; Bartoli et al, 2018; Choi et al, 2019; Eiffert and Sukkarieh, 2019; Fernando et al, 2018, 2019; Gupta et al, 2018; Hasan et al, 2018; Huang et al, 2019; Ivanovic and Pavone, 2019; Kosaraju et al, 2019; Pei et al, 2019; Pfeiffer et al, 2018; Radwan et al, 2018; Rhinehart et al, 2019; Sadeghian et al, 2019; Saleh et al, 2019; Shi et al, 2019; Su et al, 2017; van der Heiden et al, 2019; Varshneya and Srinivasaraghavan, 2017; Vemula et al, 2018; Xu et al, 2018; Xue et al, 2018; Zhao et al, 2019), implicitly as a result of pixel-wise prediction (Walker et al, 2014), or by learning a joint motion policy (Lee et al, 2017; Ma et al, 2017; Shalev-Shwartz et al, 2016; Zhan et al, 2018).…”

“…Semantic map-based approaches extend the map-aware approaches by considering various semantic attributes of the static environment. A semantic map (Ballan et al, 2016; Coscia et al, 2018; Karasev et al, 2016; Kitani et al, 2012; Muench and Gavrila, 2019; Rehder et al, 2018; Rhinehart et al, 2018a; Rhinehart et al, 2018b; Ridel et al, 2019; Saleh et al, 2019; Shen et al, 2018; Tadokoro et al, 1993; van der Heiden et al, 2019; Vasishta et al, 2017, 2018; Zhao et al, 2019) or extracted features from a top-down image (Kosaraju et al, 2019; Sadeghian et al, 2019; Tang and Salakhutdinov, 2019; Xue et al, 2018) can be used to capture people preferences in walking on a particular type of surfaces. Furthermore, planning-based methods often use prior knowledge on potential goals in the environment (Best and Fitch, 2015; Karasev et al, 2016; Previtali et al, 2016; Rudenko et al, 2017; Vasquez, 2016).…”