Human-centric Autonomous Driving in an AV-Pedestrian Interactive Environment Using SVO

Abstract

As Autonomous Vehicles (AV) are becoming a reality, the design of efficient motion control algorithms will have to deal with the unpredictable and interactive nature of other road users. Current AV motion planning algorithms suffer from the freezing robot problem, as they often tend to overestimate collision risks. To tackle this problem and design AV that behave human-like, we integrate a concept from Psychology called Social Value Orientation into the Reinforcement Learning (RL) framework. The addition of a social term in the reward function design allows us to tune the AV behaviour towards the pedestrian from a more reckless to an extremely prudent one. We train the vehicle agent with a state of the art RL algorithm and show that Social Value Orientation is an effective tool to obtain pro-social AV behaviour.

A Cooperative Optimal Control Framework for Connected and Automated Vehicles in Mixed Traffic Using Social Value Orientation

“…terms of the objective functions of CAV-1 and HDV-2, respectively, at time k. Let l 12 x 1,k+1 , u 1,k , x 2,k+1 , u 2,k be the cooperative term of their objective function at time k. Note that l 1 , l 2 , and l 12 are composed appropriately through some weights to avoid one dominating the other. The preferences of the vehicles toward the egoistic and cooperative terms can be described by social value orientation, a commonly used concept in psychology that has been recently employed in autonomous driving research [16]- [22]. Let φ 1 and φ 2 be the SVO angles for CAV-1 and HDV-2, respectively.…”

“…The probability distribution in (22) implies that the agent exponentially tends to choose the trajectory with a lower objective value. It is not possible to solve the optimization problem (21) analytically, but we can compute the gradient ∇L φ2 of the objective function in (22) with respect to the variable φ 2 using the chain rule as follows…”

“…where ∇L θ is the gradient of the probability distribution (22) with repsect to the vector of weights θ(φ 2 ). It can be shown that this gradient is the difference between the expected and the empirical feature values [26]…”

“…A number of research articles have recently focused on developing different control methods for CAVs in mixed traffic scenarios, e.g., open-loop platoon formation [8], optimal control [9]- [11], model predictive control [12], [13], or learning-based control [14], [15]. Other research efforts [16]- [22] have utilized social value orientation (SVO), a concept from psychology that quantifies the level of an agent's selfishness or altruism, in control development to predict how human drivers will interact and cooperate with others. The first attempt taking social factors of human drivers into consideration via SVO was made in [16], [17].…”

“…Toghi et al [21] proposed a cooperative sympathetic reward structure using SVO for multi-agent reinforcement learning in autonomous driving. Crosato et al [22] considered a scenario with pedestrian crossing and utilized SVO to design a reinforcement learning reward function for an automated vehicle to avoid collision with the pedestrian.…”