Social Coordination and Altruism in Autonomous Driving

Toghi, Behrad; Valiente, Rodolfo; Sadigh, Dorsa; Pedarsani, Ramtin; Fallah, Yaser P.

doi:10.48550/arxiv.2107.00200

Cited by 5 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Formally, the SVO angle φ of an AV, determines how the AV balances its own benefit against that of others. In terms of rewards, we can define the total reward R i of an AV as: R i (s, a) = cos φ i r ego i + sin φ i r social i , in which r ego i is the AV's specific reward (egoistic) and r social i is the overall reward of other vehicles (social) respect to the i th AV [2], [3]. The SVO angle can be changed from φ = 0 (purely egoistic) to φ = π/2 (purely altruistic).…”

Section: Social Value Orientation and Altruistic Avsmentioning

confidence: 99%

“…T HE development of autonomous vehicles (AVs) is on the verge of passing beyond the laboratory and simulation tests and is shifting towards addressing the challenges that limit their practicality in today's society. While there is still need for further technological improvements to enable safe and smooth operation of a single AV, a great deal of research attention is being focused on the emerging challenge of operating multiple AVs and the co-existence of AVs and human-driven vehicles (HVs) [1], [2]. A realistic outlook for the adoption of autonomous vehicles on the roads is a mixed-traffic scenario in which human drivers with different driving styles and social preferences share the road with AVs that are perhaps built by different manufacturers and hence follow different policies [3], [4].…”

Section: Introductionmentioning

confidence: 99%

“…In a pursuit to alleviate the challenges of this co-existence and enable social navigation for AVs, existing works either rely on models of human behavior derived from pre-recorded driving datasets [16], [17] or defining social utilities that can enforce a cooperative behavior among AVs and HVs [2], [18]. Other works focus on rule-based methods that use heuristics and hand-coded rules to guide the AVs [19] or probabilistic driver modeling [20]- [22].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic

Valiente¹,

Toghi²,

Pedarsani³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.

show abstract

Section: Social Value Orientation and Altruistic Avsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic

Valiente¹,

Toghi²,

Pedarsani³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Instead, we adjust the AVs' SVO through a decentralized reinforcement learning algorithm which promotes altruistic behavior in their decision-making process. As opposed to the existing works which rely on derived behavior models of human drivers, we follow our methodology in [31,32] which allows the autonomous agents to implicitly model human drivers SVO in real-time and through experience. The highway merging scenario is particularly interesting to us due to its competitive nature as the interest of the merging vehicle does not align with the interest of the cruising vehicles.…”

Section: Proposed Solutionmentioning

confidence: 99%

Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic

Toghi¹,

Valiente²,

Sadigh³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the adoption of autonomous vehicles on our roads, we will witness a mixed-autonomy environment where autonomous and human-driven vehicles must learn to coexist by sharing the same road infrastructure. To attain socially-desirable behaviors, autonomous vehicles must be instructed to consider the utility of other vehicles around them in their decision-making process. Particularly, we study the maneuver planning problem for autonomous vehicles and investigate how a decentralized reward structure can induce altruism in their behavior and incentivize them to account for the interest of other autonomous and human-driven vehicles. This is a challenging problem due to the ambiguity of a human driver's willingness to cooperate with an autonomous vehicle. Thus, in contrast with the existing works which rely on behavior models of human drivers, we take an end-to-end approach and let the autonomous agents to implicitly learn the decision-making process of human drivers only from experience. We introduce a multi-agent variant of the synchronous Advantage Actor-Critic (A2C) algorithm and train agents that coordinate with each other and can affect the behavior of human drivers to improve traffic flow and safety.

show abstract

“…Each player has an associated parameter, 𝛼, that indicates the player's interest in the other players achieving their objectives. This formulation relates to the Game Theoretic notion of altruism [1], as well as Social Value Orientation (SVO)-based models proposed in [12] which have garnered recent interest [14,15]. They show how this formulation can be used to efficiently compute interactive policies that account for each player's preferences.…”

Section: Introductionmentioning

confidence: 99%

Active Altruism Learning and Information Sufficiency for Autonomous Driving

Geary,

Gouk,

Ramamoorthy

2021

Preprint

View full text Add to dashboard Cite

Safe interaction between vehicles requires the ability to choose actions that reveal the preferences of the other vehicles. Since exploratory actions often do not directly contribute to their objective, an interactive vehicle must also able to identify when it is appropriate to perform them. In this work we demonstrate how Active Learning methods can be used to incentivise an autonomous vehicle (AV) to choose actions that reveal information about the altruistic inclinations of another vehicle. We identify a property, Information Sufficiency, that a reward function should have in order to keep exploration from unnecessarily interfering with the pursuit of an objective. We empirically demonstrate that reward functions that do not have Information Sufficiency are prone to inadequate exploration, which can result in sub-optimal behaviour. We propose a reward definition that has Information Sufficiency, and show that it facilitates an AV choosing exploratory actions to estimate altruistic tendency, whilst also compensating for the possibility of conflicting beliefs between vehicles.

show abstract

Social Coordination and Altruism in Autonomous Driving

Cited by 5 publications

References 0 publications

Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic

Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic

Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic

Active Altruism Learning and Information Sufficiency for Autonomous Driving

Contact Info

Product

Resources

About