Deep Multi-agent Reinforcement Learning in a Homogeneous Open Population

Rădulescu, Roxana; Legrand, Manon; Efthymiadis, Kyriakos; Roijers, Diederik M.; Nowé, Ann

doi:10.1007/978-3-030-31978-6_8

Cited by 13 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-agent systems (MASs) are ideally suited to model a wide range of real-world problems where autonomous actors participate in distributed decision-making. Example application domains include urban and air traffic control (Yliniemi et al , 2015; Mannion et al , 2016a), autonomous vehicles (Rădulescu et al , 2018; Talpert et al , 2019), and energy systems (Walraven & Spaan, 2016; Mannion et al , 2016b; Reymond et al , 2018). Although many such problems feature multiple conflicting objectives to optimize, most MAS research focuses on agents maximizing their return w.r.t.…”

Section: Introductionmentioning

confidence: 99%

A utility-based analysis of equilibria in multi-objective normal-form games

Rădulescu¹,

Mannion²,

Zhang

et al. 2020

The Knowledge Engineering Review

Self Cite

View full text Add to dashboard Cite

In multi-objective multi-agent systems (MOMASs), agents explicitly consider the possible trade-offs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analyzed on the basis of the utility that these compromises have for the users of a system, where an agent’s utility function maps their payoff vectors to scalar utility values. This utility-based approach naturally leads to two different optimization criteria for agents in a MOMAS: expected scalarized returns (ESRs) and scalarized expected returns (SERs). In this article, we explore the differences between these two criteria using the framework of multi-objective normal-form games (MONFGs). We demonstrate that the choice of optimization criterion (ESR or SER) can radically alter the set of equilibria in a MONFG when nonlinear utility functions are used.

show abstract

Section: Introductionmentioning

confidence: 99%

A utility-based analysis of equilibria in multi-objective normal-form games

Rădulescu¹,

Mannion²,

Zhang

et al. 2020

The Knowledge Engineering Review

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some papers propose a multiagent approach to the "navigating in traffic" scenario also. In [103], the authors used a simple discrete, three-lane highway model, with simple choices, showing how the vehicle trained in a single agent approach fails, when placed in a multiagent environment and must deal with agents with the same policy as itself. Though it is also shown, the single agent is a good starting network to begin training in MARL setup.…”

Section: Driving In Trafficmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

345

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

“…The planning/learning phase and the execution phase may be either (partially) centralised or fully decentralised. The paradigm of centralised training with decentralised execution represents a middle ground between fully centralised and decentralised settings often used in cooperative or mixed settings [30,31,55,97,108]. The aim here is to enrich and aid the training/learning phase with extra information shared between the agents, however during the policy execution phase, the agents act in a fully decentralised manner.…”

Section: Multi-agent Decision Theorymentioning

confidence: 99%

“…[1,33,47,70,74,82,106,111,112]). Very recently, single-objective multi-agent RL has received considerable attention as well [30,32,39,55,97,81,109,130]. An important next step is therefore to extend existing Deep RL methods for multi-objective multi-agent decision making settings.…”

Section: Deep Multi-objective Multi-agent Decision Makingmentioning

confidence: 99%

Multi-objective multi-agent decision making: a utility-based analysis and survey

Rădulescu

Mannion

Roijers

et al. 2019

Auton Agent Multi-Agent Syst

Self Cite

View full text Add to dashboard Cite

The majority of multi-agent system (MAS) implementations aim to optimise agents' policies with respect to a single objective, despite the fact that many real-world problem domains are inherently multi-objective in nature. Multiobjective multi-agent systems (MOMAS) explicitly consider the possible trade-offs between conflicting objective functions. We argue that, in MOMAS, such compromises should be analysed on the basis of the utility that these compromises have for the users of a system. As is standard in multi-objective optimisation, we model the user utility using utility functions that map value or return vectors to scalar values. This approach naturally leads to two different optimisation criteria: expected scalarised returns (ESR) and scalarised expected returns (SER). We develop a new taxonomy which classifies multi-objective multi-agent decision making settings, on the basis of the reward structures, and which and how utility functions are applied. This allows us to offer a structured view of the field, to clearly delineate the current state-of-the-art in multi-objective multi-agent decision making approaches and to identify promising directions for future research. Starting from the execu-

show abstract

Deep Multi-agent Reinforcement Learning in a Homogeneous Open Population

Cited by 13 publications

References 13 publications

A utility-based analysis of equilibria in multi-objective normal-form games

A utility-based analysis of equilibria in multi-objective normal-form games

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Multi-objective multi-agent decision making: a utility-based analysis and survey

Contact Info

Product

Resources

About