Flatland-RL : Multi-Agent Reinforcement Learning on Trains

Mohanty, Sharada P.; Nygren, Erik; Laurent, Florian; Schneider, Manuel K.; Scheller, Christian; Bhattacharya, Nilabha; Watson, Jeremy D.; Egli, Adrian; Christian, Eichenberger,; Baumberger, Christian; Sturm, Irene; Sartoretti, Guillaume; Spigler, Giacomo

doi:10.48550/arxiv.2012.05893

Cited by 10 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In another line of work, MARL is applied to resource optimization and scheduling problems for mobility scenarios. This includes scheduling for trains [50] or ambulances, but can also be applied to taxi repositioning [64], ride-sharing services [65], bike repositioning, or container inventory management [49]. Li et al [65] propose to solve the assignment of ride requests to specific drivers with MARL, which improves order response rates.…”

Section: Resource Optimizationmentioning

confidence: 99%

See 1 more Smart Citation

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

M.¹,

Brosig²,

Plinge³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many scenarios in mobility and traffic involve multiple different agents that need to cooperate to find a joint solution. Recent advances in behavioral planning use Reinforcement Learning to find effective and performant behavior strategies. However, as autonomous vehicles and vehicle-to-X communications become more mature, solutions that only utilize single, independent agents leave potential performance gains on the road. Multi-Agent Reinforcement Learning (MARL) is a research field that aims to find optimal solutions for multiple agents that interact with each other. This work aims to give an overview of the field to researchers in autonomous mobility. We first explain MARL and introduce important concepts. Then, we discuss the central paradigms that underlie MARL algorithms, and give an overview of state-of-the-art methods and ideas in each paradigm. With this background, we survey applications of MARL in autonomous mobility scenarios and give an overview of existing scenarios and implementations.

show abstract

Section: Resource Optimizationmentioning

confidence: 99%

“…A popular simulation used in the train scheduling and routing domain is Flatland, which has been used in several NeurIPS competitions [50]. Flatland provides a 2D-gridworld of train tracks.…”

Section: Resource Optimizationmentioning

confidence: 99%

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

M.¹,

Brosig²,

Plinge³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The Flatland library (see [8] and [14]) is an open-source framework implementing a multi-agent grid world environment, which was specifically designed to facilitate the development and testing of reinforcement learning algorithms for the rail transport system. This provides the framework of the Flatland Challenges, where the main goal is to create an adaptive controller which plans the trains' routes and actions efficiently, and has the ability to adapt to possible delays and malfunctions.…”

Section: The Flatland Challengementioning

confidence: 99%

“…Furthermore, the stochastic behaviour of the system caused by the disruption incidents adds an extra layer of complexity to the studied problem, which calls for constant monitoring and dynamic rescheduling. For more information regarding the action space, the reward signals, as well as the stochastic events manifested in the form of train malfunctions, see [8], [14] and [16].…”

Section: The Flatland Challengementioning

confidence: 99%

Standardized feature extraction from pairwise conflicts applied to the train rescheduling problem

Kopacz¹,

Mester²,

Kolumbán³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a train rescheduling algorithm which applies a standardized feature selection based on pairwise conflicts in order to serve as input for the reinforcement learning framework. We implement an analytical method which identifies and optimally solves every conflict arising between two trains, then we design a corresponding observation space which features the most relevant information considering these conflicts. The data obtained this way then translates to actions in the context of the reinforcement learning framework. We test our preliminary model using the evaluation metrics of the Flatland Challenge. The empirical results indicate that the suggested feature space provides meaningful observations, from which a sensible scheduling policy can be learned.

show abstract

“…Multi-agent reinforcement learning (MARL) is a promising technique for solving challenging problems, such as air traffic control [5], train scheduling [24], cyber defense [20], and autonomous driving [4]. In many of these scenarios, we want to train a team of cooperating agents.…”

Section: Introductionmentioning

confidence: 99%

MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning

Milani¹,

Zhang²,

Topin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many recent breakthroughs in multi-agent reinforcement learning (MARL) require the use of deep neural networks, which are challenging for human experts to interpret and understand. On the other hand, existing work on interpretable RL has shown promise in extracting more interpretable decision tree-based policies, but only in the single-agent setting. To fill this gap, we propose the first set of algorithms that extract interpretable decision-tree policies from neural networks trained with MARL. The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting. We demonstrate that IVIPER can learn high-quality decision-tree policies for each agent. To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by predicting the behavior of the other agents using their anticipated trees, and uses resampling to focus on states that are critical for its interactions with other agents. We show that both algorithms generally outperform the baselines and that MAVIPER-trained agents achieve better-coordinated performance than IVIPER-trained agents on three different multi-agent particle-world environments.

show abstract

Flatland-RL : Multi-Agent Reinforcement Learning on Trains

Cited by 10 publications

References 17 publications

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

Standardized feature extraction from pairwise conflicts applied to the train rescheduling problem

MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About