Learning-based decision making for safe and scalable autonomous separation assurance

Abstract

A novel deep multi-agent reinforcement learning framework is proposed to identify and resolve conflicts among a variable number of aircraft in a high-density, stochastic, and dynamic en route sector.The concept of using distributed vehicle autonomy to ensure separation is proposed, instead of a centralized sector air traffic controller. Our proposed framework utilizes Proximal Policy Optimization (PPO) that is customized to incorporate an attention network. This allows the agents to have access to variable aircraft information in the sector in a scalable, efficient approach to achieve high traffic throughput under uncertainty. Agents are trained using a centralized learning, decentralized execution scheme where one neural network is learned and shared by all agents. The proposed framework is validated on three case studies in the BlueSky air traffic simulator. Several baselines are introduced and the numerical results show that the proposed framework significantly reduces offline training time, increases safe separation performance, and results in a more efficient policy.

Putting humans back in the loop: An affordance conceptualization of the 4th industrial revolution

“…H dm is also extensible in decision quality, meaning that decision probability functions increase in validity over time as machine learning progresses. Both dimensions of extensibility characterise the new machine capability of 4IR expansive decision‐making (sometimes referred to as learning‐based decision‐making (Brittain, 2021) to contrast it with the rules‐based decision‐making of the information revolution).…”

“…H dm is also extensible in decision quality, meaning that decision probability functions increase in validity over time as machine learning progresses. Both dimensions of extensibility characterize the new machine capability of 4IR expansive decision making (sometimes referred to as learning-based decision making(Brittain 2021) to contrast it with the rules-based decision making of the information revolution).Depending on the principles that govern what is appropriate for a given decision context(Fayard and Weeks 2014;Stahl 2012), H dm affordances can lead to beneficial or damaging outcomes e-or, as is sometimes the case, both simultaneously.4.2 Creativity automation H caCreativity automation H ca refers to machines that create artifacts such as plans, objects, and artworks that are sufficiently new and appropriate to be valuable or useful. Machines had no such capabilities in the prior information revolution(Chen et al 2021;Edwards 2021), leading to immense value opportunities-and risk-in the 4IR.H ca action possibilities span several domains, including writing (image to caption, essays, product descriptions, etc.…”