Reinforcement Learning for Multiaircraft Autonomous Air Combat in Multisensor UCAV Platform

Kong, Weiren; Zhou, Deyun; Du, Yongjie; Zhou, Ying; Zhao, Yiyang

doi:10.1109/jsen.2022.3220324

Cited by 4 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in modern air combat, one‐on‐one air combat scenarios are rare, and most air combat are conducted in formations. (ii)Existing research on multi‐aircraft air combat focus on cooperative target assignments [40, 41], WEZ computation [42], and high‐level maneuver tactics sequence solution in fixed‐size formation. These solutions only address one or a few stages of Observe‐Orient‐Decide‐Act (OODA) loop without looking at the entire air combat process as a whole. (iii)Many studies survey either leveraged low fidelity/dimension simulation environments or abstracted the action space to high level behaviors or tactics, specifically, basic fighter maneuver (BFM) [35–38, 43]. (iv)Many studies convert air combat maneuver control into a MDP by fixing the opponent's strategy or considering the target fully observable [35, 36].…”

Section: Introductionmentioning

confidence: 99%

“…These solutions only address one or a few stages of Observe-Orient-Decide-Act (OODA) loop without looking at the entire air combat process as a whole. (iii) Many studies survey either leveraged low fidelity/ dimension simulation environments or abstracted the action space to high level behaviors or tactics, specifically, basic fighter maneuver (BFM) [35][36][37][38]43]. (iv) Many studies convert air combat maneuver control into a MDP by fixing the opponent's strategy or considering the target fully observable [35,36].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Hierarchical multi‐agent reinforcement learning for multi‐aircraft close‐range air combat

Kong

Zhou

et al. 2022

IET Control Theory & Appl

Self Cite

View full text Add to dashboard Cite

The close‐range autonomous air combat has gained significant attention from researchers involved in applications related to artificial intelligence (AI). A majority of the previous studies on autonomous air combat were focused on one‐on‐one air combat scenarios, however, the modern air combat is mostly conducted in formations. With regard to the aforementioned factors, a novel hierarchical maneuvering control architecture is introduced that is applied to the multi‐aircraft close‐range air combat scenario, which can handle air combat scenarios with variable‐size formation. Subsequently, three air combat sub‐tasks are designed, and recurrent soft actor‐critic (RSAC) algorithm combined with competitive self‐play (SP) is incorporated to learn the sub‐strategies. A novel hierarchical multi‐agent reinforcement learning (HMARL) algorithm is proposed to obtain the high‐level strategy for target and sub‐strategy selection. The training performance of the training algorithm of sub‐strategies and high‐level strategy in different air combat scenarios is evaluated. The obtained strategies are analyzed and it is found that the formations exhibit effective cooperative behavior in symmetric and asymmetric scenarios. Finally, the ideas of engineering implementation of the maneuvering control architecture are given. The study provides a solution for future multi‐aircraft autonomous air combat.

show abstract