A Continuous-Time Markov Decision Process-Based Method With Application in a Pursuit-Evasion Example

Jia, Shengde; Wang, Xiangke

doi:10.1109/tsmc.2015.2478875

Cited by 15 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…; 171 Jia et al. ; 172 Feng and Yao; 173 and Changqiang et al. 174 In Oberle, 167 the air-to-air engagement was modeled as Markov process.…”

Section: Mathematics-based Approachesmentioning

confidence: 99%

“…Later air combat-related works that utilize MDP idea are presented in Oberle; 167 Feigin et al; 168 Reuss; 169 Tunay et al; 170 Shengde et al; 171 Jia et al; 172 Feng and Yao; 173 and Changqiang et al 174 In Oberle, 167 the air-to-air engagement was modeled as Markov process. Other factors including the weaponry effectiveness was also involved to estimate the player's offensive/defensive capability in the air combat.…”

Section: Mdp-based Solutionmentioning

confidence: 99%

“…both units have the same type of platform and weaponry effectiveness. In Shengde et al 171 and Jia et al, 172 the authors dealt with PE problem. A continuous-time MDP was proposed to address the uncertainties in the game, the transition time between the adjacent state blocks was also considered.…”

Section: Mdp-based Solutionmentioning

confidence: 99%

See 2 more Smart Citations

Guidance and control for own aircraft in the autonomous air combat: A historical review and future prospects

Dong

Liu

2019

Proceedings of the Institution of Mechanical Engineers, Part G:

View full text Add to dashboard Cite

The Autonomous Air Combat technique has been a lasting research topic for decades. However, no complete solutions seem to have appeared because of the highly dynamic and complex nature of the Autonomous Air Combat problem. In devising the Autonomous Air Combat solutions, we follow similar methodologies in the robotics community, and divide the overall scheme into two folds: the perception of other (enemy/friendly) aircraft, and the guidance/control for own aircraft. While the perception in the first fold serves as a foundation, this paper is mainly focused on the second one. Based on our survey, a review of own aircraft guidance/control in the (primarily one-to-one) Autonomous Air Combat solutions is presented. We divide different Autonomous Air Combat solutions into three groups, i.e. mathematics-based, knowledge-encoded, and learning-driven. In each group, we present the representative methods first; problem definition, solution, and a brief overview of the historical development are illustrated. We also comment on both weakness and strengths for each group/method. We point out certain technical paths/challenges that need to be addressed in the future Autonomous Air Combat development, i.e. to abstract and emulate the human pilot experiences, and to develop the online learning capabilites. Inspired by the state-of-art techniques in other similar fields (robotics, autonomous driving), we also propose potential solutions, i.e. traditional approaches enhanced by the novel data-driven technique. Via this paper, we hope to deliver an in-depth analysis of past experiences and potential challenges/solutions for the Autonomous Air Combat technique. We also advocate referring to the approaches/techniques that are utilized in other similar fields in devising the Autonomous Air Combat solutions.

show abstract

“…; 171 Jia et al. ; 172 Feng and Yao; 173 and Changqiang et al. 174 In Oberle, 167 the air-to-air engagement was modeled as Markov process.…”

Section: Mathematics-based Approachesmentioning

confidence: 99%

Section: Mdp-based Solutionmentioning

confidence: 99%

See 1 more Smart Citation

Guidance and control for own aircraft in the autonomous air combat: A historical review and future prospects

Dong

Liu

2019

Proceedings of the Institution of Mechanical Engineers, Part G:

View full text Add to dashboard Cite

show abstract

“…The main objective is to ensure the capture of the targets via the clearing of the graph in minimal time. In [22], the authors propose a new approach Continuous Time Markov Decision Process (CTMDP) to address the PE problem. In relation to MDP, CTMDP takes into account the impact of the transition time between the states involving a strong robustness against changes in transition probability.…”

Section: Related Workmentioning

confidence: 99%

A New Decentralized Approach of Multiagent Cooperative Pursuit Based on the Iterated Elimination of Dominated Strategies Model

Souidi

Piao

2016

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Game Theory is a promising approach to acquire coalition formations in multiagent systems. This paper is focused on the importance of the distributed computation and the dynamic formation and reformation of pursuit groups in pursuit-evasion problems. In order to address this task, we propose a decentralized coalition formation algorithm based on the Iterated Elimination of Dominated Strategies (IEDS). This Game Theory process is common to solve problems requiring the withdrawal of dominated strategies iteratively. Furthermore, we have used the Markov Decision Process (MDP) principles to control the motion strategy of the agents in the environment. The simulation results demonstrate the feasibility and the validity of the given approach in comparison with different decentralized methods.

show abstract

“…Pan et al designed a region-based relay pursuit scheme [7] for the pursuers to capture a single evader. A policy iteration method-based continuous-time Markov decision process [20] was proposed to optimize the pursuer strategy. [21] employed a graphtheoretic approach to study the interactions of the agents and obtain distributed control policies for pursuers.…”

Section: Introductionmentioning

confidence: 99%

Progression Cognition Reinforcement Learning With Prioritized Experience for Multi-Vehicle Pursuit

Li,

Yang,

Yuan

et al. 2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Multi-vehicle pursuit (MVP) such as autonomous police vehicles pursuing suspects is important but very challenging due to its mission and safety-critical nature. While multiagent reinforcement learning (MARL) algorithms have been proposed for MVP in structured grid-pattern roads, the existing algorithms use random training samples in centralized learning, which leads to homogeneous agents showing low collaboration performance. For the more challenging problem of pursuing multiple evaders, these algorithms typically select a fixed target evader for pursuers without considering dynamic traffic situation, which significantly reduces pursuing success rate. To address the above problems, this paper proposes a Progression Cognition Reinforcement Learning with Prioritized Experience for MVP (PEPCRL-MVP) in urban multi-intersection dynamic traffic scenes. PEPCRL-MVP uses a prioritization network to assess the transitions in the global experience replay buffer according to each MARL agent's parameters. With the personalized and prioritized experience set selected via the prioritization network, diversity is introduced to the MARL learning process, which can improve collaboration and task-related performance. Furthermore, PEPCRL-MVP employs an attention module to extract critical features from dynamic urban traffic environments. These features are used to develop a progression cognition method to adaptively group pursuing vehicles. Each group efficiently targets one evading vehicle. Extensive experiments conducted with a simulator over unstructured roads of an urban area show that PEPCRL-MVP is superior to other state-of-the-art methods. Specifically, PEPCRL-MVP improves pursuing efficiency by 3.95% over Twin Delayed Deep Deterministic policy gradient-Decentralized Multi-Agent Pursuit and its success rate is 34.78% higher than that of Multi-Agent Deep Deterministic Policy Gradient. Codes are open-sourced.

show abstract

A Continuous-Time Markov Decision Process-Based Method With Application in a Pursuit-Evasion Example

Cited by 15 publications

References 16 publications

Guidance and control for own aircraft in the autonomous air combat: A historical review and future prospects

Guidance and control for own aircraft in the autonomous air combat: A historical review and future prospects

A New Decentralized Approach of Multiagent Cooperative Pursuit Based on the Iterated Elimination of Dominated Strategies Model

Progression Cognition Reinforcement Learning With Prioritized Experience for Multi-Vehicle Pursuit

Contact Info

Product

Resources

About