Air-Combat Strategy Using Approximate Dynamic Programming

Abstract

Unmanned Aircraft Systems (UAS) have the potential to perform many of the dangerous missions currently flown by manned aircraft. Yet, the complexity of some tasks, such as air combat, have precluded UAS from successfully carrying out these missions autonomously. This paper presents a formulation of a level flight, fixed velocity, one-on-one air combat maneuvering problem and an approximate dynamic programming (ADP) approach for computing an efficient approximation of the optimal policy. In the version of the problem formulation considered, the aircraft learning the optimal policy is given a slight performance advantage. This ADP approach provides a fast response to a rapidly changing tactical situation, long planning horizons, and good performance without explicit coding of air combat tactics. The method's success is due to extensive feature development, reward shaping and trajectory sampling. An accompanying fast and effective rollout based policy extraction method is used to accomplish on-line implementation. Simulation results are provided that demonstrate the robustness of the method against an opponent beginning from both offensive

A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag

“…Here control strategies are usually generated for one side while assuming some model that predicts the actions of the opposing side. One approach uses nonlinear model-predictive control to generate control inputs for an unmanned aerial vehicle [12], while another uses approximate dynamic programming for a similar application [13]. In both cases, adversary actions are predicted based upon expected opponent strategies, with feedback and re-planning used to adjust for deviations from the predicted actions at run-time.…”

Multiplayer Reach-Avoid Games via Pairwise Outcomes

“…In complex pursuit-evasion games where players may change roles over time, a nonlinear model-predictive control [8] approach has been investigated. Approximate dynamic programming [9] has also been used to analyze reach-avoid games.…”

Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix

“…Blue (10) Here, K H is also a constant and B Critical means the lower bound. These penalty terms are applied to the scoring function exponentially right after the relative orientation of two aircrafts is evaluated.…”

“…To overcome the complexity of calculation and difficulty in real-time simulation, variations of dynamic programming have been suggested by researchers. McGrew et al [9][10] have suggested an approximate dynamic programming method to find the air combat strategy. Approximate dynamic programming is different from classical dynamic programming in that it formulates a continuous function to approximately represent the future reward.…”