Adaptive Simulation-Based Training of Artificial-Intelligence Decision Makers Using Bayesian Optimization

Abstract

This work studies how an AI-controlled dog-fighting agent with tunable decisionmaking parameters can learn to optimize performance against an intelligent adversary, as measured by a stochastic objective function evaluated on simulated combat engagements. Gaussian process Bayesian optimization (GPBO) techniques are developed to automatically learn global Gaussian Process (GP) surrogate models, which provide statistical performance predictions in both explored and unexplored areas of the parameter space. This allows a learning engine to sample full-combat simulations at parameter values that are most likely to optimize performance and also provide highly informative data points for improving future predictions. However, standard GPBO methods do not provide a reliable surrogate model for the highly volatile objective functions found in aerial combat, and thus do not reliably identify global maxima.These issues are addressed by novel Repeat Sampling (RS) and Hybrid Repeat/Multipoint Sampling (HRMS) techniques. Simulation studies show that HRMS improves the accuracy of GP surrogate models, allowing AI decision-makers to more accurately predict performance and efficiently tune parameters.

The Combination of Artificial Intelligence and Extended Reality: A Systematic Review

“…To serve as a credible adversary, the AI agent must determine the human participant's skill level and then adapt accordingly (Israelsen et al, 2018). For this purpose, the AI agent can be trained in an interactive environment against an intelligent (AI) adversary to learn how to optimize performance.…”

“…For this purpose, the AI agent can be trained in an interactive environment against an intelligent (AI) adversary to learn how to optimize performance. For this AI-AI interaction in an interactive virtual environment, Gaussian process Bayesian optimization techniques have been used to optimize the AI agent's performance (Israelsen et al, 2018).…”

Weak in the NEES?: Auto-Tuning Kalman Filters with Bayesian Optimization

“…Furthermore, the best local optimum may be undesirable for learning with sparse data early on in the GPBO process, since the associated Θ values typically overfit the training data [7], [4]. This behavior is especially important to consider when trying to minimize the number of simulations for GPBO [8].…”

“…Note that the use of a non-convex optimization technique like DIRECT makes sense here, since they key idea behind Bayesian optimization is that evaluation of a(q) at multiple test points q will be cheaper and faster than evaluating y at those points directly. In this work, we use the classical approach of selecting a single new design point q on each iteration of GPBO, although variations to sample multiple design points at once or repeatedly on each iteration are also possible [8].…”

Nondominated Maneuver Strategy Set With Tactical Requirements for a Fighter Against Missiles in a Dogfight

“…That is, BVR air combat emphasizes the distance game and the key of evasive maneuvers in these situations is tactical planning. The second is "dogfight", in which the main features are relatively close distance, high-dynamic, and intense confrontation [4]. There is more of an emphasis on the game of space angle.…”