No abstract

Opponent models are necessary in games where the game state is only partially known to the player, since the player must infer the state of the game based on the opponent's actions. This paper presents an architecture and a process for developing neural network game players that utilize explicit opponent models in order to improve game play against unseen opponents. The model is constructed as a mixture over a set of cardinal opponents, i.e. opponents that represent maximally distinct game strategies. The model is trained to estimate the likelihood that the opponent will make the same move as each of the cardinal opponents would in a given game situation. Experiments were performed in the game of Guess It, a simple game of imperfect information that has no optimal strategy for defeating specific opponents. Opponent modeling is therefore crucial to play this game well. Both opponent modeling and game-playing neural networks were trained using NeuroEvolution of Augmenting Topologies (NEAT). The results demonstrate that game-playing provided with the model outperform networks not provided with the model when played against the same previously unseen opponents. The "cardinal mixture" architecture therefore constitutes a promising approach for general and dynamic opponent modeling in gameplaying.

No abstract

This paper proposes a measure-theoretic framework to study iterative stochastic optimizers that provides theoretical tools to explore how the optimization methods may be improved. Within this framework, optimizers form a closed, convex subset of a normed vector space, implying the existence of a distance metric between any two optimizers and a meaningful and computable spectrum of new optimizers between them. It is shown how the formalism applies to evolutionary algorithms in general. The analytic property of continuity is studied in the context of genetic algorithms, revealing the conditions under which approximations such as metamodeling or surrogate methods may be effective. These results demonstrate the power of the proposed analytic framework, which can be used to propose and analyze new techniques such as controlled convex combinations of optimizers, meta-optimization of algorithm parameters, and more.

Standard genetic algorithms can discover good fitness regions and later forget them due to their Markovian structure, resulting in suboptimal performance. Real-Space Evolutionary Annealing (REA) hybridizes simulated annealing and genetic algorithms into a provably convergent evolutionary algorithm for Euclidean space that relies on nonMarkovian selection. REA selects any previously observed solution from an approximated Boltzmann distribution using a cooling schedule. This method enables REA to escape local optima while retaining information about prior generations. In parallel work, REA has been generalized to arbitrary measure spaces and shown to be asymptotically convergent to the global optima. This paper compares REA experimentally to six popular optimization algorithms, including Differential Evolution, Particle Swarm Optimization, Correlated Matrix Adaptation Evolution Strategies, the real-coded Bayesian Optimization Algorithm, a realcoded genetic algorithm, and simulated annealing. REA converges faster to the global optimum and succeeds more often on two out of three multimodal, non-separable benchmarks and performs strongly on all three. In particular, REA vastly outperforms the real-coded genetic algorithm and simulated annealing, proving that the hybridization is better than either algorithm alone. REA is therefore an interesting and effective algorithm for global optimization of difficult fitness functions.