Combating Coevolutionary Disengagement by Reducing Parasite Virulence

Abstract

While standard evolutionary algorithms employ a static, absolute fitness metric, co-evolutionary algorithms assess individuals by their performance relative to populations of opponents that are themselves evolving. Although this arrangement offers the possibility of avoiding long-standing difficulties such as premature convergence, it suffers from its own unique problems, cycling, over-focusing and disengagement. Here, we introduce a novel technique for dealing with the third and least explored of these problems. Inspired by studies of natural host-parasite systems, we show that disengagement can be avoided by selecting for individuals that exhibit reduced levels of “virulence”, rather than maximum ability to defeat coevolutionary adversaries. Experiments in both simple and complex domains are used to explain how this counterintuitive approach may be used to improve the success of coevolutionary algorithms.

Relationship Between Generalization and Diversity in Coevolutionary Learning

“…In a biological coevolutionary system, highly virulent parasites could potentially incapacitate the hosts, which in turn, reduce the chances for parasites to reproduce. In coevolutionary learning, similar observations have been made whereby strong competing solutions can quickly take over the population, leading to coevolutionary disengagement where the system is unable to differentiate between the solutions for continued evolutionary search [25].…”

“…In [25], a simple approach that is inspired from the natural interactions between hosts and parasites was proposed. In a biological coevolutionary system, highly virulent parasites could potentially incapacitate the hosts, which in turn, reduce the chances for parasites to reproduce.…”

“…In the context of game playing, one form of reducing virulence is to lower the fitness of the top performing strategies in the coevolving population that outperform the largest number of the opponents. Following [25], the virulence level is defined as the tendency to maximize fitness and a reduced virulence (RV) level changes an evolved strategy's original fitness value to (3) where . A coevolutionary learning system with maximum virulence level uses , while a system with RV level has in the range of (e.g., moderate virulence level ).…”

“…For the first issue, one is concerned with the coevolutionary learning of strategies with high generalization performance, that is, strategies that can outperform against a large number of test strategies (opponents) that may not have been seen during coevolution [26], [29], [30]. For the second issue, one is concerned with diversity levels in coevolutionary learning [25], [27], [31].…”

“…Although earlier studies have turned to various empirical approaches, there is a lack of rigorous analysis to investigate this issue in terms of generalization performance measures in coevolutionary learning [29], [32], [33]. Furthermore, these past studies made little attempt to measure diversity levels in coevolutionary learning despite strong claims that they are increased while investigating their impact on performance [24], [25], [29], [33].…”