Co-Evolutionary Optimization of Autonomous Agents in a Real-Time Strategy Game

A Systematic Review of Coevolution in Real-Time Strategy Games

“…The most frequent suggestion raised by researchers was using different fitness functions; some researchers have not determined a specific function type (as in [63], [19], [51], [66], and [67]). On the other hand, other researchers were able to suggest specific function types, such as dynamic functions [75], visualisation tools to support determining the fitness [37], automated shaping functions [22], and mathematically approximated functions [25]. Exploration of different CA techniques is another CA related suggestion that was reported by researchers.…”

“…It is worth mentioning that two of the games used in the reviewed papers were turn based, rather than actual RTS games. Bellus Bellum Gratia (BBG) was used in [33], [34], and Planet Wars, the Google AI challenge introduced in 2010, was used in [18], [25], [53]. These papers are still considered in this review as the games used here were adjusted and equipped with features that made them closer to RTS games than TBS games, as will be explained later.…”

“…Apart from the previous collective studies, a number of individual works were conducted during this stage. In 2014, Fernandez-Ares et al [25] developed a system intending to enhance the action controller of the players by improving the relevant behavioural parameters in the Planet Wars game. Planet Wars itself is a pseudo-turn-based game; it allows bots to decide on their following action in one-second microturns, which then happen at the same simulated time.…”

“…[37], [15], [52], [19], [51], [10], [9], [25], [66], [11], [18], [12], [53], [67] [63], [42], [75], [33], [34] GP [44] Neural Networks vs Genetic Programming vs Modular controller [73] game strategies. We do not have an explanation for this; however, maybe the lack of information on the used memory technique hinders the appearance of using this technique for other coevolution purposes.…”

“…Exploring the use of cooperative coevolution is a possible way to improve CAs [40], [45], [60], [72]. Further improvement to CAs might be achieved by using a multipopulation approach [25], [30], [40], [53]). Utilising the coevolved plans in predicting opponents' plans was also reported by researchers [9], [11], [33], [52]).…”