Towards Game-based Metrics for Computational Co-Creativity

Canaan, Rodrigo; Menzel, Stefan; Togelius, Julian; Nealen, Andy

doi:10.1109/cig.2018.8490429

Cited by 10 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Information on the fitness landscapes are crucial for the choice of suitable optimisation algorithms as well as for putting their performance into context. The need for further analysis of existing evaluation functions has also been recognised in other publications as well, see for example recent surveys and vision papers [19,130]. This is the reason we are investigating the function suites in GBEA in more detail.…”

Section: Analysis Of Fitness Landscapesmentioning

confidence: 86%

See 1 more Smart Citation

Uncertainty Handling in Surrogate Assisted Optimisation of Games

Volz

2019

Künstl Intell

View full text Add to dashboard Cite

A detailed illustrative example of how modelling human players using AIs can bias the obtained outcomes based on published data for the game StarCraft II (Blizzard 2010). The example conclusively shows the need for uncertainty handling in simulation-based game-related optimisation.The example is based on an algorithm for StarCraft II winner prediction as published in [VPB18] with my co-authors Mike Preuss and Mathias K. Bonde. The analysis presented in this paper was however not published yet and is based on newly generated results from different datasets. The example also profited from discussion with Boris Naujoks. CHAPTER 1. INTRODUCTION Game-Benchmark for Evolutionary AlgorithmsAn extension of the existing COCO benchmarking framework for numerical black-box optimisation 3 that introduces two new functions suites based on game optimisation problems. The benchmark code, results and further information is freely available 4 . Two workshops around the benchmark were also organised for GECCO 2018 and 2019 in collaboration with Tea Tušar, Boris Naujoks and Pascal Kerschke. The interface for the new functions to the existing framework was implemented by Tea Tušar.The new function suites are based on two previous publications. The first is an optimisation problem for cards in TopTrumps that was introduced to demonstrate the feasibility of automatic game balancing with and without surrogate models in collaboration with Boris Naujoks and Günter Rudolph in [VRN16]. The second function suite is based on a significantly extended version of a procedural content generation technique for Super Mario Bros. levels proposed in [Vol+18] developed with SAPEO Algorithm SAPEO, an algorithm designed for the robust optimisation of games and similar complex simulation-based real-world problems. The algorithm is extensively evaluated theoretically and empirically. Benchmarks on artificial functions do attest SAPEO robust performance, especially for complex functions. However, we were not able to obtain a meaningful interpretation of the algorithm's performance on the game benchmark.Previous versions of SAPEO have been published for both single-and multi-objective optimisation problems in [VRN17a] and [VRN17b] in collaboration with Günter Rudolph and Boris Naujoks 6 . The version of SAPEO proposed in this thesis is however improved based on findings from previous publications and extended by model validation features as suggested by Alma Rahat. The results presented in this thesis are thus novel and not previously published. The theoretical performance assessment was supported by Michael Emmerich in context of a short term scientific mission 7 .

show abstract

Section: Analysis Of Fitness Landscapesmentioning

confidence: 86%

“…In its current version (2018), the framework uses the hypervolume of all evaluated individuals as a performance indicator. 19…”

Section: Bbob-biobj Suitementioning

confidence: 99%

Uncertainty Handling in Surrogate Assisted Optimisation of Games

Volz

2019

Künstl Intell

View full text Add to dashboard Cite

show abstract

“…Scores range from 0, for no discernible effort, over 5 for settlements where it becomes unclear if this was done by a human or AI, to 10, for an artifact that looks superhuman, or could only have been built manually by a team of experts with a lot of time. The detailed guide for the judges is also available online for competitors as reference 1 .…”

Section: Introductionmentioning

confidence: 99%

“…In other words, competitors might have created generators that achieved a high value according to an objective function while generating settlements that were unsatisfying to a human. Additionally, creating a meaningful yet well-defined function to optimize for also proved quite hard [1].…”

Section: Introductionmentioning

confidence: 99%

The AI Settlement Generation Challenge in Minecraft

et al. 2020

Self Cite

View full text Add to dashboard Cite

This article outlines what we learned from the first year of the AI Settlement Generation Competition in Minecraft, a competition about producing AI programs that can generate interesting settlements in Minecraft for an unseen map. This challenge seeks to focus research into adaptive and holistic procedural content generation. Generating Minecraft towns and villages given existing maps is a suitable task for this, as it requires the generated content to be adaptive, functional, evocative and aesthetic at the same time. Here, we present the results from the first iteration of the competition. We discuss the evaluation methodology, present the different technical approaches by the competitors, and outline the open problems.

show abstract

“…Cooperative games with imperfect information are an interesting research topic not only due to the added challenges posed to researchers, but also because many modern industrial and commercial applications can be seen as examples of cooperation between humans and machines in order to achieve a mutual goal in an uncertain environment. In previous work [4], we surveyed metrics and open problems relating to co-creative and mixed initiative systems, and argued that games, especially cooperative games, are an ideal research platform to address some of these issues.…”

Section: Introductionmentioning

confidence: 99%

Diverse Agents for Ad-Hoc Cooperation in Hanabi

Canaan

Togelius

Nealen

et al. 2019

2019 IEEE Conference on Games (CoG)

Self Cite

View full text Add to dashboard Cite

In complex scenarios where a model of other actors is necessary to predict and interpret their actions, it is often desirable that the model works well with a wide variety of previously unknown actors. Hanabi is a card game that brings the problem of modeling other players to the forefront, but there is no agreement on how to best generate a pool of agents to use as partners in ad-hoc cooperation evaluation. This paper proposes Quality Diversity algorithms as a promising class of algorithms to generate populations for this purpose and shows an initial implementation of an agent generator based on this idea. We also discuss what metrics can be used to compare such generators, and how the proposed generator could be leveraged to help build adaptive agents for the game.

show abstract

Towards Game-based Metrics for Computational Co-Creativity

Cited by 10 publications

References 28 publications

Uncertainty Handling in Surrogate Assisted Optimisation of Games

Uncertainty Handling in Surrogate Assisted Optimisation of Games

The AI Settlement Generation Challenge in Minecraft

Diverse Agents for Ad-Hoc Cooperation in Hanabi

Contact Info

Product

Resources

About