Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

Glavin, Frank G.; Madden, Michael G.

doi:10.1109/cig.2018.8490405

Cited by 10 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The game development community recognizes game balancing as a key characteristic of a successful game ( Rouse, 2000 ). One approach for difficulty adjustment is that the player selects a few static difficulty levels (for example, easy, medium, and hard) before the game starts ( Anagnostou & Maragoudakis, 2012 ; Andrade et al, 2006 ; Glavin & Madden, 2018 ; Sutoyo et al, 2015 ). The selected difficulty has a direct and usually static effect on the skill of non-player characters (AIs).…”

Section: Related Workmentioning

confidence: 99%

“…DDA is the process of dynamically adjusting the level of difficulty involving an AI’s skill according to the player’s ability in real time in a computer game. The goal of DDA is to ensure that the game remains challenging and can cater to many different players with varying abilities ( Glavin & Madden, 2018 ). Many researchers have already proposed many DDA methods.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

Fujita

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Artificial intelligence (AI) has achieved superhuman performance in board games such as Go, chess, and Othello (Reversi). In other words, the AI system surpasses the level of a strong human expert player in such games. In this context, it is difficult for a human player to enjoy playing the games with the AI. To keep human players entertained and immersed in a game, the AI is required to dynamically balance its skill with that of the human player. To address this issue, we propose AlphaDDA, an AlphaZero-based AI with dynamic difficulty adjustment (DDA). AlphaDDA consists of a deep neural network (DNN) and a Monte Carlo tree search, as in AlphaZero. AlphaDDA learns and plays a game the same way as AlphaZero, but can change its skills. AlphaDDA estimates the value of the game state from only the board state using the DNN. AlphaDDA changes a parameter dominantly controlling its skills according to the estimated value. Consequently, AlphaDDA adjusts its skills according to a game state. AlphaDDA can adjust its skill using only the state of a game without any prior knowledge regarding an opponent. In this study, AlphaDDA plays Connect4, Othello, and 6x6 Othello with other AI agents. Other AI agents are AlphaZero, Monte Carlo tree search, the minimax algorithm, and a random player. This study shows that AlphaDDA can balance its skill with that of the other AI agents, except for a random player. AlphaDDA can weaken itself according to the estimated value. However, AlphaDDA beats the random player because AlphaDDA is stronger than a random player even if AlphaDDA weakens itself to the limit. The DDA ability of AlphaDDA is based on an accurate estimation of the value from the state of a game. We believe that the AlphaDDA approach for DDA can be used for any game AI system if the DNN can accurately estimate the value of the game state and we know a parameter controlling the skills of the AI system.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

Fujita

2022

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…The game development community recognizes game balancing as a key characteristic for a successful game (Rouse, 2000). One approach for difficulty adjustment is that the player selects a few static difficulty levels (for example, easy, medium, and hard) before the game starts (Anagnostou and Maragoudakis, 2012;Andrade et al, 2006;Glavin and Madden, 2018;Sutoyo et al, 2015). The selected difficulty has a direct and usually static effect on the skill of the non-player characters (AIs).…”

Section: Related Workmentioning

confidence: 99%

“…DDA is the process of dynamically adjusting the level of difficulty containing AI's skill according to the player's ability in real-time in a computer game. The goal of DDA is to ensure that the game remains challenging and can cater to many different players of varying ability (Glavin and Madden, 2018).…”

Section: Related Workmentioning

confidence: 99%

AlphaDDA: Game artificial intelligence with dynamic difficulty adjustment using AlphaZero

Fujita¹

2021

Preprint

View full text Add to dashboard Cite

An artificial intelligence (AI) player has obtained superhuman skill for games like Go, Chess, and Othello (Reversi). In other words, the AI player becomes too strong as an opponent of human players. Then, we will not enjoy playing board games with the AI player. In order to entertain human players, the AI player is required to balance its skill with the human player's one automatically. To address this issue, I propose AlphaDDA, an AI player with dynamic difficulty adjustment based on AlphaZero. AlphaDDA consists of a deep neural network (DNN) and Monte Carlo tree search like AlphaZero. AlphaDDA estimates the value of the game state form only the board state using the DNN and changes its skill according to the value. AlphaDDA can adjust AlphaDDA's skill using only the state of a game without prior knowledge about an opponent. In this study, AlphaDDA plays Connect4, 6x6 Othello, which is Othello using a 6x6 size board, and Othello with the other AI agents. The other AI agents are AlphaZero, Monte Carlo tree search, Minimax algorithm, and a random player. This study shows that AlphaDDA achieves to balance its skill with the other AI agents except for a random player. AlphaDDA's DDA ability is derived from the accurate estimation of the value from the state of a game. We will be able to use the approach of AlphaDDA for any games in that the DNN can estimate the value from the state.

show abstract

“…The balance of the card game, Top Trumps, is optimised using both single-objective [7] and multi-objective optimisation [8], although the cards in Top Trumps are not orthogonally differentiated. Reinforcement learning (RL) techniques have been used for dynamic difficulty adjustment of single player games [9], [10], but RL has not yet been adopted as a tool to balance multiplayer games. Other approaches to automated game analysis include [11] which measures the interestingness of games as a function of several aspects such as game length, drawing quota and variability.…”

Section: Introductionmentioning

confidence: 99%

Balancing Turn-Based Games With Chained Strategy Generation

et al. 2021

View full text Add to dashboard Cite

Probabilistic model checking can overcome much of the complexity inherent in balancing games. Game balancing is the careful maintenance of relationships between the ways in which a game can be played, to ensure no single way is strictly better than all others and that players are offered a wide variety of ways to play successfully. We introduce a novel approach towards automating game balancing using probabilistic model checking called chained strategy generation (CSG). This involves generating chains of adversarial strategies which mimic the way players adapt their approach during repeated plays of a game. We use CSG to map out the evolving metagame. The trends identified can allow game developers to identify strategies which will be too strong and ways of playing the game which a player may want to use, but are never viable for successful competitive play. We introduce a case study, a game called RPGLite, and use CSG to compare five candidate configurations for the game. We show how to determine which configurations of RPGLite lead to a more fair and interesting experience for players. We also identify unexpected trends in how the strategies evolve. Our approach introduces a new technique for improving game development and player experience.

show abstract

Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

Cited by 10 publications

References 11 publications

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

AlphaDDA: Game artificial intelligence with dynamic difficulty adjustment using AlphaZero

Balancing Turn-Based Games With Chained Strategy Generation

Contact Info

Product

Resources

About