Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

Brown, Noam; Bakhtin, Anton; Lerer, Adam; Gong, Qucheng

doi:10.48550/arxiv.2007.13544

Cited by 9 publications

(12 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We have shown that many We see at least three benefits of the public-state formulation of CFR. The first is conceptual: Many recent extensions of CFR (e.g., [2,19]) heavily rely on decomposition and public states. Since PS-CFR is also formulated in terms of public states, PS-CFR serves as a much more suitable basis for these extensions than Vanilla-CFR.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game

Kovařík¹,

Milec²,

Šustr³

et al. 2021

Preprint

View full text Add to dashboard Cite

Many recent results in imperfect information games were only formulated for, or evaluated on, poker and poker-like games such as liar's dice. We argue that sequential Bayesian games constitute a natural class of games for generalizing these results. In particular, this model allows for an elegant formulation of the counterfactual regret minimization algorithm, called publicstate CFR (PS-CFR), which naturally lends itself to an efficient implementation. Empirically, solving a poker subgame with 10 7 states by public-state CFR takes 3 minutes and 700 MB while a comparable version of vanilla CFR takes 5.5 hours and 20 GB. Additionally, the public-state formulation of CFR opens up the possibility for exploiting domain-specific assumptions, leading to a quadratic reduction in asymptotic complexity (and a further empirical speedup) over vanilla CFR in poker and other domains. Overall, this suggests that the ability to represent poker as a Bayesian extensive game played a key role in the success of CFR-based methods. Finally, we extend public-state CFR to general extensive-form games, arguing that this extension enjoys some -but not all -of the benefits of the version for sequential Bayesian games.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The joint probability distribution over types is common knowledge, but each player only knows their own type. (2) The players play a perfect information extensiveform game. (3) Rewards in this game depend not only on actions taken but also on the private types of all players.…”

Section: Introductionmentioning

confidence: 99%

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game

Kovařík¹,

Milec²,

Šustr³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…RL methods have achieved superhuman levels of performance in sequential decision making, famously for arcade games (Mnih et al, 2013), board games (Silver et al, 2017), online multiplayer games (Berner et al, 2019) and imperfect-information games (Brown et al, 2020). These successes demonstrated RL's ability to operate on ambiguous data, understand complex environments and infer high-level causal relationships.…”

Section: Reinforcement Learning Reasoningmentioning

confidence: 99%

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

Edwards¹,

Nilsson²,

Rózemberczki³

et al. 2021

Preprint

View full text Add to dashboard Cite

For Artificial Intelligence to have a greater impact in biology and medicine, it is crucial that recommendations are both accurate and transparent. In other domains, a neurosymbolic approach of multi-hop reasoning on knowledge graphs has been shown to produce transparent explanations. However, there is a lack of research applying it to complex biomedical datasets and problems. In this paper, the approach is explored for drug discovery to draw solid conclusions on its applicability. For the first time, we systematically apply it to multiple biomedical datasets and recommendation tasks with fair benchmark comparisons. The approach is found to outperform the best baselines by 21.7% on average whilst producing novel, biologically relevant explanations.

show abstract

“…Nash equilibrium [39], an important concept in game theory, which is the best strategy for any player no matter what strategies the other players chose. Due to the above characteristic, researchers have paid much attention on approaching Nash equilibrium [40], [41]. Tree search methods have long been a mainstream for turn based games.…”

Section: How To Reach Nash Equilibrium?mentioning

confidence: 99%

AI in Human-computer Gaming: Techniques, Challenges and Opportunities

Yin¹,

Yang²,

Huang³

et al. 2021

Preprint

View full text Add to dashboard Cite

With breakthrough of AlphaGo, AI in human-computer game has become a very hot topic attracting researchers all around the world, which usually serves as an effective standard for testing artificial intelligence. Various game AI systems (AIs) have been developed such as Libratus, OpenAI Five and AlphaStar, beating professional human players. In this paper, we survey recent successful game AIs, covering board game AIs, card game AIs, first-person shooting game AIs and real time strategy game AIs. Through this survey, we 1) compare the main difficulties among different kinds of games for the intelligent decision making field ; 2) illustrate the mainstream frameworks and techniques for developing professional level AIs; 3) raise the challenges or drawbacks in the current AIs for intelligent decision making; and 4) try to propose future trends in the games and intelligent decision making techniques. Finally, we hope this brief review can provide an introduction for beginners, inspire insights for researchers in the filed of AI in games.

show abstract

Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

Cited by 9 publications

References 28 publications

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

AI in Human-computer Gaming: Techniques, Challenges and Opportunities

Contact Info

Product

Resources

About