Google’s DeepMind and the Future of Reinforcement Learning

Nandy, Abhishek; Biswas, Manisha

doi:10.1007/978-1-4842-3285-9_6

Cited by 6 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These networks are modeled on the complex structure and functionality of the human brain [ 1 ], featuring interconnected layers and nodes that can process intricate information. Consequently, neural networks can accomplish sophisticated tasks such as recognizing images [ 2 ], making predictions [ 3 ], enabling reinforcement learning [ 4 , 5 ], and providing the foundation for various generative technologies [ 6 ].…”

Section: Introductionmentioning

confidence: 99%

Robustness of Sparsely Distributed Representations to Adversarial Attacks in Deep Neural Networks

Sardar,

Khan,

Hintze

et al. 2023

Entropy

View full text Add to dashboard Cite

Deep learning models have achieved an impressive performance in a variety of tasks, but they often suffer from overfitting and are vulnerable to adversarial attacks. Previous research has shown that dropout regularization is an effective technique that can improve model generalization and robustness. In this study, we investigate the impact of dropout regularization on the ability of neural networks to withstand adversarial attacks, as well as the degree of “functional smearing” between individual neurons in the network. Functional smearing in this context describes the phenomenon that a neuron or hidden state is involved in multiple functions at the same time. Our findings confirm that dropout regularization can enhance a network’s resistance to adversarial attacks, and this effect is only observable within a specific range of dropout probabilities. Furthermore, our study reveals that dropout regularization significantly increases the distribution of functional smearing across a wide range of dropout rates. However, it is the fraction of networks with lower levels of functional smearing that exhibit greater resilience against adversarial attacks. This suggests that, even though dropout improves robustness to fooling, one should instead try to decrease functional smearing.

show abstract

Section: Introductionmentioning

confidence: 99%

Robustness of Sparsely Distributed Representations to Adversarial Attacks in Deep Neural Networks

Sardar,

Khan,

Hintze

et al. 2023

Entropy

View full text Add to dashboard Cite

show abstract

“…For this to happen, at least one of the child nodes of 2,1 N at level-3 (e.g., 1 C in Figure 9), say node 3,1 N , must lead to a definite win so that the opponent has no valid move to avoid the computer's win, and at least one of the child nodes of 2,2 N at level-3 (e.g., 3 C in Figure 9), say node Figure 9) must each lead to a definite win, and at least one child node of each of these nodes (e.g., 1 4 , E E in Figure 9) must lead to a definite win. The process repeats until level-k is reached.…”

mentioning

confidence: 99%

Prediction Distortion in Monte Carlo Tree Search and an Improved Algorithm

2018

JILSA

View full text Add to dashboard Cite

Teaching computer programs to play games through machine learning has been an important way to achieve better artificial intelligence (AI) in a variety of real-world applications. Monte Carlo Tree Search (MCTS) is one of the key AI techniques developed recently that enabled AlphaGo to defeat a legendary professional Go player. What makes MCTS particularly attractive is that it only understands the basic rules of the game and does not rely on expert-level knowledge. Researchers thus expect that MCTS can be applied to other complex AI problems where domain-specific expert-level knowledge is not yet available. So far there are very few analytic studies in the literature. In this paper, our goal is to develop analytic studies of MCTS to build a more fundamental understanding of the algorithms and their applicability in complex AI problems. We start with a simple version of MCTS, called random playout search (RPS), to play Tic-Tac-Toe, and find that RPS may fail to discover the correct moves even in a very simple game position of Tic-Tac-Toe. Both the probability analysis and simulation have confirmed our discovery. We continue our studies with the full version of MCTS to play Gomoku and find that while MCTS has shown great success in playing more sophisticated games like Go, it is not effective to address the problem of sudden death/win. The main reason that MCTS often fails to detect sudden death/win lies in the random playout search nature of MCTS, which leads to prediction distortion. Therefore, although MCTS in theory converges to the optimal minimax search, with real world computational resource constraints, MCTS has to rely on RPS as an important step in its search process, therefore suffering from the same fundamental prediction distortion problem as RPS does. By examining the detailed statistics of the scores in MCTS, we investigate a variety of scenarios where MCTS fails to detect sudden death/win. Finally, we propose an improved MCTS algorithm by incorporating minimax search to overcome prediction distortion. Our simulation has confirmed the effectiveness of the proposed algorithm. We provide an estimate of the additional computational

show abstract