Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments

Yan, Zheng; Meng, Zhaopeng; Hao, Jianye; Zhang, Zongzhang

doi:10.1007/978-3-319-97310-4_48

Cited by 36 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later this idea was applied to arbitrary function approximators, including deep neural networks, i.e., Double DQN [92], which were naturally applied since two networks were already used in DQN (see Section 2.2). These ideas have also been recently applied to MDRL [178].…”

Section: Avoiding Deep Learning Amnesia: Examples In Mdrlmentioning

confidence: 99%

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

439

240

View full text Add to dashboard Cite

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.$ Earlier versions of this work had the title: "Is multiagent deep reinforcement learning the answer or the question? A brief survey" arXiv:1810.05587v3 [cs.MA] 30 Aug 2019 Go [14,15], poker [16,17], and games of two competing teams, e.g., DOTA 2 [18] and StarCraft II [19].While different techniques and algorithms were used in the above scenarios, in general, they are all a combination of techniques from two main areas: reinforcement learning (RL) [20] and deep learning [21,22].RL is an area of machine learning where an agent learns by interacting (i.e., taking actions) within a dynamic environment. However, one of the main challenges to RL, and traditional machine learning in general, is the need for manually designing quality features on which to learn. Deep learning enables efficient representation learning, thus allowing the automatic discovery of features [21,22]. In recent years, deep learning has had successes in different areas such as computer vision and natural language processing [21,22]. One of the key aspects of deep learning is the use of neural networks (NNs) that can find compact representations in high-dimensional data [23].In deep reinforcement learning (DRL) [23,24] deep neural networks are trained to approximate the optimal policy and/or the value function. In this way the deep NN, serving as function approximator, enables powerful generalization. One of the key advantages of DRL is that it enables RL to scale to problems with high-dimensional state and action spaces. However, most existing successful DRL applications so far have been on visual domains (e.g., Atari games), and there is still a lot of work to be done for more realistic applications [25,26] with complex dynamics, which are not necessarily vision-based.DRL h...

show abstract

Section: Avoiding Deep Learning Amnesia: Examples In Mdrlmentioning

confidence: 99%

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

439

240

View full text Add to dashboard Cite

show abstract

“…The main idea of DRL is to maximize the cumulative reward rather than an immediate reward. The principle of DRL is illustrated in Figure 4(d [56], asynchronous actor-critic (A3C) [57], duelling DQN [58], double DQN [59], and multiagent DRL [60]. Researchers are still working to improve the DRL method by integrating LSTM [61] or CNN [62] with it and utilizing the advantages of both architectures in the same network.…”

Section: ) Deep Reinforcement Learning (Drl)mentioning

confidence: 99%

A Survey on Applications of Deep Learning in Cloud Radio Access Network

Rodoshi

Choi

2021

IEEE Access

View full text Add to dashboard Cite

The necessity for high-speed and low-latency connectivity of the vast number of mobile users is rising with the immense usage of mobile applications. A cloud radio access network (C-RAN) is a promising framework for next-generation cellular communication, which can satisfy the requirements of significantly increasing data traffic and user demands. In C-RAN, the data processing unit can be centralized and virtualized in data centers and can be shared among distributed base stations. Deep learning (DL) appears to be a feasible approach for facilitating the data processing capability, resource management in the cloud, and predicting dynamic traffic in cellular communication. The convergence of C-RAN and DL is expected to bring new prospects to both interdisciplinary research and industrial applications. In this regard, different approaches have been proposed for DL-based C-RAN in the literature. This article provides a comprehensive survey of the state-of-the-art DL techniques applied in C-RAN. A brief introduction of the C-RAN architecture and DL techniques is given to provide insights into these two emerging technologies. Existing surveys are also discussed to highlight the research gap. The reviewed works are categorized into power consumption optimization, network performance maximization, and QoS maximization based on their optimization objectives. The key ideas of DL applied in the reviewed schemes are also mentioned, and the performance evaluation techniques used in the research are discussed and compared. Lastly, research challenges and open research issues are highlighted to provide future research directions.INDEX TERMS Cloud radio access network, deep learning, evaluation techniques, optimization objectives, performance metrics.

show abstract

“…Game playing [23] defines the state based on the outputs of APIs and Android testing [25] uses an existing tool UIAutomator [28] to extract structures as the states, which are both not applicable in web testing. Secondly, one fundamental challenge of RL is how to perform effective exploration especially when the space of the environment is huge [29,30]. The existing techniques [21,23,31] mainly guide the exploration with simple reward functions, which could be ineffective for web applications that have complex business logic and frequent dynamic update.…”

Section: Introductionmentioning

confidence: 99%

Automatic Web Testing Using Curiosity-Driven Reinforcement Learning

Yan

Liu

Xie

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

Web testing has long been recognized as a notoriously difficult task. Even nowadays, web testing still heavily relies on manual efforts while automated web testing is far from achieving human-level performance. Key challenges in web testing include dynamic content update and deep bugs hiding under complicated user interactions and specific input values, which can only be triggered by certain action sequences in the huge search space. In this paper, we propose WebExplor, an automatic end-to-end web testing framework, to achieve an adaptive exploration of web applications. WebExplor adopts curiosity-driven reinforcement learning to generate high-quality action sequences (test cases) satisfying temporal logical relations. Besides, WebExplor incrementally builds an automaton during the online testing process, which provides high-level guidance to further improve the testing efficiency. We have conducted comprehensive evaluations of WebExplor on six real-world projects, a commercial SaaS web application, and performed an in-thewild study of the top 50 web applications in the world. The results demonstrate that in most cases WebExplor can achieve significantly higher failure detection rate, code coverage and efficiency than existing state-of-the-art web testing techniques. WebExplor also detected 12 previously unknown failures in the commercial web application, which have been confirmed and fixed by the developers. Furthermore, our in-the-wild study further uncovered 3,466 exceptions and errors.

show abstract

Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments

Cited by 36 publications

References 8 publications

A survey and critique of multiagent deep reinforcement learning

A survey and critique of multiagent deep reinforcement learning

A Survey on Applications of Deep Learning in Cloud Radio Access Network

Automatic Web Testing Using Curiosity-Driven Reinforcement Learning

Contact Info

Product

Resources

About