A Survey of Deep Q-Networks used for Reinforcement Learning: State of the Art

Hafiz, Abdul Mueed

doi:10.1007/978-981-19-1844-5_30

Cited by 9 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the characteristics and requirements of the problem, choosing a suitable centralized reinforcement learning method can improve the learning effect and decision quality of the agent. Common algorithms include Q-learning, DQNs (deep Q-networks) [127], policy gradient methods [128], proximal policy optimization, etc. Q-learning is a basic centralized reinforcement learning method to make optimal decisions by learning a value function.…”

Section: Concentrated Reinforcement Learningmentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

Extensive research has been carried out on reinforcement learning methods. The core idea of reinforcement learning is to learn methods by means of trial and error, and it has been successfully applied to robotics, autonomous driving, gaming, healthcare, resource management, and other fields. However, when building reinforcement learning solutions at the edge, not only are there the challenges of data-hungry and insufficient computational resources but also there is the difficulty of a single reinforcement learning method to meet the requirements of the model in terms of efficiency, generalization, robustness, and so on. These solutions rely on expert knowledge for the design of edge-side integrated reinforcement learning methods, and they lack high-level system architecture design to support their wider generalization and application. Therefore, in this paper, instead of surveying reinforcement learning systems, we survey the most commonly used options for each part of the architecture from the point of view of integrated application. We present the characteristics of traditional reinforcement learning in several aspects and design a corresponding integration framework based on them. In this process, we show a complete primer on the design of reinforcement learning architectures while also demonstrating the flexibility of the various parts of the architecture to be adapted to the characteristics of different edge tasks. Overall, reinforcement learning has become an important tool in intelligent decision making, but it still faces many challenges in the practical application in edge computing. The aim of this paper is to provide researchers and practitioners with a new, integrated perspective to better understand and apply reinforcement learning in edge decision-making tasks.

show abstract

Section: Concentrated Reinforcement Learningmentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…DQN [50] is used to train an agent for gameplay, in which a convolution neural network is adopted to extract the features of input frames. The states are frame sequences, and the actions are game operations.…”

Section: Training Processmentioning

confidence: 99%

Balanced-DRL: A DQN-Based Job Allocation Algorithm in BaaS

Guo

et al. 2023

Mathematics

View full text Add to dashboard Cite

Blockchain as a Service (BaaS) combines features of cloud computing and blockchain, making blockchain applications more convenient and promising. Although current BaaS platforms have been widely adopted by both industry and academia, concerns arise regarding their performance, especially in job allocation. Existing BaaS job allocation strategies are simple and do not guarantee load balancing due to the dynamic nature and complexity of BaaS job execution. In this paper, we propose a deep reinforcement learning-based algorithm, Balanced-DRL, to learn an optimized allocation strategy in BaaS based on analyzing the execution process of BaaS jobs and a set of job scale characteristics. Following extensive experiments with generated job request workloads, the results show that Balanced-DRL significantly improves BaaS performance, achieving a 5% to 8% increase in job throughput and a 5% to 20% decrease in job latency.

show abstract

“…The traditional deep Q network (DQN) algorithm is commonly used in reinforcement learning. It was the first mature algorithm to combine deep learning and reinforcement learning [13]. The deep Q network algorithm has demonstrated better performance than the previous algorithm in many experiments [14,15].…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Intelligent Penetration Testing Path Design

Yi,

Liu

2023

Applied Sciences

View full text Add to dashboard Cite

Penetration testing is an important method to evaluate the security degree of a network system. The importance of penetration testing attack path planning lies in its ability to simulate attacker behavior, identify vulnerabilities, reduce potential losses, and continuously improve security strategies. By systematically simulating various attack scenarios, it enables proactive risk assessment and the development of robust security measures. To address the problems of inaccurate path prediction and difficult convergence in the training process of attack path planning, an algorithm which combines attack graph tools (i.e., MulVAL, multi-stage vulnerability analysis language) and the double deep Q network is proposed. This algorithm first constructs an attack tree, searches paths in the attack graph, and then builds a transfer matrix based on depth-first search to obtain all reachable paths in the target system. Finally, the optimal path for target system attack path planning is obtained by using the deep double Q network (DDQN) algorithm. The MulVAL double deep Q network(MDDQN) algorithm is tested in different scale penetration testing environments. The experimental results show that, compared with the traditional deep Q network (DQN) algorithm, the MDDQN algorithm is able to reach convergence faster and more stably and improve the efficiency of attack path planning.

show abstract

A Survey of Deep Q-Networks used for Reinforcement Learning: State of the Art

Cited by 9 publications

References 34 publications

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Balanced-DRL: A DQN-Based Job Allocation Algorithm in BaaS

Deep Reinforcement Learning for Intelligent Penetration Testing Path Design

Contact Info

Product

Resources

About