Reinforcement learning–based QoS/QoE‐aware service function chaining in software‐driven 5G slices

Chen, Xi; Li, Zonghang; Zhang, Yupeng; Long, Ruiming; Yu, Hongfang; Du, Xiaojiang; Guizani, Mohsen

doi:10.1002/ett.3477

Cited by 48 publications

(34 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [149] propose a DQL scheme for QoS/QoEaware SFC in NFV-enabled 5G systems. Typical QoS metrics are bandwidth, delay, throughput, etc.…”

Section: B Resource Sharing and Schedulingmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,476

481

View full text Add to dashboard Cite

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

show abstract

“…The authors in [149] propose a DQL scheme for QoS/QoEaware SFC in NFV-enabled 5G systems. Typical QoS metrics are bandwidth, delay, throughput, etc.…”

Section: B Resource Sharing and Schedulingmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,476

481

View full text Add to dashboard Cite

show abstract

“…The mini-batch size is set at 64. Both the Q-learning algorithm and the deep reinforcement learning algorithms use -greedy algorithm with the initial value of is 1, and its final value is 0.1 [39], [45]. The maximum size of the experience replay buffer is 10,000, and the target Q-network is updated every 1,000 iterations [35], [46].…”

Section: Performance Evaluation a Parameter Settingmentioning

confidence: 99%

Optimal and Fast Real-Time Resource Slicing With Deep Dueling Neural Networks

Huynh

Hoang

Nguyen

et al. 2019

IEEE J. Select. Areas Commun.

123

View full text Add to dashboard Cite

Effective network slicing requires an infrastructure/network provider to deal with the uncertain demand and real-time dynamics of network resource requests. Another challenge is the combinatorial optimization of numerous resources, e.g., radio, computing, and storage. This article develops an optimal and fast real-time resource slicing framework that maximizes the long-term return of the network provider while taking into account the uncertainty of resource demand from tenants. Specifically, we first propose a novel system model which enables the network provider to effectively slice various types of resources to different classes of users under separate virtual slices. We then capture the real-time arrival of slice requests by a semi-Markov decision process. To obtain the optimal resource allocation policy under the dynamics of slicing requests, e.g., uncertain service time and resource demands, a Q-learning algorithm is often adopted in the literature. However, such an algorithm is notorious for its slow convergence, especially for problems with large state/action spaces. This makes Q-learning practically inapplicable to our case in which multiple resources are simultaneously optimized. To tackle it, we propose a novel network slicing approach with an advanced deep learning architecture, called deep dueling that attains the optimal average reward much faster than the conventional Q-learning algorithm. This property is especially desirable to cope with real-time resource requests and the dynamic demands of users. Extensive simulations show that the proposed framework yields up to 40% higher long-term average return while being few thousand times faster, compared with state of the art network slicing approaches. {Hoang.Dinh, Diep.Nguyen, and Eryk.Dutkiewicz}@uts.edu.au. and development. In order to enhance operators' products for vertical enterprises and provide service customization for emerging massive connections, as well as to give more control to enterprises and mobile virtual network operators, the concept of network slicing has been recently introduced to allow the independent usage of a part of network resources by a group of mobile terminals with special requirements.Network slicing was introduced by Next Generation Mobile Networks Alliance [2], and it has quickly received a lot of attention from both academia and industry. In general, network slicing is a novel virtualization paradigm that enables multiple logical networks, i.e., slices, to be created according to specific technical or commercial demands and simultaneously run on top of the physical network infrastructure. The core idea of the network slicing is using software-defined networking (SDN) and network functions virtualization (NFV) technologies for virtualizing the physical infrastructure and controlling network operations. In particular, SDN provides a separation between the network control and data planes, improving the flexibility of network function management and efficiency of data transfer. Meanwhile, NFV allows various network functio...

show abstract

“…Yu et al 25 exploited the Deep RL concept to sort out the challenges in wireless multiple access control and addresses the benefit of Deep RL in terms of fast convergence to optimal outcomes and robustness against nonoptimal hyper parameter settings. The work Chen et al 26 has addressed a Deep Q-learning method to tackle QOS aware service function chaining in a network function virtualization activated 5G system. The QOS parameters are throughput, delay, and system bandwidth.…”

Section: Introductionmentioning

confidence: 99%

An efficient Deep reinforcement learning with extended Kalman filter for device‐to‐device communication underlaying cellular network

Khuntia

Hazra

2019

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In this paper, a novel Deep Q‐learning in addition with an extended Kalman filter (EKF) is proposed to solve the channel and power allocation issue for a device‐to‐device enabled cellular network, when the prior traffic information is not known to the base station. Furthermore, this paper work explores an optimal policy for resource and power allocation between users with the aim of maximizing the sum‐rate of the overall system. The proposed work comprises of four phases, ie, cell splitting, clustering, queuing model, channel allocation, and power allocation, simultaneously. The implementation of cell splitting with novel K‐means++ clustering technique increases the network coverage, reduces co‐channel cell interference, and minimizes the transmission power of nodes, whereas the M/M/C:N queuing model solves the issue of waiting time for users in a priority based data transmission. The difficulty with the Q‐learning and Deep Q‐learning environment is to achieve an optimal policy. This is because of the uncertainty of various parameters associated with the system, especially when the state space is extremely big. In order to improve the robustness of the learner, EKF together with the Deep Q‐network is presented in this paper, which incorporates weight uncertainty of the Q‐function as well as the state uncertainty during the transition. Furthermore, the use of EKF provides an improved version of the loss function that helps the learner in achieving an optimal policy. Through numerical simulation, the advantage of our resource sharing scheme over the other existing schemes is also verified.

show abstract

Reinforcement learning–based QoS/QoE‐aware service function chaining in software‐driven 5G slices

Cited by 48 publications

References 33 publications

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Optimal and Fast Real-Time Resource Slicing With Deep Dueling Neural Networks

An efficient Deep reinforcement learning with extended Kalman filter for device‐to‐device communication underlaying cellular network

Contact Info

Product

Resources

About