Continuous Multi-objective Zero-touch Network Slicing via Twin Delayed DDPG and OpenAI Gym

Rezazadeh, Farhad; Chergui, Hatim; Alonso, Luis; Verikoukis, Christos

doi:10.1109/globecom42002.2020.9322237

“…The DNNs structure for the actor-critic networks and target networks are the same. We have set the hyperparameters following extensive experiments [27]. The evaluation computes every 20000 iterations concerning the average return over the best 3 of 5 episodes.…”

Section: Numerical Resultsmentioning

confidence: 99%

Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

Rezazadeh¹,

Chergui²,

Christofi³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

To harness the full potential of beyond 5G (B5G) communication systems, zero-touch network slicing (NS) is viewed as a promising fully-automated management and orchestration (MANO) system. This paper proposes a novel knowledge plane (KP)-based MANO framework that accommodates and exploits recent NS technologies and is termed KB5G. Specifically, we deliberate on algorithmic innovation and artificial intelligence (AI) in KB5G. We invoke a continuous model-free deep reinforcement learning (DRL) method to minimize energy consumption and virtual network function (VNF) instantiation cost. We present a novel Actor-Critic-based NS approach to stabilize learning called, twin-delayed double-Q soft Actor-Critic (TDSAC) method. The TDSAC enables central unit (CU) to learn continuously to accumulate the knowledge learned in the past to minimize future NS costs. Finally, we present numerical results to showcase the gain of the adopted approach and verify the performance in terms of energy consumption, CPU utilization, and time efficiency.

show abstract

“…Table I presents the network parameters. We set hyperparameters of DNNs through extensive experiments [25]- [26] and adopt a similar architecture for both Actor-Critic and target DNN models. We use 5 hidden layers and 128 units per layer with batch size 128.…”

Section: Numerical Resultsmentioning

confidence: 99%

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Rezazadeh¹,

Chergui²,

Blanco³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Artificial intelligence (AI)-driven zero-touch massive network slicing is envisioned to be a disruptive technology in beyond 5G (B5G)/6G, where tenancy would be extended to the final consumer in the form of advanced digital use-cases. In this paper, we propose a novel model-free deep reinforcement learning (DRL) framework, called collaborative statistical Actor-Critic (CS-AC) that enables a scalable and farsighted slice performance management in a 6G-like RAN scenario that is built upon mobile edge computing (MEC) and massive multiple-input multipleoutput (mMIMO). In this intent, the proposed CS-AC targets the optimization of the latency cost under a long-term statistical service-level agreement (SLA). In particular, we consider the Q-th delay percentile SLA metric and enforce some slice-specific preset constraints on it. Moreover, to implement distributed learners, we propose a developed variant of soft Actor-Critic (SAC) with less hyperparameter sensitivity. Finally, we present numerical results to showcase the gain of the adopted approach on our built OpenAIbased network slicing environment and verify the performance in terms of latency, SLA Q-th percentile, and time efficiency. To the best of our knowledge, this is the first work that studies the feasibility of an AI-driven approach for massive network slicing under statistical SLA.

show abstract

“…There exist 10 APs and a maximum of 17 registered subscribers that are assigned randomly to different slices in each decision time Table I presents the network parameters. We set hyperparameters of DNNs through extensive experiments [25] and adopt a similar architecture for both Actor-Critic and target DNN models. We use 5 hidden layers and 128 units per layer with batch size 128.…”

Section: Numerical Resultsmentioning

confidence: 99%

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Rezazadeh

¹

,

Chergui

²

,

Blanco

³

et al. 2021

2021 IEEE Global Communications Conference (GLOBECOM)

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI)-driven zero-touch massive network slicing is envisioned to be a disruptive technology in beyond 5G (B5G)/6G, where tenancy would be extended to the final consumer in the form of advanced digital use-cases. In this paper, we propose a novel model-free deep reinforcement learning (DRL) framework, called collaborative statistical Actor-Critic (CS-AC) that enables a scalable and farsighted slice performance management in a 6G-like RAN scenario that is built upon mobile edge computing (MEC) and massive multiple-input multipleoutput (mMIMO). In this intent, the proposed CS-AC targets the optimization of the latency cost under a long-term statistical service-level agreement (SLA). In particular, we consider the Q-th delay percentile SLA metric and enforce some slice-specific preset constraints on it. Moreover, to implement distributed learners, we propose a developed variant of soft Actor-Critic (SAC) with less hyperparameter sensitivity. Finally, we present numerical results to showcase the gain of the adopted approach on our built OpenAIbased network slicing environment and verify the performance in terms of latency, SLA Q-th percentile, and time efficiency. To the best of our knowledge, this is the first work that studies the feasibility of an AI-driven approach for massive network slicing under statistical SLA.

show abstract

Continuous Multi-objective Zero-touch Network Slicing via Twin Delayed DDPG and OpenAI Gym

Cited by 32 publications

References 11 publications

Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Contact Info

Product

Resources

About