Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

Rezazadeh, Farhad; Chergui, Hatim; Christofi, Loizos; Verikoukis, Christos

doi:10.1109/icc42927.2021.9500265

Cited by 22 publications

(21 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table I presents the network parameters. We set hyperparameters of DNNs through extensive experiments [25]- [26] and adopt a similar architecture for both Actor-Critic and target DNN models. We use 5 hidden layers and 128 units per layer with batch size 128.…”

Section: Numerical Resultsmentioning

confidence: 99%

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Rezazadeh¹,

Chergui²,

Blanco³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI)-driven zero-touch massive network slicing is envisioned to be a disruptive technology in beyond 5G (B5G)/6G, where tenancy would be extended to the final consumer in the form of advanced digital use-cases. In this paper, we propose a novel model-free deep reinforcement learning (DRL) framework, called collaborative statistical Actor-Critic (CS-AC) that enables a scalable and farsighted slice performance management in a 6G-like RAN scenario that is built upon mobile edge computing (MEC) and massive multiple-input multipleoutput (mMIMO). In this intent, the proposed CS-AC targets the optimization of the latency cost under a long-term statistical service-level agreement (SLA). In particular, we consider the Q-th delay percentile SLA metric and enforce some slice-specific preset constraints on it. Moreover, to implement distributed learners, we propose a developed variant of soft Actor-Critic (SAC) with less hyperparameter sensitivity. Finally, we present numerical results to showcase the gain of the adopted approach on our built OpenAIbased network slicing environment and verify the performance in terms of latency, SLA Q-th percentile, and time efficiency. To the best of our knowledge, this is the first work that studies the feasibility of an AI-driven approach for massive network slicing under statistical SLA.

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Rezazadeh¹,

Chergui²,

Blanco³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The simulation results show that this method converges faster than the double DQN and AC methods and outperforms in terms of the mentioned parameters. Also, in [130], a method called twin-delayed double-Q soft Actor-Critic (TDSAC) is provided for allocating CPU resources and controlling power consumption in DU-CU based on the C-RAN architecture. To increase the stability and improve the decision quality in TDSAC, double DQN is used to tune the AC parameters.…”

Section: ) Resource Virtualizationmentioning

confidence: 99%

“…It can be learned from [130] that by tuning the AC parameters at specified times, the decision fluctuation in [35], [129] can be reduced. In the Q-learning-based method ( [109]), if the state and action spaces are very large, then the size of the Q-Table will be very large.…”

Section: ) Lessons Learnedmentioning

confidence: 99%

“…In recent years, the use of distributed methods (such as transfer learning and federated learning) in ML has been considered due to the common challenges in centralized methods [79], [80], [81], [82], [84], [85], [86], [87], [88], [89], [90], [91], [92], [83], [93], [94], [95], [110], [111], [112], [113], [114], [115], [116], [117], [118], [136] ✓ ✓ ---- [96], [97], [98], [99], [100], [103], [104], [105], [123], [124], [135] [107], [108], [109], [126], [127], [128], [35], [129], [130] such as centralization, security, privacy, and time and computational complexity. In distributed methods, a ML mode...…”

Section: Distributed Learningmentioning

confidence: 99%

See 1 more Smart Citation

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

et al. 2022

View full text Add to dashboard Cite

One of the key foundations of 5 th Generation (5G) and beyond 5G (B5G) networks is network slicing, in which the network is partitioned into several separated logical networks, taking into account the requirements of diverse applications. In this context, resource management is of great importance to instantiate and operate network slices and meet their performance and functional requirements. Resource management in Radio Access Networks (RANs) is associated with a range of challenges due to network dynamics and the specific requirements of each application while ensuring performance isolation. In this paper, we present a survey on state-of-the-art works that employ Machine Learning (ML) techniques in RAN slicing. We begin by reviewing the challenges, then we review the existing papers on resource management in a comprehensive manner, and classify the papers based on the used ML algorithm, the addressed challenges, and the type of allocated resources. We evaluate the maturity of current methods and state a number of open challenges and some solutions to address these challenges in RAN resource management.

show abstract

“…For example, Reference [10] proposes a security enhancement to the ZSM architecture, addressing its various identified security threats. The work in [11] introduces the concept of Knowledge Plane (KP) in ZSM. As a pervasive system, the KP provides high-level models of the network's overall functioning.…”

Section: Related Workmentioning

confidence: 99%

ZSM-based Management and Orchestration of 3GPP Network Slicing: An Architectural Framework and Deployment Options

Sajjad¹,

Jayalath²,

Tian³

et al. 2022

Preprint

View full text Add to dashboard Cite

Driven by closed-loop automation, the Zero-Touch Network and Services Management (ZSM) framework offers invigorating features for the end-to-end management and orchestration of a sliced network. Although ZSM is considered a promising framework by 3GPP, there is a lack of concrete ZSM-based solutions for the 3GPP Network Slicing management. This article presents an architectural framework of a ZSM-based management system for 3GPP Network Slicing. The proposed framework employs recursive ZSM management domains to meet the specific management requirements of the standard 3GPP Network Slicing framework. Two deployment options are considered for the presented framework. The first option features the integration of the standard services of the 3GPP Management System within ZSM. The other one considers ZSM as a complementary system for the standard 3GPP Management System. From these deployment options, some key questions are identified on ZSM's interoperability with the existing 3GPP Network Slicing systems. Finally, for each option, the architectural and operational feasibility of the ZSM interoperation with 3GPP Network Slicing systems is shown through an example use case.

show abstract

Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

Cited by 22 publications

References 12 publications

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

ZSM-based Management and Orchestration of 3GPP Network Slicing: An Architectural Framework and Deployment Options

Contact Info

Product

Resources

About