A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Rezazadeh, Farhad; Chergui, Hatim; Blanco, Luis; Alonso, Luis; Verikoukis, Christos

doi:10.1109/globecom46510.2021.9685218

Cited by 10 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can measure the model trustworthiness via a KL divergence distance between original predictions and those emanating from an XAI-based masked dataset. Moreover, It can also be an extra reward signal in DRL [27] or a logical reasoning term in neuro-symbolic AI formulations, as explained in the following. Finally, iii) a recall constraint might be considered to enforce the desired fairness level [69].…”

Section: Preliminaries a In-hoc Explainable Artificial Intelligence I...mentioning

confidence: 99%

“…Within the scope of our research, we extend the boundaries of XAI by introducing a novel approach to in-hoc explainability [26], [27] (See Section II-A) solutions based on a symbolic subsystem. Here, XAI techniques can be utilized to incorporate explanations through regularization, supervision, or intervention mechanisms [28] directly into the training phase of GRL.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Rezazadeh,

Zanzi,

Devoti

et al. 2023

IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)

View full text Add to dashboard Cite

The move toward artificial intelligence (AI)-native sixth-generation (6G) networks has put more emphasis on the importance of explainability and trustworthiness in network management operations, especially for mission-critical use-cases. Such desired trust transcends traditional post-hoc explainable AI (XAI) methods to using contextual explanations for guiding the learning process in an in-hoc way. This paper proposes a novel graph reinforcement learning (GRL) framework named TANGO which relies on a symbolic subsystem. It consists of a Bayesian-graph neural network (GNN) Explainer, whose outputs, in terms of edge/node importance and uncertainty, are periodically translated to a logical GRL reward function. This adjustment is accomplished through defined symbolic reasoning rules within a Reasoner. Considering a real-world testbed proof-of-concept (PoC), a gNodeB (gNB) radio resource allocation problem is formulated, which aims to minimize under-and over-provisioning of physical resource blocks (PRBs) while penalizing decisions emanating from the uncertain and less important edge-nodes relations. Our findings reveal that the proposed in-hoc explainability solution significantly expedites convergence compared to standard GRL baseline and other benchmarks in the deep reinforcement learning (DRL) domain. The experiment evaluates performance in AI, complexity, energy consumption, robustness, network, scalability, and explainability metrics. Specifically, the results show that TANGO achieves a noteworthy accuracy of 96.39% in terms of optimal PRB allocation in inference phase, outperforming the baseline by 1.22×.

show abstract

Section: Preliminaries a In-hoc Explainable Artificial Intelligence I...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Rezazadeh,

Zanzi,

Devoti

et al. 2023

IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)

View full text Add to dashboard Cite

show abstract

“…Motivated by the need for a more cost-effective and agile RAN, the Open RAN (O-RAN) Alliance recently presented a vendor-neutral alternative way of building mobile networks [4], based on disaggregated hardware and interoperable interfaces that allow secure network sharing by means of virtualization. Despite the revolutionary approach, it is still not clear how to efficiently support slicing scenarios [5] characterized by a large number of vertical services. Therefore, we take on this challenge and propose a hierarchical architecture for network slice resource orchestration.…”

mentioning

confidence: 99%

On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration

Rezazadeh¹,

Zanzi²,

Devoti³

et al. 2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

Network slicing enables multiple virtual networks to be instantiated and customized to meet heterogeneous use case requirements over 5G and beyond network deployments. However, most of the solutions available today face scalability issues when considering many slices, due to centralized controllers requiring a holistic view of the resource availability and consumption over different networking domains. In order to tackle this challenge, we design a hierarchical architecture to manage network slices resources in a federated manner. Driven by the rapid evolution of deep reinforcement learning (DRL) schemes and the Open RAN (O-RAN) paradigm, we propose a set of traffic-aware local decision agents (DAs) dynamically placed in the radio access network (RAN). These federated decision entities tailor their resource allocation policy according to the long-term dynamics of the underlying traffic, defining specialized clusters that enable faster training and communication overhead reduction. Indeed, aided by a traffic-aware agent selection algorithm, our proposed Federated DRL approach provides higher resource efficiency than benchmark solutions by quickly reacting to end-user mobility patterns and reducing costly interactions with centralized controllers.

show abstract

“…Despite the revolutionary approach, it is still not clear how to efficiently support slicing scenarios [69] characterized by a large number of vertical services. Therefore, we take on this challenge and propose a hierarchical architecture for network slice resource orchestration.…”

Section: Ran Monitoring Slice Schedulermentioning

confidence: 99%

“…We set ι = 10 PRBs as the minimum resource allocation step. The framework leverages Python programming language, exploiting OpenAI Gym library [69] and interfacing DRL agents with a custom gNB simulator environment [106].…”

Section: Network Architecture and Experiments Parametersmentioning

confidence: 99%

Lifelong AI-driven zero-touch network slicing

Rezazadeh

View full text Add to dashboard Cite

(English) The sixth-generation (6G) network's evolution necessitates advancements in algorithms and architecture to transition from an AI-native to an intrinsic trustworthy automation-native system. Network slicing creates multiple virtual networks for diverse service requirements and is a crucial technology for future communication systems and 6G networks. This technology will facilitate numerous aspects of life, society, and industry, meeting the communication needs of both humans and intelligent machines. This is made possible by employing softwarization and virtualization technologies, including software-defined networking (SDN) and network functions virtualization (NFV), to enable the necessary flexibility and programmability required for network slicing. However, the implementation of network slicing in radio access networks (RAN) is challenging due to the complexity of managing RAN operations. This gap is addressed with zero-touch network slicing, a fully automated management and orchestration scheme, which eliminates the need for fixed contracts and manual intervention. The decision engine, a main component of this technology, employs algorithmic innovation to optimize network resource allocation and handle the challenges of RAN slicing, including energy efficiency, latency, scalability, and trustworthiness. In resource allocation, a joint optimization approach is used to balance energy and latency with service quality. A stochastic Actor-Critic approach is proposed to streamline the learning procedure and reduce the need for hyperparameter tuning in an energy-aware network slicing setup. Additionally, a massive deep reinforcement learning-based actor-learner framework is introduced to tackle complexity issues and control challenges in network slicing. The temporal variations of traffic demand pose a significant challenge for resource planning and allocation in the RAN domain. To mitigate this, a distributed architecture for RAN slice resource orchestration is proposed, featuring multiple AI-enabled decision agents that perform local radio allocation decisions without a centralized control entity. A federated learning scheme, aligned with the recent development of the Open RAN architecture, is designed to enhance local decision-making capabilities. Despite AI's effectiveness, concerns persist regarding the lack of transparency in deep neural networks, posing risks to reliability and security in network scenarios. This lack of trust prevents telecommunications operators from widely deploying AI models in their networks. To address this, we proposed SliceOps architecture that consolidates explainable ML operations in a standalone slice, offering AI services to other slices. This setup enhances the reliability and interpretability of AI models, facilitating swift deployment in the network with greater consistency. By transitioning from AI-native to automation-native, this framework manages both service and underlying AI functions, allowing full potential harnessing of slicing in RAN while reducing complexities. (Español) La evolución hacia la red de sexta generación (6G) requiere avances en algoritmos y arquitectura para pasar de un sistema nativo de IA a un sistema nativo de automatización intrínsecamente confiable. El slicing de red crea múltiples redes virtuales para diversos requisitos de servicio y es una tecnología crucial para los sistemas de comunicación futuros y las redes 6G. Esta tecnología facilitará numerosos aspectos de la vida, la sociedad y la industria, satisfaciendo las necesidades de comunicación tanto de humanos como de máquinas inteligentes. Esto se hace posible mediante el uso de tecnologías de softwarización y virtualización, incluyendo redes definidas por software (SDN) y virtualización de funciones de red (NFV), que dotan la flexibilidad y la programabilidad necesarias para el slicing de red. Sin embargo, la implementación de slicing de red en las redes de acceso por radio (RAN) es todo un desafío debido a la complejidad de gestionar las operaciones en el dominio radio. Esta brecha se aborda con el slicing de red zero-touch, un esquema de gestión y orquestación totalmente automatizado, que elimina la necesidad de contratos fijos e intervención manual. El motor de decisiones, un componente principal de esta tecnología, emplea la innovación algorítmica para optimizar la asignación de recursos de la red y manejar los desafíos del slicing de RAN, incluyendo la eficiencia energética, la latencia, la escalabilidad y la confiabilidad. En la asignación de recursos, se utiliza un enfoque de optimización conjunta para equilibrar la energía y la latencia con la calidad del servicio. En concreto, se propone un enfoque Actor-Crítico estocástico para agilizar el procedimiento de aprendizaje y reducir la necesidad de ajustar hiperparámetros en una configuración de slicing de red consciente del consumo energético. Además, se introduce un marco de actor-aprendiz basado en un aprendizaje profundo masivo para abordar los problemas de complejidad y los desafíos de control en el slicing de red. A su vez, las variaciones temporales de la demanda de tráfico representan un desafío significativo para la planificación y asignación de recursos en el dominio de RAN. Para mitigar esto, se propone una arquitectura distribuida para la orquestación de recursos de slices de RAN, que cuenta con múltiples agentes de decisión habilitados por IA que materializan decisiones locales de asignación de radio sin una entidad de control centralizada. Se diseña un esquema de aprendizaje federado al respecto, alineado con el reciente desarrollo de la arquitectura Open RAN, para mejorar las capacidades de toma de decisiones locales. A pesar de la efectividad de la IA, persisten las preocupaciones sobre la falta de transparencia en las redes neuronales profundas, lo que supone riesgos para la fiabilidad y la seguridad en los escenarios de red. Esta falta de confianza impide que los operadores de telecomunicaciones desplieguen ampliamente modelos de IA en sus redes. Para abordar esto, propusimos la arquitectura SliceOps que consolida las operaciones de ML explicables en un slice independiente, ofreciendo servicios de IA a otros slices. Esta configuración mejora la fiabilidad y la interpretabilidad de los modelos de IA, facilitando una implementación rápida en la red con mayor consistencia. Al pasar de ser nativo de IA a nativo de automatización, este marco gestiona tanto las funciones de servicio como las subyacentes de IA, permitiendo el aprovechamiento total del slicing en RAN y reduciendo las complejidades.

show abstract

A Collaborative Statistical Actor-Critic Learning Approach for 6G Network Slicing Control

Cited by 10 publications

References 19 publications

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration

Lifelong AI-driven zero-touch network slicing

Contact Info

Product

Resources

About