A Constrained Reinforcement Learning Based Approach for Network Slicing

Liu, Yongshuai; Ding, Jiaxin; Liu, Xin

doi:10.1109/icnp49622.2020.9259378

Cited by 42 publications

(29 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the Lagrangian Relaxation method they are using to handle the constraints is not as efficient as our IPO [19] where CLARA relies on. An earlier short version of our work appears at a workshop [40] and another short version appeared as a poster [41]. In comparison, in this paper, we expand the system architecture and description, we prove the policy improvement in Theorem 4, we show CLARA can handle multiple cumulative constraints that are more common in realworld scenarios, we demonstrate the feasibility of warm-start to speed up convergence, and we expand the evaluation to compare with more baselines.…”

Section: B Reinforcement Learning In Network Slicingmentioning

confidence: 85%

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

Liu¹,

Ding²,

Zhang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

As mobile networks proliferate, we are experiencing a strong diversification of services, which requires greater flexibility from the existing network. Network slicing is proposed as a promising solution for resource utilization in 5G and future networks to address this dire need. In network slicing, dynamic resource orchestration and network slice management are crucial for maximizing resource utilization. Unfortunately, this process is too complex for traditional approaches to be effective due to a lack of accurate models and dynamic hidden structures. We formulate the problem as a Constrained Markov Decision Process (CMDP) without knowing models and hidden structures. Additionally, we propose to solve the problem using CLARA, a Constrained reinforcement LeArning based Resource Allocation algorithm. In particular, we analyze cumulative and instantaneous constraints using adaptive interior-point policy optimization and projection layer, respectively. Evaluations show that CLARA clearly outperforms baselines in resource allocation with service demand guarantees.

show abstract

Section: B Reinforcement Learning In Network Slicingmentioning

confidence: 85%

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

Liu¹,

Ding²,

Zhang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both requirements are inefficient and infeasible for a stochastic environments such as 5G RANs. Thus, the DRL is an attractive approach to solve the previously defined slicing resource allocation problem [11].…”

Section: Reinforcement Learning-based Slicing Resource Allocationmentioning

confidence: 99%

Transfer Learning-Based Accelerated Deep Reinforcement Learning for 5G RAN Slicing

Nagib

Abou-zeid

Hassanein

2021

2021 IEEE 46th Conference on Local Computer Networks (LCN)

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) algorithms have been recently proposed to solve dynamic Radio Resource Management (RRM) problems in 5G networks. However, the slow convergence experienced by traditional DRL agents puts many doubts on their practical adoption in cellular networks. In this paper, we first discuss the need to have accelerated DRL algorithms. We then analyze the exploration behavior of various state-of-the-art DRL algorithms for slice resource allocation, and compare it with the traditional 5G Radio Access Network (RAN) slicing baselines. Finally, we propose a transfer learningaccelerated DRL-based solution for slice resource allocation. In particular, we tackle the challenge of slow convergence by transferring the policy learned by a DRL agent at an expert base station (BS) to newly deployed agents at target learner BSs. Our approach shows a remarkable reduction in convergence time and a significant performance improvement compared with its nonaccelerated counterparts when tested against multiple traffic load variations.

show abstract

“…The results obtained through simulations indicate that the proposed approach provides substantial potential gains in terms of system utilization while not violating the tenants' SLA by relying on appropriately tuned schemes. The authors in [91] proposed a constrained RLbased approach in order to address the lack of accurate resource orchestration models and hidden problem structures. The RA problem was focused on resource virtualization and the problem was formulated as a CMDP, subject to both cumulative and instantaneous constraints.…”

Section: ) Cognitive Radio Networkmentioning

confidence: 99%

AI Meets CRNs: A Prospective Review on the Application of Deep Architectures in Spectrum Management

Hlophe

Maharaj

2021

IEEE Access

View full text Add to dashboard Cite

The spectrum low utilization and high demand conundrum created a bottleneck towards fulfilling the requirements of next-generation networks. The cognitive radio (CR) technology was advocated as a de facto technology to alleviate the scarcity and under-utilization of spectrum resources by exploiting temporarily vacant spectrum holes of the licensed spectrum bands. As a result, the CR technology became the first step towards the intelligentization of mobile and wireless networks, and in order to strengthen its intelligent operation, the cognitive engine needs to be enhanced through the exploitation of artificial intelligence (AI) strategies. Since comprehensive literature reviews covering the integration and application of deep architectures in cognitive radio networks (CRNs) are still lacking, this article aims at filling the gap by presenting a detailed review that addresses the integration of deep architectures into the intricacies of spectrum management. This is a prospective review whose primary objective is to provide an indepth exploration of the recent trends in AI strategies employed in mobile and wireless communication networks. The existing reviews in this area have not considered the relevance of incorporating the mathematical fundamentals of each AI strategy and how to tailor them to specific mobile and wireless networking problems. Therefore, this review addresses that problem by detailing how deep architectures can be integrated into spectrum management problems. Beyond reviewing different ways in which deep architectures can be integrated into spectrum management, model selection strategies and how different deep architectures can be tailored into the CR space to achieve better performance in complex environments are then reported in the context of future research directions.

show abstract

A Constrained Reinforcement Learning Based Approach for Network Slicing

Cited by 42 publications

References 9 publications

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

Transfer Learning-Based Accelerated Deep Reinforcement Learning for 5G RAN Slicing

AI Meets CRNs: A Prospective Review on the Application of Deep Architectures in Spectrum Management

Contact Info

Product

Resources

About