Q-Placement: Reinforcement-Learning-Based Service Placement in Software-Defined Networks

Zhang, Ziyao; Ma, Liang; Leung, Kin K.; Tucker, Jeremy

doi:10.1109/icdcs.2018.00159

Cited by 29 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) Reinforcement Learning in SDN: Some recent highprofile successes [25], [45] attract enormous interests in using RL techniques to solve complicated decision-making problems. In the context of SDN, the authors in [46] apply RL-based algorithms to solve service placement problem on SDN switches. A routing-focused controller synchronization scheme is developed using DRL-based approaches in [47], where the MDP the authors formulated is easier to solve, as they assume an uniform synchronization budget and coarsegrained synchronization decisions where the decisions are at the level of controller pairs.…”

Section: Evaluation Resultsmentioning

confidence: 99%

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

Zhang

Poularakis

et al. 2019

2019 IEEE 27th International Conference on Network Protocols (ICNP)

Self Cite

View full text Add to dashboard Cite

In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distributed SDN controller architectures, most existing works only aim at eliminating anomalies arising from the inconsistencies in different controllers' network views. However, the performance aspect of controller synchronization designs with respect to given SDN applications are generally missing. To fill this gap, we formulate the controller synchronization problem as a Markov decision process (MDP) and apply reinforcement learning techniques combined with deep neural networks (DNNs) to train a smart, scalable, and fine-grained controller synchronization policy, called the Multi-Armed Cooperative Synchronization (MACS), whose goal is to maximise the performance enhancements brought by controller synchronizations. Evaluation results confirm the DNN's exceptional ability in abstracting latent patterns in the distributed SDN environment, rendering significant superiority to MACS-based synchronization policy, which are 56% and 30% performance improvements over ONOS and greedy SDN controller synchronization heuristics.

show abstract

Section: Evaluation Resultsmentioning

confidence: 99%

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

Zhang

Poularakis

et al. 2019

2019 IEEE 27th International Conference on Network Protocols (ICNP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since a controller has the global view of the network, it can observe the reward and next state for current state and current action. This advantage can make the proposed Q-learning with guarantees both performance and convergence rates [16]. As a result, SDN Fog controller can find optimal actions which help nodes to optimally select a neighboring node to which it can offload its requested tasks.…”

Section: A Contributionsmentioning

confidence: 99%

“…Towards this end, the pro-posed model merges with SDN-based Fog computing where SDN Fog controller directly controls, programs, orchestrates and manages network resources. Further, SDN Fog nodes serve the end users' request and deliver information based on collected traffic information to the controller [16]. Besides, the proposed reward function is defined with the aim to minimize the processing time and the overall overloading probability.…”

Section: A Contributionsmentioning

confidence: 99%

“…• The proposed Q-learning based offloading decision can expect how good the present offloading is over the future, which achieves the exceptional overall system performance. • The decision-making process using reinforcement learning is on-demand and pay-as-you-go, which makes the proposed method compatible with SDN architecture [16]. Hence, it can be regarded as a load balancing application running on an SDN Fog controller.…”

Section: A Contributionsmentioning

confidence: 99%

“…The existing schemes proposed in the literature have majorly concentrated on load balancing and cooperative offloading in Fog environment [11]- [15]. However, most of these approaches impose many restrictions and specific assumptions over the networks; which often do not relate with realistic Fog networks [16]. Additionally, these approaches require nontrivial mathematical equations when more and more restrictions are considered.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Managing Fog Networks using Reinforcement Learning Based Load Balancing Algorithm

Baek

Kaddoum

Garg

et al. 2019

2019 IEEE Wireless Communications and Networking Conference (WCNC)

View full text Add to dashboard Cite

The powerful paradigm of Fog computing is currently receiving major interest, as it provides the possibility to integrate virtualized servers into networks and brings cloud service closer to end devices. To support this distributed intelligent platform, Software-Defined Network (SDN) has emerged as a viable network technology in the Fog computing environment. However, uncertainties related to task demands and the different computing capacities of Fog nodes, inquire an effective load balancing algorithm. In this paper, the load balancing problem has been addressed under the constraint of achieving the minimum latency in Fog networks. To handle this problem, a reinforcement learning based decision-making process has been proposed to find the optimal offloading decision with unknown reward and transition functions. The proposed process allows Fog nodes to offload an optimal number of tasks among incoming tasks by selecting an available neighboring Fog node under their respective resource capabilities with the aim to minimize the processing time and the overall overloading probability. Compared with the traditional approaches, the proposed scheme not only simplifies the algorithmic framework without imposing any specific assumption on the network model but also guarantees convergence in polynomial time. The results show that, during average delays, the proposed reinforcement learning-based offloading method achieves significant performance improvements over the variation of service rate and traffic arrival rate. The proposed algorithm achieves 1.17%, 1.02%, and 3.21% lower overload probability relative to random, least-queue and nearest offloading selection schemes, respectively.

show abstract

Energy aware cloud‐edge service placement approaches in the Internet of Things communications

Heng

Yin

Zhao

2021

Int J Communication

View full text Add to dashboard Cite

Summary In recent years, applying Internet of Things (IoT) applications has significantly increased to facilitate and improve quality of human life activities in various fields such as healthcare, education, industry, economics, etc. The energy aware cloud‐edge computing paradigm has developed as a hybrid computing solution to provide IoT applications using available cloud service providers and fog nodes for the smart devices and mobile applications. Since the IoT applications are developed in the form of several IoT services with various quality of service (QoS) metrics which can deploy on the cloud‐edge providers with different resource capabilities, finding an efficient placement solution as one of challenging topics to be measured for IoT applications. The service placement issue arranges IoT applications on the cloud‐edge providers with various capabilities of atomic services though sufficient different QoS factors to support service level agreement (SLA) contracts. This paper presents a technical analysis on the cloud‐edge service placement approaches in IoT systems. The key point of this technical analysis is to identify substantial studies in the service placement approaches which need additional consideration to progress more efficient and effective placement strategies in IoT environments. In addition, a side‐by‐side taxonomy is proposed to categorize the relevant studies on cloud‐edge service placement approaches and algorithms. A statistical and technical analysis of reviewed existing approaches is provided, and evaluation factors and attributes are discussed. Finally, open issues and forthcoming challenges of service placement approaches are presented.

show abstract

Q-Placement: Reinforcement-Learning-Based Service Placement in Software-Defined Networks

Cited by 29 publications

References 15 publications

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

Managing Fog Networks using Reinforcement Learning Based Load Balancing Algorithm

Energy aware cloud‐edge service placement approaches in the Internet of Things communications

Contact Info

Product

Resources

About