MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

Abstract

In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distributed SDN controller architectures, most existing works only aim at eliminating anomalies arising from the inconsistencies in different controllers' network views. However, the performance aspect of controller synchronization designs with respect to given SDN applications are generally missing. To fill this gap, we formulate the controller synchronization problem as a Markov decision process (MDP) and apply reinforcement learning techniques combined with deep neural networks (DNNs) to train a smart, scalable, and fine-grained controller synchronization policy, called the Multi-Armed Cooperative Synchronization (MACS), whose goal is to maximise the performance enhancements brought by controller synchronizations. Evaluation results confirm the DNN's exceptional ability in abstracting latent patterns in the distributed SDN environment, rendering significant superiority to MACS-based synchronization policy, which are 56% and 30% performance improvements over ONOS and greedy SDN controller synchronization heuristics.

Intelligent and resizable control plane for software defined vehicular network: a deep reinforcement learning approach

“…They prove that it outperforms anti-entropy algorithm. They extended their work by proposing a policy design for Controller Synchronization based on DRL too [34]. They proved the efficiency of their DRL policy in maximizing the performance enhancements brought by controller synchronizations in terms of delay over a period of time.…”

Optimizing the Lifetime of Software Defined Wireless Sensor Network via Reinforcement Learning

“…An RL-based algorithm uses the deep neural network (DNN) to represent its value function, called the Deep-Q (DQ) Scheduler, which provides nearly twofold performance improvement compared to the state-ofthe-art SDN controller synchronization solutions. However, some authors [29] use the RL for autonomous cyber defense in SDN and also use RL to resolve the synchronization issues of multiple controllers [30], [31]. Several AI techniques used in the SDN context have been introduced in [32], including different security and placement issues.…”

Reinforcement and transfer learning for distributed analytics in fragmented software defined coalitions

“…The DAIS ITA team has addressed these challenges. For example, the fundamental understanding of controller synchronization has been studied in [1], while techniques for controller synchronization has been developed in [2], [3] and [4]. A hybrid control architecture for SDN and ad-hoc networks to combine the advantages of central and distributed control mechanisms is proposed in [5] and efficient techniques for sharing of coalition resources across domains in SDC are investigated in [6] and [7].…”

“…It is important to ensure the dynamic configuration and re-configuration of resources and services can be carried out efficiently in presence of possible domain fragmentation and re-joining. Since RL techniques (e.g., [3,4]) have been commonly used to control infrastructures, it is desirable to understand and improve operations of such learning techniques when the SDC can change suddenly from being connected to fragmented, and vice versa. In particular, as SDC fragmentation and re-joining of domains represent sudden changes of operating environment, the learning-based control algorithms in use are expected to quickly adapt to such rapid changes while ensuring satisfactory performance and robustness.…”