Aquarius—Enable Fast, Scalable, Data-Driven Service Management in the Cloud

Abstract

In order to dynamically manage and update networking policies in cloud data centers, Virtual Network Functions (VNFs) use, and therefore actively collect, networking state information -and in the process, incur additional control signaling and management overhead, especially in larger data centers. In the meantime, VNFs in production prefer distributed and straightforward heuristics over advanced learning algorithms to avoid intractable additional processing latency under highperformance and low-latency networking constraints. This paper identifies the challenges of deploying learning algorithms in the context of cloud data centers, and proposes Aquarius to bridge the application of machine learning (ML) techniques on distributed systems and service management. Aquarius passively yet efficiently gathers reliable observations, and enables the use of ML techniques to collect, infer, and supply accurate networking state information -without incurring additional signaling and management overhead. It offers fine-grained and programmable visibility to distributed VNFs, and enables both open-and closeloop control over networking systems. This paper illustrates the use of Aquarius with a traffic classifier, an auto-scaling system, and a load balancer -and demonstrates the use of three different ML paradigms -unsupervised, supervised, and reinforcement learning, within Aquarius, for network state inference and service management. Testbed evaluations show that Aquarius suitably improves network state visibility and brings notable performance gains for various scenarios with low overhead.

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

“…Each LB agent contains a replay buffer and can learn using 1 of the 3 different RL algorithms-independent SAC (I-SAC), QMix, or single-agent SAC (S-SAC). The 3 RL algorithms consume the network features (on-going flows and flow duration statistics on each server collected based on the framework proposed in [28]) as well as the actions from last time step, and they generate server load state estimations as the next time-step action for making fair per-flow-level decision based on the shortest expected delay algorithm. initial observed instant queue length on server k by the i-th LB: for LB agent i do 13:…”