Spotlight: Scalable Transport Layer Load Balancing for Data Center Networks

Abstract

Balancing plays a vital role in cloud data centers to distribute traffic among instances of network functions or services. State-of-the-art load balancers dispatch traffic obliviously without considering the real-time utilization of service instances and therefore can lead to uneven load distribution and sub-optimal performance.In this paper, we design and implement Spotlight, a scalable and distributed load balancing architecture that maintains connection-to-instance mapping consistency at the edge of data center networks. Spotlight uses a new stateful flow dispatcher which periodically polls instances' load and dispatches incoming connections to instances in proportion to their available capacity. Our design utilizes a distributed control plane and in-band flow dispatching; thus, it scales horizontally in data center networks. Through extensive flow-level simulation and packet-level experiments on a testbed with HTTP traffic on unmodified Linux kernel, we demonstrate that compared to existing methods Spotlight distributes traffic more efficiently and has near-optimum performance in terms of overall service utilization. Compared to existing solutions, Spotlight improves aggregated throughput and average flow completion time by at least 20% with infrequent control plane updates. Moreover, we show that Spotlight scales horizontally as it updates the switches at O(100ms) and is resilient to lack of control plane convergence.

Aquarius—Enable Fast, Scalable, Data-Driven Service Management in the Cloud

“…First, this paper identifies the challenges of gathering networking features to make valuable inference and informed operational decisions in high-performance and large-scale cloud DCs: Experimental evaluations demonstrate that traditional mechanisms for feature collection (e.g., active probing [12], [17], [32]- [36] and trace capture [37]- [42]) causes substantial overhead. Using a real-world testbed, it is shown that networking features gathered by Aquarius in real time can provide valuable information for system states inference, with no additional control message and limited resource consumption.…”

“…Data-driven mechanisms based on machine learning (ML) [7], [8] and reinforcement learning (RL) algorithms [9], [10] are applied and show performance gains in various network applications. For instance, autoscaling systems and load balancers can achieve improved QoS with reduced cost based on periodically polled resource utilisation of distributed network devices (e.g., application servers) [11], [12]. Traffic classification and anomaly detection help detect security threats with increased accuracy based on network traffic characteristics extracted from offline-collected network traces [13], [14].…”

“…ML and RL algorithms require fine-grained observations of network and system states [15]: Conventionally, periodically polling resource utilisation and system performance allows for obtaining timely and dedicated observations to make data-driven management decisions [11], [12], [16]- [19]. However, the active polling scheme incurs additional control messages and reduces system scalability, especially for large-scale distributed systems.…”

Charon: Load-Aware Load-Balancing in P4

“…To guarantee PCC, stateful LBs keep tracking the state of the connections [1], [6]- [8]. Using advanced hashing mechanism (e.g.…”

“…Segment Routing (SR) [14] and powerof-2-choice [15] are used in [7], [9] to daisy chaine 2 servers and let them decide, based on their actual load states, whether or not the new flow should be accepted. Another approach is to periodically poll servers' instant "available capacities" [8]. Ridge Regression is used in [16] to predict server load states and compute the relative "weight" of each server for Weighted Costed Multi-Path (WCMP).…”