Using Network Knowledge to Improve Workload Performance in Virtualized Data Centers

Erickson, David; Heller, Brandon; McKeown, Nick; Rosenblum, Mendel

doi:10.1109/ic2e.2014.81

Cited by 8 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To achieve energyefficient workload placement for DCs, authors in [14] constructed a rack-level power model that mapped the workload directly to its power dissipation and pursued an optimal workload allocation with minimized power consumption. Authors in [15] presented a virtual data center resource manager that exploited the network and topology knowledge, since they assumed traffic and topology were informed before so that the overall performance would be improved drastically. HUG [16] proposed a coflow scheduling algorithm to achieve multiresource fairness and maximize network utilization without sacrificing strategy-proofness.…”

Section: Related Workmentioning

confidence: 99%

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Wei

Wang

et al. 2023

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

With the development of edge-cloud computing technologies, distributed data centers (DCs) have been extensively deployed across the global Internet. Since different users/applications have heterogeneous requirements on specific types of ICT resources in distributed DCs, how to optimize such heterogeneous resources under dynamic and even uncertain environments becomes a challenging issue. Traditional approaches are not able to provide effective solutions for multi-dimensional resource allocation that involves the balanced utilization across different resource types in distributed DC environments. This paper presents a reinforcement learning based approach for multidimensional resource allocation (termed as NESRL-MRM) that is able to achieve balanced utilization and availability of resources in dynamic environments. To train NESRL-MRM's agent with sufficiently quick wall-clock time but without the loss of exploration diversity in the search space, a natural evolution strategy (NES) is employed to approximate the gradient of the reward function. To realistically evaluate the performance of NESRL-MRM, our simulation evaluations are based on real-world workload traces from Amazon EC2 and Google datacenters. Our results show that NESRL-MRM is able to achieve significant improvement over the existing approaches in balancing the utilization of multi-dimensional DC resources, which leads to substantially reduced blocking probability of future incoming workload demands.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Wei

Wang

et al. 2023

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

show abstract

“…A primary mechanism to reduce the amount of communication and improve the efficiency of network usage is to optimize the placement of an application's VMs to co-locate those with heavy flows between them on the same racks [3], [4]. For such an approach to work, a distributed load balancer such as Airfoil that can actually recognize the co-located VMs and preferentially route traffic to them is a prerequisite.…”

Section: Vm Placement Optimizationmentioning

confidence: 99%

“…Methods to optimize the placement of virtual machines and to increase communication locality have also been proposed as a means for decreasing the utilization of bottleneck links [3], [4]. A VM migration strategy is introduced in [18] that tries to avoid resource contention, including the network, using utilization information.…”

Section: F Coordinator Performancementioning

confidence: 99%

Airfoil: A Topology Aware Distributed Load Balancing Service

Matsuba

Joshi

Hiltunen

et al. 2015

2015 IEEE 8th International Conference on Cloud Computing

View full text Add to dashboard Cite

Load balancing is one of the most basic services needed by cloud applications. While today's clouds and load balancers provide highly customizable load distribution policies, they are forced out of necessity to ignore the impact of load balancing on the network. However, such network agnostic behavior can lead to inefficient utilization of cloud resources and poor performance, especially for upcoming network centric applications such as network function virtualization (NFV). But can network topology, a cloud property that is hidden from tenants, be made to effectively influence load balancing, a function that is intimately tied to per-tenant application structure that is largely invisible to the cloud? We answer that question in this paper by presenting Airfoil, a novel topology-aware distributed load balancer as a service (LBaaS) that takes network topology into consideration while providing cloud tenants with application-specific load balancing policies. By finding the best communication pattern with linear programming, the most efficient load balancing strategy is calculated without using any heuristics or approximations. We evaluate Airfoil over a range of scenarios from classical web applications to NFV pipelines, and show that it can decrease the network utilization of the most highly loaded network links by over 50%. For network constrained applications, such reductions imply a doubling of application capacity without any additional infrastructure.

show abstract

“…As part of other research [10,11], Beacon has run the network for the last two and a half years of an 80 server, 320 virtual machine cluster containing 80 virtual switches (one per server), and 20 physical switches wired as a k=4 fat tree. Beacon's modularity enabled a custom routing engine to extend the default, and the creation of a custom Web UI for tracking experiments.…”

Section: Deployment Experiencementioning

confidence: 99%

The beacon openflow controller

Erickson

2013

Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking

Self Cite

383

217

View full text Add to dashboard Cite

Beacon is a Java-based open source OpenFlow controller created in 2010. It has been widely used for teaching, research, and as the basis of Floodlight. This paper describes the architectural decisions and implementation that achieves three of Beacon's goals: to improve developer productivity, to provide the runtime ability to start and stop existing and new applications, and to be high performance.

show abstract

Using Network Knowledge to Improve Workload Performance in Virtualized Data Centers

Cited by 8 publications

References 26 publications

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Airfoil: A Topology Aware Distributed Load Balancing Service

The beacon openflow controller

Contact Info

Product

Resources

About