Online Balanced Repartitioning

Abstract

This paper initiates the study of a fundamental online problem called online balanced repartitioning. Unlike the classic graph partitioning problem, our input is an arbitrary sequence of communication requests between nodes, with patterns that may change over time. The objective is to dynamically repartition the n nodes into clusters, each of size k. Every communication request needs to be served either locally (cost 0), if the communicating nodes are collocated in the same cluster, or remotely (cost 1), using intercluster communication, if they are located in different clusters. The algorithm can also dynamically update the partitioning by migrating nodes between clusters at cost α per node migration. Therefore, we are interested in online algorithms which find a good tradeoff between the communication cost and the migration cost, maintaining partitions which minimize the number of inter-cluster communications.We consider settings both with and without cluster-size augmentation. For the former, we prove a lower bound which is strictly larger than k, which highlights an interesting difference to online paging. Somewhat surprisingly, and unlike online paging, we prove that any deterministic online algorithm has a non-constant competitive ratio of at least k, even with augmentation. Our main technical contribution is an O(k log k)-competitive algorithm for the setting with (constant) augmentation.We believe that our model finds interesting applications, e.g., in the context of datacenters, where virtual machines need to be dynamically embedded on a set of (multi-core) servers, and where machines migrations are possible, but costly.

Efficient Distributed Workload (Re-)Embedding

“…The paper most closely related to ours is by Avin et al [8,7] who studied a more general version of the problem considered in our paper. In their model, request patterns can change arbitrarily over time, and in particular, do not have to follow a partition and hence "cannot be learned".…”

“…The lower bound has the following two main consequences: (1) If an algorithm is only allowed to use constant augmentation (i.e., servers of capacity n/ + O(1)), then the lower bound implies that any algorithm must have a competitive ratio of Ω(n). 8 (2) The lower bound holds even in the setting in which there are only two servers. Thus, the algorithm from Section 3 for the two server setting is close to optimal (up to a O(min{1/ε, log n}) factor) and the generalized algorithm from Section 4 is optimal up to a O( log min{1/ε, log n}) factor.…”

“…After each request, the algorithm can reconfigure the infrastructure and move communication endpoints from one server to another, essentially repartitioning the communication partners; however, each move incurs a cost of α > 1.In other words, this paper considers the problem of learning a partition, i.e., an optimal assignment of communication partners to servers, at low communication and moving cost. Interestingly, while the problem is natural and fundamental, not much is known today about the algorithmic challenges underlying this problem, except for the negative result that no good competitive algorithm can exist if communication partners can change arbitrarily over time [8]. This lower bound motivates us, in this paper, to focus on the online learning variant where the communication partners are unknown but fixed.…”

Self-adjusting Linear Networks

“…Communication networks are becoming increasingly flexible, along three main dimensions: routing (enabler: software-defined networking), embedding (enabler: virtualization), and topology (enabler: reconfigurable optical technologies, for example [15]). In particular, the possibility to quickly reconfigure communication networks, e.g., by migrating (virtualized) communication endpoints [8] or by reconfiguring the (optical) topology [11], allows these networks to become demand-aware: i.e., to adapt to the traffic pattern they serve, in an online and self-adjusting manner. For example, in a self-adjusting network, frequently communicating node pairs can be moved topologically closer, saving communication costs (e.g., bandwidth, energy) and improving performance (e.g., latency, throughput).…”

“…, y8. The values w(2, 5), w(8,4), and w(7, 7) are visualised as the area of rectangles highlighted in red, green, and blue respectively.…”