Load balancing non-uniform parallel computations

Abstract

Dynamic load balancing is critical in achieving high performance in parallel systems, especially for applications with unpredictable workloads. Traditional load balancing approaches such as work-sharing and work-stealing often assume that the computations can be divided into smaller computations based on some granularity. These approaches do not scale when the computations are not dividable and their sizes are highly variant. In this paper, we present a novel approach -founded in the Actor model known for its programmability -for dynamically load-balancing nonuniform parallel computations. Specifically, our approach is to explicitly reason about resource requirements and commitments at run-time in order to make fine-grained scheduling decisions. We prototype our approach by extending an efficient Java implementation of Actors, ActorFoundry. We have also implemented a tuner which dynamically balances the resources used by the computations vs. those used by the reasoning mechanism, essentially balancing the quality of scheduling against the resources needed for achieving it. We propose a new benchmark -the Unbalanced Cobwebbed Tree (UCT) -to fairly compare our approach to existing approaches, which captures the non-uniform nature of computations. Experimental results show that despite the higher overhead of providing Actors' programmability features, for a diverse enough set of computations, our approach outperforms both work-sharing and work-stealing approaches, and shows better scalability.

Savina - An Actor Benchmark Suite

“…The Unbalanced Cobwebbed Tree (UCT) [54] benchmark tries to capture the non-uniform nature of tree exploration computations. It extends the Unbalanced Tree Search benchmark by introducing substantial computations at each tree node, and allowing the computations to be non-uniform.…”

Energy-aware resource allocation for multicores with per-core frequency scaling

“…In the most recent work [22], this approach have been repurposed to achieve dynamic load balancing for computations which do not constrained by deadlines. Fundamental to this work is a fine grained accounting of available resources, as well as the resources required by computations.…”