Optimal Parallel Algorithms in the Binary-Forking Model

Abstract

In this paper we develop optimal algorithms in the binary-forking model for a variety of fundamental problems, including sorting, semisorting, list ranking, tree contraction, range minima, and ordered set union, intersection and difference. In the binary-forking model, tasks can only fork into two child tasks, but can do so recursively and asynchronously. The tasks share memory, supporting reads, writes and test-and-sets. Costs are measured in terms of work (total number of instructions), and span (longest dependence chain).The binary-forking model is meant to capture both algorithm performance and algorithm-design considerations on many existing multithreaded languages, which are also asynchronous and rely on binary forks either explicitly or under the covers. In contrast to the widely studied PRAM model, it does not assume arbitrary-way forks nor synchronous operations, both of which are hard to implement in modern hardware. While optimal PRAM algorithms are known for the problems studied herein, it turns out that arbitrary-way forking and strict synchronization are powerful, if unrealistic, capabilities. Natural simulations of these PRAM algorithms in the binary-forking model (i.e., implementations in existing parallel languages) incur an Ω(log n) overhead in span. This paper explores techniques for designing optimal algorithms when limited to binary forking and assuming asynchrony. All algorithms described in this paper are the first algorithms with optimal work and span in the binary-forking model. Most of the algorithms are simple. Many are randomized.

Parallel Minimum Cuts in O ( m log 2 n ) Work and Low Depth

“…They describe a random-mate technique that ensures that any tree contracts to a single vertex in 𝑂 (log 𝑛) rounds w.h.p., and using a total of 𝑂 (𝑛) work in expectation. Gazit, Miller, and Teng [13] give a deterministic version with the same bounds, and Blelloch et al [7] give a version that works in the binary-forking model. Miller and Reif's algorithm applies to bounded-degree trees, but arbitrary-degree trees can typically be handled by converting them into bounded-degree trees.…”

“…Work is defined as the total number of instructions performed by the algorithm and depth (also called span) is the length of the longest chain of sequentially dependent instructions [6]. The model can work-efficiently cross simulate the classic CRCW PRAM model [6], and the more recent Binary Forking model [7] with at most a logarithmic-factor difference in the depth. Randomness.…”

“…The preprocessing just builds an RC tree on the source tree, and sets the values for each cluster based on the initial values on the leaves. This can be implemented with the Miller-Reif algorithm [29], in the binary forking model [7], or deterministically [13]. All take linear work and logarithmic depth (w.h.p for the randomized versions).…”

Efficient Parallel Self-Adjusting Computation

“…We analyze algorithms in the work-span model, where work is the total number of instructions performed by the algorithm and span (also called depth) is the length of the longest chain of sequentially dependent instructions [16]. The model can work-efficiently cross simulate the classic CRCW PRAM model [16], and the more recent Binary Forking model [18], incurring at most an additional (log * ( )) factor overhead in the depth due to load balancing. An algorithm with work and span can be ran on a -processor PRAM in ( / + ) time [19].…”

Sage

“…The PSAM has a set of threads that share both the large-memory and small-memory. The underlying mechanisms for parallelism are identical to the T-RAM or binary forking model, which is discussed in detail in [13,17,37]. In the model, each thread acts like a sequential RAM that also has a fork instruction.…”