Optimistic parallelism requires abstractions

Abstract

Irregular applications, which manipulate large, pointer-based data structures like graphs, are difficult to parallelize manually. Automatic tools and techniques such as restructuring compilers and runtime speculative execution have failed to uncover much parallelism in these applications, in spite of a lot of effort by the research community. These difficulties have even led some researchers to wonder if there is any coarse-grain parallelism worth exploiting in irregular applications.In this paper, we describe two real-world irregular applications: a Delaunay mesh refinement application and a graphics application that performs agglomerative clustering. By studying the algorithms and data structures used in these applications, we show that there is substantial coarse-grain, data parallelism in these applications, but that this parallelism is very dependent on the input data and therefore cannot be uncovered by compiler analysis. In principle, optimistic techniques such as thread-level speculation can be used to uncover this parallelism, but we argue that current implementations cannot accomplish this because they do not use the proper abstractions for the data structures in these programs.These insights have informed our design of the Galois system, an object-based optimistic parallelization system for irregular applications. There are three main aspects to Galois: (1) a small number of syntactic constructs for packaging optimistic parallelism as iteration over ordered and unordered sets, (2) assertions about methods in class libraries, and (3) a runtime scheme for detecting and recovering from potentially unsafe accesses to shared memory made by an optimistic computation.We show that Delaunay mesh generation and agglomerative clustering can be parallelized in a straight-forward way using the Galois approach, and we present experimental measurements to show that this approach is practical. These results suggest that Galois is a practical approach to exploiting data parallelism in irregular programs.

A GPU implementation of inclusion-based points-to analysis

“…The sequential implementation is written in C++ and compiled with gcc and the -O3 flag. The reference implementation is written in Java on top of the Galois framework [21]. The Java Virtual Machine used is the 64-bit Sun HotSpot server version 1.6.0 24.…”

Semantics of Concurrent Revisions

“…Isolation types are also similar to the idea of transactional boosting, coarse-grained transactions, and semantic commutativity [16,19,20], which eliminate false conflicts by raising the abstraction level. Isolation types go farther though: for example, the type versioned T does not just avoid false conflicts, but resolves true conflicts deterministically (in a not necessarily serializable way).…”

Just-In-Time Locality and Percolation for Optimizing Irregular Applications on a Manycore Architecture

“…The current threading library (e.g., Pthreads), a combination of compiler directives and libraries (e.g., OpenMP) and optimistic parallelization [1][2][3] were not designed to support programming for tolerating off-chip latency, or to handle efficient allocation and movement of data across hierarchy levels. It is often the case that in the underline thread execution model, a thread is enabled and activated as soon as all data and control dependencies are satisfied.…”