The tao of parallelism in algorithms

Pingali, Keshav; Méndez-Lojo, Mario; Prountzos, Dimitrios; Sui, Xin; Nguyen, Donald; Kulkarni, Milind; Burtscher, Martin; Hassaan, M. Amber; Kaleem, Rashid; Lee, Tsung-Hsien; Lenharth, Andrew; Manevich, Roman

doi:10.1145/2345156.1993501

Cited by 7 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many parallel programming languages based on procedural, object-oriented, and functional programming languages have been developed. Systems extending C/C++ include Cilk/Cilk++ [Blumofe et al 1995;Frigo et al 2009;Intel Corporation 2009a], Intel TBB [Intel Corporation 2009b], and Galois [Kulkarni et al 2007;Pingali et al 2011]. The Rust language offers a type-safe option for systems-level programming [Rust Team 2019]; the type system of Rust is powerful enough to outlaw races statically [Jung et al 2018a], though it is difficult (if not impossible) to implement efficient parallel algorithms, such as the algorithms that we consider in our evaluation, by using safe primitives only.…”

Section: Related Workmentioning

confidence: 99%

Provably space-efficient parallel functional programming

Arora

Westrick

Acar

2021

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Because of its many desirable properties, such as its ability to control effects and thus potentially disastrous race conditions, functional programming offers a viable approach to programming modern multicore computers. Over the past decade several parallel functional languages, typically based on dialects of ML and Haskell, have been developed. These languages, however, have traditionally underperformed procedural languages (such as C and Java). The primary reason for this is their hunger for memory, which only grows with parallelism, causing traditional memory management techniques to buckle under increased demand for memory. Recent work opened a new angle of attack on this problem by identifying a memory property of determinacy-race-free parallel programs, called disentanglement, which limits the knowledge of concurrent computations about each other’s memory allocations. The work has showed some promise in delivering good time scalability. In this paper, we present provably space-efficient automatic memory management techniques for determinacy-race-free functional parallel programs, allowing both pure and imperative programs where memory may be destructively updated. We prove that for a program with sequential live memory of R * , any P -processor garbage-collected parallel run requires at most O ( R * · P ) memory. We also prove a work bound of O ( W + R * P ) for P -processor executions, accounting also for the cost of garbage collection. To achieve these results, we integrate thread scheduling with memory management. The idea is to coordinate memory allocation and garbage collection with thread scheduling decisions so that each processor can allocate memory without synchronization and independently collect a portion of memory by consulting a collection policy, which we formulate. The collection policy is fully distributed and does not require communicating with other processors. We show that the approach is practical by implementing it as an extension to the MPL compiler for Parallel ML. Our experimental results confirm our theoretical bounds and show that the techniques perform and scale well.

show abstract

Section: Related Workmentioning

confidence: 99%

Provably space-efficient parallel functional programming

Arora

Westrick

Acar

2021

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…We have implemented our graph-grammar-based system in GALOIS [8,30] framework, allowing for concurrent graph processing. The code is compiled on node13 on the Atari Linux cluster from the Adaptive Algorithms and Systems (a2s.agh.edu.pl) research group from the Department of Computer Science, AGH University.…”

Section: Computations Of the Pollution Propagationmentioning

confidence: 99%

“…This paper utilizes the hypergraphs for modeling mesh refinement algorithm and interfacing with the GMRES iterative solver. We have implemented our graph-grammar-based system in GALOIS [2,8,14,24,30] framework, allowing for concurrent graph processing.…”

Section: Introductionmentioning

confidence: 99%

“…The graph grammar productions are basic undividable tasks that can be executed concurrently. The graph-grammar model allows for implementation in the graph processing system like, e.g., GALOIS environment [30]. The parallelization is obtained for free since the GALOIS system automatically manages concurrent processing of graph-grammar productions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel graph-grammar-based algorithm for the longest-edge refinement of triangular meshes and the pollution simulations in Lesser Poland area

Podsiadło

Oliver

Paszyńska

et al. 2021

Engineering with Computers

Self Cite

View full text Add to dashboard Cite

In this paper, we propose parallel graph-grammar-based algorithm for the longest-edge refinements and the pollution simulations in Lesser Poland area. We introduce graph-grammar productions for Rivara’s longest-edged algorithm for the local refinement of unstructured triangular meshes. We utilize the hyper-graph to represent the computational mesh and the graph-grammar productions to express the longest-edge mesh refinement algorithm. The parallelism in the original Rivara’s longest edge refinement algorithm is obtained by processing different longest edge refinement paths in different three ads. Our graph-grammar-based algorithm allows for additional parallelization within a single longest-edge refinement path. The graph-grammar-based algorithm automatically guarantees the validity and conformity of the generated mesh; it prevents the generation of duplicated nodes and edges, elongated elements with Jacobians converging to zero, and removes all the hanging nodes automatically from the mesh. We test the algorithm on generating a surface mesh based on a topographic data of Lesser Poland area. The graph-grammar productions also generate the layers of prismatic three-dimensional elements on top of the triangular mesh, and they break each prismatic element into three tetrahedral elements. Next, we propose graph-grammar productions generating element matrices and right-hand-side vectors for each tetrahedral element. We utilize the Streamline Upwind Petrov–Galerkin (SUPG) stabilization for the pollution propagation simulations in Lesser Poland area. We use the advection–diffusion-reaction model, the Crank–Nicolson time integration scheme, and the graph-grammar-based interface to the GMRES solver.

show abstract

“…The benchmark inputs for DMR are used from the Galois project by Pingali et al 24 Each experiment is repeated 5 times, and a graph is plotted with the arithmetic mean of the observed execution times. The execution times are in logarithmic scale to address skewness in values of observed data.…”

Section: Comparative Evaluation Of Par-deadlockfree With Existing Workmentioning

confidence: 99%

A deadlock‐free lock‐based synchronization for GPUs

Anand

Srivastava

Shyamasundar

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Graphics Processing Units (GPUs) have evolved from pure graphics applications toward general purpose applications, often referred to as GPGPU computing. However, its scope is still limited to data‐parallel applications that require little synchronization. As synchronization on GPUs is quite costly, synchronization requirements in GPUs are usually realized using existing synchronization primitives like atomic operations and barriers. These approaches either incur significant overhead or place certain restrictions in their usage, affecting the scalability/scope of such applications. The lack of adequate support for fine‐grained synchronization has restricted the realization of irregular algorithms on GPUs, wherein control flow and memory access patterns are data‐dependent and unpredictable. Recently, there has been an interest in building relationship between lock‐step semantics and interleaving semantics and to develop lock‐based synchronization mechanism for GPUs to overcome these issues. GPUs follow SIMD, and hence, when adapted for general purpose computing, new distinct deadlock scenarios arise. In this paper, we discuss various deadlock scenarios that can happen in GPUs, and present a modeling of deadlocks in GPUs. We shall first illustrate such deadlock scenarios in GPU applications, and then describe a novel lock‐based deadlock‐free, fine‐grained synchronization mechanism for GPU architectures that overcomes deadlocks without a significant overhead. We further establish the correctness of our methods and discuss the performance overheads.

show abstract

The tao of parallelism in algorithms

Cited by 7 publications

References 5 publications

Provably space-efficient parallel functional programming

Provably space-efficient parallel functional programming

Parallel graph-grammar-based algorithm for the longest-edge refinement of triangular meshes and the pollution simulations in Lesser Poland area

A deadlock‐free lock‐based synchronization for GPUs

Contact Info

Product

Resources

About