iPUG: Accelerating Breadth-First Graph Traversals Using Manycore Graphcore IPUs

Burchard, Luk; Moe, Johannes; Schroeder, Daniel Thilo; Pogorelov, Konstantin; Langguth, Johannes

doi:10.1007/978-3-030-78713-4_16

Cited by 7 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparison with the GPU, we use the Gunrock framework [6], on up to 8 Volta V100 GPUs inside a DGX-2 system. Unlike previous work [8], we omit comparison with the CPU. The reason is that the CPU typically exceeds GPU and IPU performance only on high-diameter graphs where the available concurrency is too low for the GPU or IPU to benefit from their high number of parallel threads.…”

Section: Methodsmentioning

confidence: 99%

“…However, these are difficult to implement effectively on distributed memory, and they did not lead to performance gains in our IPU implementation. While AI processors only recently became available, the first papers that evaluate their usefulness for non-AI workloads have appeared recently, both for the Graphcore IPU [8], [30], [31] and for the Cerebras Wafer Scale Engine [32], presenting improvements on the state-of-the-art for a wide range of problems.…”

Section: Related Workmentioning

confidence: 99%

“…Our work builds upon the iPUG BFS code [8] which was written for a single GC2 IPU. In this paper we present a code that is capable of scaling to multiple IPUs, as well as using the newer GC200 IPU.…”

Section: Introductionmentioning

confidence: 99%

“…Second, IPU communication follows static patterns, and has to be planned at compile time. To get around this limitation, a 2-competitive solution was designed for iPUG [8]. However, when moving to the multi-IPU scenario, communication becomes much more costly since the deviceto-device links are considerably slower than the core-to-core communication inside the IPU.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

iPUG for Multiple Graphcore IPUs: Optimizing Performance and Scalability of Parallel Breadth-First Search

Burchard

Cai

Langguth

2021

2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)

Self Cite

View full text Add to dashboard Cite

Parallel graph algorithms have become one of the principal applications of high-performance computing besides numerical simulations and machine learning workloads. However, due to their highly unstructured nature, graph algorithms remain extremely challenging for most parallel systems, with large gaps between observed performance and theoretical limits. Furthermore, most mainstream architectures rely heavily on single instruction multiple data (SIMD) processing for high floatingpoint rates, which is not beneficial for graph processing which instead requires high memory bandwidth, low memory latency, and efficient processing of unstructured data.On the other hand, we are currently observing an explosion of new hardware architectures, many of which are adapted to specific purposes and diverge from traditional designs. A notable example is the Graphcore Intelligence Processing Unit (IPU), which is developed to meet the needs of upcoming machine intelligence applications.Its design eschews the traditional cache hierarchy, relying on SRAM as its main memory instead. The result is an extremely high-bandwidth, low-latency memory at the cost of capacity. In addition, the IPU consists of a large number of independent cores, allowing for true multiple instruction multiple data (MIMD) processing. Together, these features suggest that such a processor is well suited for graph processing.We test the limits of graph processing on multiple IPUs by implementing a low-level, high-performance code for breadth-first search (BFS), following the specifications of Graph500, the most widely used benchmark for parallel graph processing. Despite the simplicity of the BFS algorithm, implementing efficient parallel codes for it has proven to be a challenging task in the past. We show that our implementation scales well on a system with 8 IPUs and attains roughly twice the performance of an equal number of NVIDIA V100 GPUs using state-of-the-art CUDA code.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

iPUG for Multiple Graphcore IPUs: Optimizing Performance and Scalability of Parallel Breadth-First Search

Burchard

Cai

Langguth

2021

2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Motivated by the extreme computing power that theoretically can be delivered by the massively tiled AI processors, researchers have recently started applying these AI processors to "traditional" computational science. For example, Graphcore IPUs have been used for particle physics [7], computer vision [8], and graph processing [9,10]. Regarding mesh-based computations for numerically solving partial differential equations (PDEs), the current research effort is limited to porting stencil methods that are based on uniform meshes and finite difference discretization.…”

Section: Introductionmentioning

confidence: 99%

Enabling unstructured-mesh computation on massively tiled AI processors: An example of accelerating in silico cardiac simulation

et al. 2023

Self Cite

View full text Add to dashboard Cite

A new trend in processor architecture design is the packaging of thousands of small processor cores into a single device, where there is no device-level shared memory but each core has its own local memory. Thus, both the work and data of an application code need to be carefully distributed among the small cores, also termed as tiles. In this paper, we investigate how numerical computations that involve unstructured meshes can be efficiently parallelized and executed on a massively tiled architecture. Graphcore IPUs are chosen as the target hardware platform, to which we port an existing monodomain solver that simulates cardiac electrophysiology over realistic 3D irregular heart geometries. There are two computational kernels in this simulator, where a 3D diffusion equation is discretized over an unstructured mesh and numerically approximated by repeatedly executing sparse matrix-vector multiplications (SpMVs), whereas an individual system of ordinary differential equations (ODEs) is explicitly integrated per mesh cell. We demonstrate how a new style of programming that uses Poplar/C++ can be used to port these commonly encountered computational tasks to Graphcore IPUs. In particular, we describe a per-tile data structure that is adapted to facilitate the inter-tile data exchange needed for parallelizing the SpMVs. We also study the achievable performance of the ODE solver that heavily depends on special mathematical functions, as well as their accuracy on Graphcore IPUs. Moreover, topics related to using multiple IPUs and performance analysis are addressed. In addition to demonstrating an impressive level of performance that can be achieved by IPUs for monodomain simulation, we also provide a discussion on the generic theme of parallelizing and executing unstructured-mesh multiphysics computations on massively tiled hardware.

show abstract