IBM's POWER10 Processor

Starke, William J.; Thompto, Brian W.; Stuecheli, Jeff; Moreira, José E.

doi:10.1109/mm.2021.3058632

Cited by 19 publications

(15 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…System Setup. Our POWER10 system has one single-chip module (SCM) with 15 SMT8 cores, equivalent to 30 SMT4 cores [41]. Each SMT4 core has 32 KB private L1 and 1MB private L2 caches.…”

Section: Methodsmentioning

confidence: 99%

“…These specialized units can execute dense matrix-multiply operations on small matrices (e.g., blocks of size 4 × 4). Examples of these are NVIDIA's Tensor Cores [2,5], IBM's POWER10 Matrix-Multiply Assist (MMA) facilities [12,41], and Intel's AMX [6]. These units are successfully utilized to maximize performance for dense matrix operations [2,5,32].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dense dynamic blocks

Yesil

Moreira²,

Torrellas

2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Recent processors have been augmented with matrix-multiply units that operate on small matrices, creating a functional unit-rich environment. These units have been successfully employed on dense matrix operations such as those found in the Basic Linear Algebra Subprograms (BLAS). In this work, we exploit these new matrixmultiply facilities to speed up Sparse Matrix Dense Matrix Multiplications (SpMM) for highly sparse matrices.SpMM is hard to optimize. The sparsity patterns lead to a highly irregular memory access behavior. Additionally, each sparse matrix has unique characteristics, making it hard to find a single SpMM strategy that works well for all sparse matrices. The addition of matrix-multiply units makes this even more challenging.In this paper, we address these challenges. First, we design Dense Dynamic Blocks (DDB), a method to utilize the new matrix units. DDB has two specialized versions: DDB-MM and DDB-HYB. DDB-MM is a strategy that only utilizes the matrix-multiply facilities. DDB-HYB is a hybrid approach that maximizes the floating-point throughput by utilizing both vector and matrix units. Furthermore, we design a prediction mechanism for identifying the best SpMM strategy for a given sparse matrix and dense matrix pair: SpMM-OPT. SpMM-OPT selects among vector unit oriented, matrix unit oriented, and hybrid strategies for the highest floating-point throughput while taking cache optimizations into account.We experiment with 440 matrices from the well-known SuiteSparse matrix collection on a POWER10 system with vector and matrix units. We show that DDB-MM and DDB-HYB can achieve a floating-point throughput of up to 1.1 and 2.5 TFLOPs/s on a POWER10 single-chip module for double-and single-precision SpMM, respectively. Our analysis also shows that SpMM-OPT effectively chooses the best SpMM strategy and can achieve an average speedup of up to 2× compared to an optimized CSR baseline.

show abstract

“…System Setup. Our POWER10 system has one single-chip module (SCM) with 15 SMT8 cores, equivalent to 30 SMT4 cores [41]. Each SMT4 core has 32 KB private L1 and 1MB private L2 caches.…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Dense dynamic blocks

Yesil

Moreira²,

Torrellas

2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…In addition, custom hardware for memory sharing relies on emerging open standards, such as Open Memory Interface (OMI) and Compute Express Link (CXL). IBM's Power10 has a memory clustering feature called memory inception, which allows one PM to map its address space to the physical memory of another PM by leveraging OMI memory [37]. It allows a PM to run large memory workloads that go beyond its capacity by borrowing memory from other PMs.…”

Section: Commercial Alternative Solutionsmentioning

confidence: 99%

Memory sharing for handling memory overload on physical machines in cloud data centers

Tian

Zhou

et al. 2023

J Cloud Comp

View full text Add to dashboard Cite

Over-committing computing resources is a widely adopted strategy for increased cluster utilization in Infrastructure as a Service (IaaS) cloud data centers. A potential consequence of over-committing computing resources is memory overload of physical machines (PMs). Memory overload occurs if memory usage exceeds a defined alarm threshold, exposing running computation tasks at a risk of being terminated by the operating system. A prevailing measure to handle memory overload of a PM is live migration of virtual machines (VMs). However, this not only consumes network bandwidth, CPU, and other resources, but also compels a temporary unavailability of the VMs being migrated. To handle memory overload, we present a memory sharing system in this paper for PMs in cloud data centers. With memory sharing, a PM automatically borrows memory from a remote PM when necessary, and releases the borrowed memory when memory overload disappears. This is implemented through swapping inactive memory pages to remote memory resource. Experimental studies conducted on InfiniBand-networked PMs show that the memory sharing system is fully functional. The measured throughput and latency are around 929 Mbps and 1.3 $$\mu$$ μ s, respectively, on average for remote memory access. They are similar to those from accessing a local-volatile memory express solid-state drive, and thus are promising in real applications.

show abstract

“…This is inline with the throughput of the two matrix pipelines being double that of the four vector pipes. For more performance results on both HPL and ResNet-50 (also 4× the per core performance of POWER9), we refer the reader to [21]. We take a closer look at the performance of the 128 × 128 DGEMM kernel in Figure 11.…”

Section: Performance Measurementsmentioning

confidence: 99%

“…The IBM POWER10 processor [20], [21] is the compute engine for the next generation of Power Systems and successor to the current POWER9 [16], [19] processor. As such, it has to offer superior performance on applications of interest to Power Systems users.…”

Section: Introductionmentioning

confidence: 99%

A matrix math facility for Power ISA(TM) processors

Moreira¹,

Barton²,

Battle³

et al. 2021

Preprint

View full text Add to dashboard Cite

Power ISA™ Version 3.1 has introduced a new family of matrix math instructions, collectively known as the Matrix-Multiply Assist (MMA) facility. The instructions in this facility implement numerical linear algebra operations on small matrices and are meant to accelerate computation-intensive kernels, such as matrix multiplication, convolution and discrete Fourier transform. These instructions have led to a power-and area-efficient implementation of a high throughput math engine in the future POWER10 processor. Performance per core is 4 times better, at constant frequency, than the previous generation POWER9 processor. We also advocate the use of compiler builtins as the preferred way of leveraging these instructions, which we illustrate through case studies covering matrix multiplication and convolution.

show abstract

IBM's POWER10 Processor

Cited by 19 publications

References 5 publications

Dense dynamic blocks

Dense dynamic blocks

Memory sharing for handling memory overload on physical machines in cloud data centers

A matrix math facility for Power ISA(TM) processors

Contact Info

Product

Resources

About