Numerical reproducibility for the parallel reduction on multi- and many-core architectures

Collange, Sylvain; Defour, David; Graillat, Stef; Iakymchuk, Roman

doi:10.1016/j.parco.2015.09.001

Cited by 52 publications

(67 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other side, we want to preserve every bit of information before the final rounding to the desired format to assure correct-rounding and, therefore, reproducibility. To accomplish our goal, we merge together FPE and long accumulators, tune them, and efficiently implement them on various architectures, including conventional CPUs, Nvidia and AMD GPUs, and Intel Xeon Phi co-processors (for details we refer to Reference [13]).…”

Section: Exblas: Accurate and Bitwise Reproducible Basic Linear Algebmentioning

confidence: 99%

Reproducibility, accuracy and performance of the Feltor code and library on parallel computer architectures

Wiesenberger

Einkemmer

Held

et al. 2019

Computer Physics Communications

Self Cite

View full text Add to dashboard Cite

Feltor is a modular and free scientific software package. It allows developing platform independent code that runs on a variety of parallel computer architectures ranging from laptop CPUs to multi-GPU distributed memory systems. Feltor consists of both a numerical library and a collection of application codes built on top of the library. Its main target are two-and three-dimensional drift-and gyro-fluid simulations with discontinuous Galerkin methods as the main numerical discretization technique.We observe that numerical simulations of a recently developed gyro-fluid model produce non-deterministic results in parallel computations. First, we show how we restore accuracy and bitwise reproducibility algorithmically and programmatically. In particular, we adopt an implementation of the exactly rounded dot product based on long accumulators, which avoids accuracy losses especially in parallel applications. However, reproducibility and accuracy alone fail to indicate correct simulation behaviour. In fact, in the physical model slightly different initial conditions lead to vastly different end states. This behaviour translates to its numerical representation. Pointwise convergence, even in principle, becomes impossible for long simulation times. We briefly discuss alternative methods to ensure the correctness of results like the convergence of reduced physical quantities of interest, ensemble simulations, invariants or reduced simulation times.In a second part, we explore important performance tuning considerations. We identify latency and memory bandwidth as the main performance indicators of our routines. Based on these, we propose a parallel performance model that predicts the execution time of algorithms implemented in Feltor and test our model on a selection of parallel hardware architectures. We are able to predict the execution time with a relative error of less than 25% for problem sizes between 10 −1 and 10 3 MB. Finally, we find that the product of latency and bandwidth gives a minimum array size per compute node to achieve a scaling efficiency above 50% (both strong and weak).

show abstract

Section: Exblas: Accurate and Bitwise Reproducible Basic Linear Algebmentioning

confidence: 99%

Reproducibility, accuracy and performance of the Feltor code and library on parallel computer architectures

Wiesenberger

Einkemmer

Held

et al. 2019

Computer Physics Communications

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are two other academic efforts to guarantee reproducibility. The first [7], which is built upon [33], Alternatively, Collange et al [6] proposed a multi-level approach to compute the reproducible summation.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical approach for deriving a reproducible unblocked LU factorization

Iakymchuk

Graillat²,

Defour

et al. 2019

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

We propose a reproducible variant of the unblocked LU factorization for graphics processor units (GPUs).For this purpose, we build upon Level-1/2 BLAS kernels that deliver correctly-rounded and reproducible results for the dot (inner) product, vector scaling, and the matrix-vector product. In addition, we draw a strategy to enhance the accuracy of the triangular solve via iterative refinement. Following a bottom-up approach, we finally construct a reproducible unblocked implementation of the LU factorization for GPUs, which accommodates partial pivoting for stability and can be eventually integrated in a high performance and stable algorithm for the (blocked) LU factorization.The IEEE 754 standard, created in 1985 and then revised in 2008, has led to a considerable enhancement in the reliability of numerical computations by rigorously specifying the properties of floating-point arithmetic. This standard is now adopted by most processors, thus leading to a much better portability of numerical applications.Exascale computing (10 18 operations per second) is likely to be reached within a decade. For the type of systems yielding such performance rate, getting accurate and reproducible 1 results in floating-point arithmetic will represent two considerable challenges [9,28]. Reproducibility is also an important and useful property when debugging and checking the correctness of codes as well as for legal issues.Email addresses: riakymch@kth.se (Roman Iakymchuk), stef.graillat@lip6.fr (Stef Graillat), david.defour@univ-perp.fr (David Defour), quintana@uji.es (Enrique S. Quintana-Ortí) 1 By accuracy, we mean the relative error between the exact result and the computed result. We define reproducibility as the ability to obtain a bit-wise identical floating-point result from multiple runs of the code on the same input data.

show abstract

“…Significance arithmetics is regaining interest thanks to the Unum proposal [3] or indirectly through numerous problems encountered with exascale computers and the lack of confidence in numerical results [4]. If the Unum system is based on real problems, the proposed solution is subject to criticism for numerous reasons as pointed out by W. Kahan [5].…”

Section: Introductionmentioning

confidence: 99%

“…If the Unum system is based on real problems, the proposed solution is subject to criticism for numerous reasons as pointed out by W. Kahan [5]. On the other hand, indirect solutions based on software solutions to detect cancellation [6], [7], or avoiding rounding errors [4] are not meant to be efficient nor effective for real time execution.…”

Section: Introductionmentioning

confidence: 99%

FP-ANR: A representation format to handle floating-point cancellation at run-time

Defour

2018

2018 IEEE 25th Symposium on Computer Arithmetic (ARITH)

Self Cite

View full text Add to dashboard Cite

When dealing with floating-point numbers, there are several sources of error which can drastically reduce the numerical quality of computed results. One of those error sources is the loss of significance or cancellation, which occurs during for example, the subtraction of two nearly equal numbers. In this article, we propose a representation format named Floating-Point Adaptive Noise Reduction (FP-ANR). This format embeds cancellation information directly into the floating-point representation format thanks to a dedicated pattern. With this format, insignificant trailing bits lost during cancellation are removed from every manipulated floating-point number. The immediate consequence is that it increases the numerical confidence of computed values. The proposed representation format corresponds to a simple and efficient implementation of significance arithmetic based and compatible with the IEEE Standard 754 standard.

show abstract

Numerical reproducibility for the parallel reduction on multi- and many-core architectures

Cited by 52 publications

References 18 publications

Reproducibility, accuracy and performance of the Feltor code and library on parallel computer architectures

Reproducibility, accuracy and performance of the Feltor code and library on parallel computer architectures

Hierarchical approach for deriving a reproducible unblocked LU factorization

FP-ANR: A representation format to handle floating-point cancellation at run-time

Contact Info

Product

Resources

About