In search of numerical consistency in parallel programming

Robey, Robert W.; Robey, Jonathan M.; Aulwes, Rob

doi:10.1016/j.parco.2011.02.009

Cited by 26 publications

(26 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As with the atmospheric model code, they were able to dramatically increase the reproducibility of their results by employing a form of custom high-precision arithmetic in several critical summation loops [47].…”

Section: Numerical Reproducibility Problems In Real Applicationsmentioning

confidence: 99%

Facilitating Reproducibility in Scientific Computing: Principles and Practice

Bailey

Borwein²,

Stodden³

2016

Reproducibility: Principles, Problems, Practices, and Prospects

View full text Add to dashboard Cite

Section: Numerical Reproducibility Problems In Real Applicationsmentioning

confidence: 99%

Facilitating Reproducibility in Scientific Computing: Principles and Practice

Bailey

Borwein²,

Stodden³

2016

Reproducibility: Principles, Problems, Practices, and Prospects

View full text Add to dashboard Cite

“…These instabilities are detected if the subtraction of two close floating-point numbers leads to a loss of accuracy of at least four digits. Table 1 presents the number of exact significant digits estimated by CADNA for scheme (5), the occurrence number of each accuracy, and the absolute value range of the analysed results. Similar results are observed with the other scheme.…”

Section: Estimation Of Numerical Reproducibility By Means Of Dsamentioning

confidence: 99%

“…Questions whether such simulated results are reproducible or not have been reported more or less recently, e.g. in energy science [1], dynamic weather forecasting [2], atomic or molecular dynamic [3,4], fluid dynamic [5]. This paper focuses on numerical non-reproducibility due to the finite precision of computer arithmetic -see [6] for other issues about "reproducible research" in computational mathematics.…”

mentioning

confidence: 99%

First steps towards more numerical reproducibility

2014

View full text Add to dashboard Cite

Abstract. Questions whether numerical simulation is reproducible or not have been reported in several sensitive applications. Numerical reproducibility failure mainly comes from the finite precision of computer arithmetic. Results of floating-point computation depends on the computer arithmetic precision and on the order of arithmetic operations. Massive parallel HPC which merges, for instance, many-core CPU and GPU, clearly modifies these two parameters even from run to run on a given computing platform. How to trust such computed results? This paper presents how three classic approaches in computer arithmetic may provide some first steps towards more numerical reproducibility. Numerical reproducibility: context and motivationsAs computing power increases towards exascale, more complex and larger scale numerical simulations are performed in various domains. Questions whether such simulated results are reproducible or not have been reported more or less recently, e.g. in energy science [1], dynamic weather forecasting [2], atomic or molecular dynamic [3,4], fluid dynamic [5]. This paper focuses on numerical non-reproducibility due to the finite precision of computer arithmetic -see [6] for other issues about "reproducible research" in computational mathematics.The following example illustrates a typical failure of numerical reproducibility. In the energetic field, power system state simulation aims to compute at "real time" a reliable estimate of the bus voltages for a given power grid topology and a set of on-line measures. Numerically speaking, a large and sparse linear system is solved at every iteration of a Newton-Raphson process. The core computation is a sparse matrix-vector product automatically parallelised by the computing environment. The authors of [1] exhibit a significant variability (up to 25% relative difference) between two runs on a massively multithreaded system. The culprit? Here as in the previously cited references: non deterministic sums.Floating-point summation is not associative. Parallelism introduces non deterministic events from run to run, even using a unique binary file and a given computing platform. The order of communications, the number of computing units (threads, processors) and the associated data placement may vary, and hence the parallel partial sums. Even sequential frames that comply with the IEEE-754 floating-point arithmetic standard [7] are still numerically very sensitive to many features: low-level arithmetic unit properties (variable precision registers, fused operators), compiler optimizations, language flaws or library versions reduce numerical repeatability and numerical portability [8]. * Authors thank Cl.-P. Jeannerod (INRIA) for his significant contribution, I. Said (LIP6) for his help in the numerical experiments related to the acoustic wave equation and the GT Arith, GDR Informatique Mathématique, for its support.

show abstract

“…Such differences are particularly noticeable with new computing architectures such as multicore processors, GPUs (Graphics Processing Units) and APUs (Accelerated Processing Units). In high performance numerical simulations, reproducibility problems have been identified in various domains: energy science [1], climate science [2], atomic or molecular dynamics [3], [4], fluid dynamics [5]. Various studies have been carried out on numerical reproducibility on different architectures.…”

Section: Introductionmentioning

confidence: 99%

“…Various studies have been carried out on numerical reproducibility on different architectures. On the one hand, strategies have been proposed [2], [3], [4], [5] to improve numerical accuracy, using for instance accurate summations. Other works aim at forcing the reproducibility of results, either affected by the same rounding errors [6], [7] or correctly rounded [8], [9], [10], [11].…”

Section: Introductionmentioning

confidence: 99%

Estimation of numerical reproducibility on CPU and GPU

Jézéquel¹,

Lamotte²,

Said³

2015

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Abstract-Differences in simulation results may be observed from one architecture to another or even inside the same architecture. Such reproducibility failures are often due to different rounding errors generated by different orders in the sequence of arithmetic operations. Reproducibility problems are particularly noticeable on new computing architectures such as multicore processors or GPUs (Graphics Processing Units). DSA (Discrete Stochastic Arithmetic) enables one to estimate rounding error propagation in simulation programs. In this paper, it is shown that DSA can be used to estimate which digits in simulation results may be different from one environment to another because of rounding errors. A particular implementation of DSA, which enables numerical validation in hybrid CPU-GPU environments, is described. The estimation of numerical reproducibility using DSA is illustrated by a wave propagation code which can be affected by reproducibility problems when executed on different architectures.

show abstract

In search of numerical consistency in parallel programming

Cited by 26 publications

References 10 publications

Facilitating Reproducibility in Scientific Computing: Principles and Practice

Facilitating Reproducibility in Scientific Computing: Principles and Practice

First steps towards more numerical reproducibility

Estimation of numerical reproducibility on CPU and GPU

Contact Info

Product

Resources

About