Deterministic Atomic Buffering

Chou, Yuan Hsi; Ng, Curtise; Cattell, Shaylin; Intan, Jeremy; Sinclair, Matthew; Devietti, Joseph; Rogers, Timothy G.; Aamodt, Tor M.

doi:10.1109/micro50266.2020.00083

Cited by 7 publications

(1 citation statement)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While most sources of variability can be removed by a proper and well-documented setup (fixed seeds, well-defined hyperparameters, etc. ), a computational variability remains due to the numerical nondeterminism issued from the rounding of numbers associated with the stochastic order of arithmetic operations (Whitehead and Fit-Florea, 2011; Chou et al, 2020). It is notably observed when the same code is evaluated in different hardware (Jézéquel et al, 2015) or with different floating-point precisions (Seznec et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

On the reproducibility of fully convolutional neural networks for modeling time–space-evolving physical systems

Pinto

Alguacil

Bauerheim

2022

DCE

View full text Add to dashboard Cite

Reproducibility of a deep-learning fully convolutional neural network is evaluated by training several times the same network on identical conditions (database, hyperparameters, and hardware) with nondeterministic graphics processing unit operations. The network is trained to model three typical time–space-evolving physical systems in two dimensions: heat, Burgers’, and wave equations. The behavior of the networks is evaluated on both recursive and nonrecursive tasks. Significant changes in models’ properties (weights and feature fields) are observed. When tested on various benchmarks, these models systematically return estimations with a high level of deviation, especially for the recurrent analysis which strongly amplifies variability due to the nondeterminism. Trainings performed with double floating-point precision provide slightly better estimations and a significant reduction of the variability of both the network parameters and its testing error range.

show abstract

Section: Introductionmentioning

confidence: 99%