Random numbers for parallel computers: Requirements and methods, with emphasis on GPUs

Abstract

We examine the requirements and the available methods and software to provide (or imitate) uniform random numbers in parallel computing environments. In this context, for the great majority of applications, independent streams of random numbers are required, each being computed on a single processing element at a time. Sometimes, thousands or even millions of such streams are needed. We explain how they can be produced and managed. We devote particular attention to multiple streams for GPU devices.

History of uniform random number generation

“…Several other encryption methods have been proposed recently, such as SHA, TEA, ChaCha, Threefish, etc. ; see Salmon et al (2011), L'Ecuyer et al (2017, and the references given there. Many of them are simpler and less secure than AES and BBS, but they are faster.…”

“…Multiple streams and substreams that can evolve independently of each other once initialized are very convenient for example to facilitate the synchronization when comparing different systems or different parameters of a system with common random numbers (Law and Kelton 1982, Bratley et al 1983, L'Ecuyer 2007 and are essential for running simulations on parallel processors (L'Ecuyer 2015, L'Ecuyer et al 2017.…”

“…Haramoto et al (2008aHaramoto et al ( , 2008b have proposed faster methods to do that via polynomial arithmetic. In parallel settings where thousands or millions of random streams are required, and more particularly on GPUs, RNGs with such a large state are not appropriate, because of the excessive overhead for initializing these streams and copying them across the different levels of computer memory (L'Ecuyer et al 2017). Mascagni and Srinivasan (2000) proposed a different type of software tool with multiple streams defined in a tree structure, for parallel simulation.…”

Pseudo-Random Number Generators for Vector Processors and Multicore Processors

“…Five different methods for producing independent streams have been proposed (L'Ecuyer, 1994;Salmon, 2011;L'Ecuyer, Oreshkin and Simard, 2014;Bauke and Mertens, 2007):…”

A Strategy for Parallel Implementations of Stochastic Lagrangian Simulation

“…SAF: the Strategy of Attachment to the (FPUs) where each FPU received a Virtual Random Number Generator (VRNG) which is either different independent Random Number Generators (RNGs) or copies of the same RNG in different states [10]. In this strategy, the random numbers are generated on demand and do not bear any attachment to the particles.…”