Parallel Resampling in the Particle Filter

Abstract

Modern parallel computing devices, such as the graphics processing unit (GPU), have gained significant traction in scientific and statistical computing. They are particularly well-suited to dataparallel algorithms such as the particle filter, or more generally Sequential Monte Carlo (SMC), which are increasingly used in statistical inference. SMC methods carry a set of weighted particles through repeated propagation, weighting and resampling steps. The propagation and weighting steps are straightforward to parallelise, as they require only independent operations on each particle. The resampling step is more difficult, as standard schemes require a collective operation, such as a sum, across particle weights. Focusing on this resampling step, we analyse two alternative schemes that do not involve a collective operation (Metropolis and rejection resamplers), and compare them to standard schemes (multinomial, stratified and systematic resamplers). We find that, in certain circumstances, the alternative resamplers can perform significantly faster on a GPU, and to a lesser extent on a CPU, than the standard approaches. Moreover, in single precision, the standard approaches are numerically biased for upwards of hundreds of thousands of particles, while the alternatives are not. This is particularly important given greater single-than double-precision throughput on modern devices, and the consequent temptation to use single precision with a greater number of particles. Finally, we provide auxiliary functions useful for implementation, such as for the permutation of ancestry vectors to enable in-place propagation.

A practical introduction to butterfly and adaptive resampling in Sequential Monte Carlo

“…There are various existing works on parallel implementations of SMC. Amongst several others Brun et al (2002) considered particle filters in a distributed computing setting, Bolić et al (2005) devised algorithms in which interaction occurs occasionally between blocks of particles, Hendeby et al (2007) considered a GPU implementation, Vergé et al (2013) have suggested algorithms with resampling on two hierarchical levels, and Murray et al (2014) outlines some different approaches to parallel implementation of various existing resampling techniques and compares their efficiencies. Paige et al (2014) propose an algorithm substantially different form of SMC method and, which involves a branching rather than resampling mechanism, so to deal with issues of synchronicity issues.…”

Moving-Target Position Estimation Using GPU-Based Particle Filter for IoT Sensing Applications

“…In our implementation, we use systematic resampling algorithm [25,26] in which the random number generation is required in resampling process. There is a study [27] to accelerate resampling in GPU-based PF. In our study, for reducing the additional overhead running random number calculation during the resampling process, we utilized thread-based running method, which is provided by host CPU, in the process of copying data to the GPU.…”

MapReduce particle filtering with exact resampling and deterministic runtime

“…One strategy for parallel implementation (discussed in [44] and explored in more detail elsewhere [32]) is to deliberately choose an alternative resampling algorithm such that the alternative algorithm is more amenable to parallel implementation. This paper focuses on systematic resampling specifically.…”