On Optimizing the Arithmetic Precision of MCMC Algorithms

Mingas, Grigorios; Rahman, Farhan; Bouganis, Christos-Savvas

doi:10.1109/fccm.2013.31

Cited by 8 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous works on FPGAs using custom precision can be found in [16], [17], [22], [23]. [23] propose the use of custom precision arithmetic for population-based MCMC methods where multiple parallel chains are used to improve the mixing properties of the chain.…”

Section: Specialised Hardware Approachesmentioning

confidence: 99%

“…However, their method requires knowledge of the function of interest during the design of the system, and as such the generated samples cannot be used for other estimates. [16] propose a method under which they can estimate using short pre-runs of the system the bias of the estimate under various custom precision schemes. The final run is performed utilising the lowest possible precision that does not violate the user's acceptable bias at the estimate.…”

Section: Specialised Hardware Approachesmentioning

confidence: 99%

“…Following [16], M can be estimated by short MCMC pre-runs. As the LD n (θ) − LC n (θ) LD n (θ) factor in equation (19) takes small values, the variance of the estimation of M will drop down fast as the number of samples increases.…”

Section: Custom Precision Tuningmentioning

confidence: 99%

“…However, both approaches lead to biased estimates that exhibit large variance due to the approximations of the target distribution. Even though a controlled biased estimate can be accepted in certain applications [16], there is a large number of applications where unbiased estimates 1 of the given parameters of the sampling distribution are required, and MCMC algorithms are expected to perform exact inference in these problems [17].…”

Section: Introductionmentioning

confidence: 99%

“…// partial sampling of z n for dark data 15: if n%ResampleF raction = RandInteger(1, ResampleF raction) then 16:…”

mentioning

confidence: 99%

See 4 more Smart Citations

An Unbiased MCMC FPGA-Based Accelerator in the Land of Custom Precision Arithmetic

Liu

Mingas

Bouganis

2017

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract-Markov Chain Monte Carlo (MCMC) based methods have been the main tool used for Bayesian Inference by practitioners and researchers due to their flexibility and theoretical properties that guarantee unbiased sampling-based estimates. Nevertheless, with the availability of large data sets and the constant need to develop more complex models that better capture the targeted problem, significant computational challenges have been presented. Current approaches, based on multi-core CPUs, GPUs, and FPGAs, aim to accelerate the execution time of the MCMC methods using subsampling techniques or custom precision arithmetic, resulting to biased estimates. In this work, a novel FPGA-based construction is proposed that utilises the custom precision support of FPGA devices in order to accelerate the computations, guaranteeing at the same time asymptotically unbiased estimates. Key to this approach is the extension of the parameter space by an extra parameter that indicates the required precision in the computation of the likelihood of a data point. The work proposes an FPGA architecture for the above algorithm, as well as discuss its tuning for maximising the performance of the system. The performance of the FPGA-mapped sampler is evaluated using two Bayesian logistic regression case studies of varying complexity, which show significant speedups compared to existing FPGA-and CPU-based works that utilise double floating point arithmetic, without any bias on the sampling-based estimates.

show abstract

Section: Specialised Hardware Approachesmentioning

confidence: 99%

Section: Specialised Hardware Approachesmentioning

confidence: 99%

Section: Custom Precision Tuningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…// partial sampling of z n for dark data 15: if n%ResampleF raction = RandInteger(1, ResampleF raction) then 16:…”

mentioning

confidence: 99%

See 3 more Smart Citations

An Unbiased MCMC FPGA-Based Accelerator in the Land of Custom Precision Arithmetic

Liu

Mingas

Bouganis

2017

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

Communication-Aware MCMC Method for Big Data Applications on FPGAs

Liu

Bouganis

2017

2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)

Self Cite

View full text Add to dashboard Cite

Markov Chain Monte Carlo (MCMC) based methods have been the main tool for Bayesian Inference for some years now, and recently they find increasing applications in modern statistics and machine learning. Nevertheless, with the availability of large datasets and increasing complexity of Bayesian models, MCMC methods are becoming prohibitively expensive for real-world problems. At the heart of these methods, lies the computation of likelihood functions that requires access to all input data points in each iteration of the method. Current approaches, based on data subsampling, aim to accelerate these algorithms by reducing the number of the data points for likelihood evaluations at each MCMC iteration. However the existing work doesn't consider the properties of memory hierarchies in modern computational systems, but treats the memory as one monolithic storage space. This paper proposes a communicationaware MCMC framework that takes into account the underlying performance of the memory subsystem during the sampling process, leading to faster execution times. The framework is based on a novel subsampling algorithm that utilises an unbiased likelihood estimator based on Probability Proportional-to-Size (PPS) sampling, allowing information on the performance of the memory system to be taken into account during the sampling stage. The proposed MCMC sampler is mapped to an FPGA device and its performance is evaluated using the Bayesian logistic regression model on MNIST dataset. The proposed system achieves a 3.37x speed up over a highly optimised MCMC-based FPGA design, and therefore the risk in the estimates based on the generated samples is largely decreased.

show abstract