Accelerating a Random Forest Classifier: Multi-Core, GP-GPU, or FPGA?

Abstract

Abstract-Random forest classification is a well known machine learning technique that generates classifiers in the form of an ensemble ("forest") of decision trees. The classification of an input sample is determined by the majority classification by the ensemble. Traditional random forest classifiers can be highly effective, but classification using a random forest is memory bound and not typically suitable for acceleration using FPGAs or GP-GPUs due to the need to traverse large, possibly irregular decision trees. Recent work at Lawrence Livermore National Laboratory has developed several variants of random forest classifiers, including the Compact Random Forest (CRF), that can generate decision trees more suitable for acceleration than traditional decision trees. Our paper compares and contrasts the effectiveness of FPGAs, GP-GPUs, and multi-core CPUs for accelerating classification using models generated by compact random forest machine learning classifiers.Taking advantage of training algorithms that can produce compact random forests composed of many, small trees rather than fewer, deep trees, we are able to regularize the forest such that the classification of any sample takes a deterministic amount of time. This optimization then allows us to execute the classifier in a pipelined or single-instruction multiple thread (SIMT) fashion. We show that FPGAs provide the highest performance solution, but require a multi-chip / multi-board system to execute even modest sized forests. GP-GPUs offer a more flexible solution with reasonably high performance that scales with forest size. Finally, multi-threading via OpenMP on a shared memory system was the simplest solution and provided near linear performance that scaled with core count, but was still significantly slower than the GP-GPU and FPGA.

Small Random Forest Models for Effective Chemogenomic Active Learning

“…To date, several non-chemogenomic studies have shown that the number of trees in a random forest could be reduced to a certain degree without losing model performance [64][65][66][67][68], and often rule-of-thumb suggestions are made and accepted within the different computational communities, which allow model training without previous parameter optimization. To probe whether parameter optimization might be an advantageous step instead of accepting rule-of-thumb guidelines, we investigated how chemogenomic active learning performance changed when reducing the number of trees.…”

Efficient disease detection in gastrointestinal videos – global features versus neural networks

“…Our implementation of the random forest classifier uses the version provided by the Weka machine learning library 5 [16], which is a collection of algorithms for machine learning and data mining. We chose the random forest approach, because it is fast and achieves good results [49]. It is important to point out that for this step, another classification algorithm can also be used.…”

FPGA Implementation of Decision Trees and Tree Ensembles for Character Recognition in Vivado Hls

“…It is notable that in most systems this design will be I/O It is notable that all tested software implementations were single threaded, but decision tree ensembles can be effectively implemented as multithreaded programs running on multicore processors [17].…”

“…Parallel implementations of random forests for URL classification on multicore CPU, GPGPU and FPGA are presented in [17]. Publicly available dataset is used by the au- …”

“…In recent years, many parallel implementations of both random forest learning and prediction were presented, targeted at multicore CPUs, GPGPUs and FPGAs [11,13,14,15,17]. Each sample from input data set is classified independently of others, making it possible to exploit datalevel parallelism by processing different samples by the same classification/regression tree simultaneously.…”