Efficient Gradient Boosted Decision Tree Training on GPUs

Wen, Zeyi; He, Bingsheng; Ramamohanarao, Kotagiri; Lu, Shengliang; Shi, Jiashuai

doi:10.1109/ipdps.2018.00033

Cited by 39 publications

(19 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Regarding previous work on GBT training acceleration techniques, multiple CPU threads, distributed system and GPU accelerated method [6], [7], [8], [9], [10] have been reported, and some of their implementations are being widely used. However, the degree of speedup remains small compared to that of Random Forest, which is another popular decision tree ensemble technique.…”

Section: Related Workmentioning

confidence: 99%

Scalable Hardware Architecture for fast Gradient Boosted Tree Training

Sadasue

Tanaka

Kasahara

et al. 2021

IPSJ Transactions on System LSI Design Methodology

View full text Add to dashboard Cite

Gradient Boosted Tree is a powerful machine learning method that supports both classification and regression, and is widely used in fields requiring high-precision prediction, particularly for various types of tabular data sets. Owing to the recent increase in data size, the number of attributes, and the demand for frequent model updates, a fast and efficient training is required. FPGA is suitable for acceleration with power efficiency because it can realize a domain specific hardware architecture; however it is necessary to flexibly support many hyper-parameters to adapt to various dataset sizes, dataset properties, and system limitations such as memory capacity and logic capacity. We introduce a fully pipelined hardware implementation of Gradient Boosted Tree training and a design framework that enables a versatile hardware system description with high performance and flexibility to realize highly parameterized machine learning models. Experimental results show that our FPGA implementation achieves a 11-to 33-times faster performance and more than 300-times higher power efficiency than a state-of-the-art GPU accelerated software implementation.

show abstract

Section: Related Workmentioning

confidence: 99%

Scalable Hardware Architecture for fast Gradient Boosted Tree Training

Sadasue

Tanaka

Kasahara

et al. 2021

IPSJ Transactions on System LSI Design Methodology

View full text Add to dashboard Cite

show abstract

“…In order to increase the accuracy of our decision tree model, we employed gradient boosting which is an ensemble learning technique with the capability of increasing the accuracy of our weak learner; decision tree into a strong learner by sequentially generating the decision tree in such a way that present one is always better than the previous. This is achieved by adding a new adaptive model to optimize the loss function of the previous decision tree algorithm [38][39][40]. GridSearchCV was employed to obtain the optimized hyperparameters in this paper.…”

Section: Gradient Boosted Decision Treementioning

confidence: 99%

Mobile Money Fraud Prediction—A Cross-Case Analysis on the Efficiency of Support Vector Machines, Gradient Boosted Decision Trees, and Naïve Bayes Algorithms

2020

View full text Add to dashboard Cite

The onset of COVID-19 has re-emphasized the importance of FinTech especially in developing countries as the major powers of the world are already enjoying the advantages that come with the adoption of FinTech. Handling of physical cash has been established as a means of transmitting the novel corona virus. Again, research has established that, been unbanked raises the potential of sinking one into abject poverty. Over the years, developing countries have been piloting the various forms of FinTech, but the very one that has come to stay is the Mobile Money Transactions (MMT). As mobile money transactions attempt to gain a foothold, it faces several problems, the most important of them is mobile money fraud. This paper seeks to provide a solution to this problem by looking at machine learning algorithms based on support vector machines (kernel-based), gradient boosted decision tree (tree-based) and Naïve Bayes (probabilistic based) algorithms, taking into consideration the imbalanced nature of the dataset. Our experiments showed that the use of gradient boosted decision tree holds a great potential in combating the problem of mobile money fraud as it was able to produce near perfect results.

show abstract

“…Em um trabalho posterior, [Mitchell et al 2018] usam uma abordagem XGBoost Multi-GPU, em que os dados de treinamento são particionados e processados por várias GPUs. [Wen et al 2018] paralelizam asárvores do XGBoost em nível de nó, nível de atributo e nível de divisão. Porúltimo, [Browne et al 2018] reorganizam uma floresta de decisão pronta na memória usando um leiaute planejado para acelerar o processo de classificação.…”

Section: Trabalhos Relacionadosunclassified

“…Como os dados de documentos textuais geralmente possuem muitos termos com valores iguais a zero, o conjunto pode ser representado de forma esparsa. Uma matriz esparsa ganha no armazenamento (apenas os valores diferentes de zero são guardados), sua desvantagemé o acesso [Wen et al 2018]. Existem vários esquemas para armazenamento de dados esparsos com suas próprias características.…”

Section: Paralelização Do Algoritmounclassified

Impulsionando Árvores Extremamente Aleatórias em Paralelo para a Classificação de Dados Textuais

Pires

Martins

2019

Anais Do XX Simpósio Em Sistemas Computacionais De Alto Desempenho (SSCAD 2019)

View full text Add to dashboard Cite

Os algoritmos de aprendizado usando conjuntos de árvores de decisão têm se destacado na classificação de documentos, mas não sem pagar um alto custo computacional. A exploração de paralelismo tem sido uma alternativa para viabilizar o uso destes algoritmos mais sofisticados. Neste trabalho propomos um algoritmo paralelo para acelerar a construção destas árvores de decisão utilizadas num método recente que demonstrou superar os classificadores de última geração para dados textuais. Resultados experimentais, utilizando bases de dados textuais padronizadas, mostram que o algoritmo implementado em uma arquitetura manycore (GPU) é capaz de reduzir o tempo de execução em até 26 vezes em comparação a um algoritmo sequencial equivalente.

show abstract

Efficient Gradient Boosted Decision Tree Training on GPUs

Cited by 39 publications

References 21 publications

Scalable Hardware Architecture for fast Gradient Boosted Tree Training

Scalable Hardware Architecture for fast Gradient Boosted Tree Training

Mobile Money Fraud Prediction—A Cross-Case Analysis on the Efficiency of Support Vector Machines, Gradient Boosted Decision Trees, and Naïve Bayes Algorithms

Impulsionando Árvores Extremamente Aleatórias em Paralelo para a Classificação de Dados Textuais

Contact Info

Product

Resources

About