Evolutionary induction of a decision tree for large-scale data: a GPU-based approach

Jurczuk, Krzysztof; Czajkowski, Marcin; Krȩtowski, Marek

doi:10.1007/s00500-016-2280-1

Cited by 33 publications

(12 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accelerated version of the GDT is built on the GPU-based predecessor (Jurczuk et al, 2017). As it is illustrated in Figure 2, the core evolution is still run sequentially on a CPU.…”

Section: Repository-supported Gpu-based Approachmentioning

confidence: 99%

“…In this paper, we extend a GPU-based global induction of classification trees (Jurczuk et al, 2017) to accelerate the evolutionary search for the large-scale data. Noting that a lot of trees or their parts reappear during the evolutionary search, we examine if and when it is worth to archive the most popular individuals (DTs) and reuse them.…”

Section: Introductionmentioning

confidence: 99%

“…Our research was performed on a Global Decision Tree (GDT) system (Kretowski, 2019) which allows evolving different types of classification and regression trees and may be applied in real-life applications (Czajkowski and Kretowski, 2019). In the GPU-supported version of the GDT (Jurczuk et al, 2017), the main evolutionary loop (selection, genetic operators, etc.) is performed sequentially on a CPU, while the most time-consuming operations like fitness calculation are delegated to a GPU.…”

Section: Introductionmentioning

confidence: 99%

“…However, it seems that the problem of speeding up the evolutionary DT induction is still not adequately explored. To the best of the authors’ knowledge, only the GDT system was investigated in the context of MPI/OpenMP (Czajkowski et al, 2015), Spark (Reska et al, 2018) and GPU (Jurczuk et al, 2017) parallelizations. Remaining studies cover either greedy inducers (Lo et al, 2014; Strnad and Nerat, 2016) either random forests (Grahn et al, 2011; Marron et al, 2014) which are beyond the scope and interest of the research presented in this paper.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Fitness evaluation reuse for accelerating GPU-based evolutionary induction of decision trees

Jurczuk

Czajkowski

Krȩtowski

2020

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

Decision trees (DTs) are one of the most popular white-box machine-learning techniques. Traditionally, DTs are induced using a top-down greedy search that may lead to sub-optimal solutions. One of the emerging alternatives is an evolutionary induction inspired by the biological evolution. It searches for the tree structure and tests simultaneously, which results in less complex DTs with at least comparable prediction performance. However, the evolutionary search is computationally expensive, and its effective application to big data mining needs algorithmic and technological progress. In this paper, noting that many trees or their parts reappear during the evolution, we propose a reuse strategy. A fixed number of recently processed individuals (DTs) is stored in a so-called repository. A part of the repository entry (related to fitness calculations) is maintained on a CPU side to limit CPU/GPU memory transactions. The rest of the repository entry (tree structures) is located on a GPU side to speed up searching for similar DTs. As the most time-demanding task of the induction is the DTs’ evaluation, the GPU first searches similar DTs in the repository for reuse. If it fails, the GPU has to evaluate DT from the ground up. Large artificial and real-life datasets and various repository strategies are tested. Results show that the concept of reusing information from previous generations can accelerate the original GPU-based solution further. It is especially visible for large-scale data. To give an idea of the overall acceleration scale, the proposed solution can process even billions of objects in a few hours on a single GPU workstation.

show abstract

“…The accelerated version of the GDT is built on the GPU-based predecessor (Jurczuk et al, 2017). As it is illustrated in Figure 2, the core evolution is still run sequentially on a CPU.…”

Section: Repository-supported Gpu-based Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fitness evaluation reuse for accelerating GPU-based evolutionary induction of decision trees

Jurczuk

Czajkowski

Krȩtowski

2020

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additional efficiency improvements, especially in context of storing and preprocessing full list of non-dominated solutions, need to be considered. Performance issue may also be partially mitigated with parallelization of GMT with, e.g., MPI-OpenMP (Czajkowski et al 2015), GPGPU (Jurczuk et al 2017), or Apache Spark (Reska et al 2018) approaches. In addition, we are constantly working on further comprehensibility improvement of the generated Pareto front and plan to extend our research to cover all types of the decision trees.…”

Section: Fitting Pareto Front Based On Analytical Preferencesmentioning

confidence: 99%

A multi-objective evolutionary approach to Pareto-optimal model trees

Czajkowski

Krȩtowski

2018

Soft Comput

Self Cite

View full text Add to dashboard Cite

This paper discusses the multi-objective evolutionary approach to induction of model trees. The model tree is a particular case of a decision tree designed to solve regression problems. Although the decision tree induction is inherently a multiobjective task, most of the conventional learning algorithms can only deal with a single objective that may possibly aggregate multiple objectives. The goal of this paper is to demonstrate how a set of non-dominated model trees can be obtained using the Global Model Tree (GMT) system. The GMT framework can be used for the evolutionary induction of different types of decision trees, including univariate, oblique or mixed; regression and model trees. Proposed Pareto approach for GMT allows the decision maker to select desired output model according to his preferences on the conflicting objectives. Performed study covers the regression trees and the model trees with two or three objectives that relate to the tree error and the tree comprehensibility. Experimental evaluation discusses the importance of multi-objective components like crowding function and archive elitist selection, using real-life datasets. Finally, the presented multi-objective GMT solution is confronted with competitive regression and model tree inducers.

show abstract

Parallel Computations for Evolutionary Induction

Krȩtowski

2019

Studies in Big Data

View full text Add to dashboard Cite

Evolutionary induction of a decision tree for large-scale data: a GPU-based approach

Cited by 33 publications

References 42 publications

Fitness evaluation reuse for accelerating GPU-based evolutionary induction of decision trees

Fitness evaluation reuse for accelerating GPU-based evolutionary induction of decision trees

A multi-objective evolutionary approach to Pareto-optimal model trees

Parallel Computations for Evolutionary Induction

Contact Info

Product

Resources

About