Interpretable decision-tree induction in a big data parallel framework

Abstract

When running data-mining algorithms on big data platforms, a parallel, distributed framework, such as MAPREDUCE, may be used. However, in a parallel framework, each individual model fits the data allocated to its own computing node without necessarily fitting the entire dataset. In order to induce a single consistent model, ensemble algorithms such as majority voting, aggregate the local models, rather than analyzing the entire dataset directly. Our goal is to develop an efficient algorithm for choosing one representative model from multiple, locally induced decision-tree models. The proposed SySM (syntactic similarity method) algorithm computes the similarity between the models produced by parallel nodes and chooses the model which is most similar to others as the best representative of the entire dataset. In 18.75% of 48 experiments on four big datasets, SySM accuracy is significantly higher than that of the ensemble; in about 43.75% of the experiments, SySM accuracy is significantly lower; in one case, the results are identical; and in the remaining 35.41% of cases the difference is not statistically significant. Compared with ensemble methods, the representative tree models selected by the proposed methodology are more compact and interpretable, their induction consumes less memory, and, as confirmed by the empirical results, they allow faster classification of new records.

Statistical measurement of trees’ similarity

“…Edit-distance is a poor general comparator for diagnostic trees for several reasons. One is that existing algorithms do not take into account that nodes in the tree are not equally important (Jiang et al 1995;Weinberg and Last 2017 Trees B and C are each inconsistent with Tree A in exactly one way. Tree B swaps nodes 2 and 5, while Tree C swaps nodes 2 and 3.…”

Parallelizing user–defined functions in the ETL workflow using orchestration style sheets

“…MapReduce (Dean and Ghemawat, 2008) is one of the popular programming models focusing on automatic data-flow parallelism. It is a popular choice to perform big data analysis with data mining algorithms in a parallel distributed computing environment (Weinberg and Last, 2017). The MapReduce programming model has proven a significant decrease in the execution time of computing-intensive workflows or processes when executing in a distributed parallel environment, e.g., Hadoop (González-Vélez and Kontagora, 2011).…”

Regression Function and Noise Variance Tracking Methods for Data Streams with Concept Drift

“…Although the number of the algorithms for data stream mining is not as large as in the case of traditional data mining, in the recent decade there has been a considerable progress in this field. The most successful seem algorithms based on decision trees (Domingos and Hulten, 2000;Jaworski et al, 2017;Rutkowski et al, 2015;Weinberg and Last, 2017) and ensemble methods (Pietruczuk et al, 2017;Wang et al, 2003). They are mainly devoted to data classification problems.…”