Scaling up data mining algorithms: review and taxonomy

Abstract

The overwhelming amount of data that are now available in any field of research poses new problems for data mining and knowledge discovery methods. Due to this huge amount of data, most of the current data mining algorithms are inapplicable to many real-world problems. Data mining algorithms become ineffective when the problem size becomes very large. In many cases, the demands of the algorithm in terms of the running time are very large, and mining methods cannot be applied when the problem grows. This aspect is closely related to the time complexity of the method. A second problem is linked with performance; although the method might be applicable, the size of the search space prevents an efficient execution, and the resulting solutions are unsatisfactory. Two approaches have been used to deal with this problem: scaling up data mining algorithms and data reduction. However, because data reduction is a data mining task itself, this technique also suffers from scalability problems. Thus, for many problems, especially when dealing with very large datasets, the only way to deal with the aforementioned problems is to scale up the data mining algorithm. Many efforts have been made to obtain methods that can be used to scale up existing data mining algorithms. In this paper, we review the methods that have been used to address the problem of scalability. We focus on general ideas, rather than specific implementations, that can be used to provide a general view of the current approaches for scaling up data mining methods. A taxonomy of the algorithms is proposed, and many examples of different tasks are presented. Among the different techniques used for data mining, we will pay special attention to evolutionary methods, because these methods have been used very successfully in many data mining tasks.

Boosting for class-imbalanced datasets using genetically evolved supervised non-linear projections

“…The main problem of the method described above is the scalability [24]. When we deal with a large dataset, the cost of the RCGA is high.…”

An Approach to Data Reduction for Learning from Big Datasets: Integrating Stacking, Rotation, and Agent Population Learning Techniques

“…Big data, so far, does not have a formal definition, although it is generally accepted that the concept refers to datasets that are too large to be processed using conventional data processing tools and techniques. Contemporary information systems produce data in huge quantities that are difficult to be measured [1]. It means that we already have found ourselves in the "big data era," and the question of how to solve largescale machine learning problems is open and requires a lot of research efforts.…”

Towards Scalable Prototype Selection by Genetic Algorithms with Fast Criteria

“…It is usually assumed in the literature that linear-time algorithms are acceptable for scaling up to large datasets [8]. We also adopt this assumption.…”

“…8 show the results for the SVHN and XM2VTS datasets when using spherical hashing to speed-up the proposed methods, Here we name this version LSH, from Limited Spherical Hashing since we do not use the original procedure. Instead, we use a limited version which uses the dissimilarities instead of the binary codification since the latter increases the classification error, while the use of dissimilarities still provides Accuracy and execution times results when using LSH to speed up the proposed prototype selection methods in SVHN dataset Accuracy and execution times results when using SH to speed up the proposed prototype selection methods in XM2VTS dataset…”