A combined MapReduce-windowing two-level parallel scheme for evolutionary prototype generation

Triguero, Isaac; Peralta, Daniel; Bacardit, Jaume; García, Salvador; Herrera, Francisco

doi:10.1109/cec.2014.6900490

Cited by 6 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reduce phase offers several ways to aggregate the partial instance sets, either by: concatenating all partial results (baseline), filtering noisy prototypes, or merging redundant samples. An extension to this method was proposed in [98]. A second phase of parallelization based on windowing is included in this extension on the mappers side.…”

Section: Instance Reductionmentioning

confidence: 99%

Big data preprocessing: methods and prospects

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Instance Reductionmentioning

confidence: 99%

Big data preprocessing: methods and prospects

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…Current research is typically focused on the use of divide-and-conquer approaches, implemented with big data technologies, to parallelise the execution of instance reduction approaches. We can also find an approximation strategy, called windowing [31], which estimates the fitness value of RS using a random subset of T R at every iteration of the search (this reduces significantly the cost, but could mislead the search). However, the use of more sophisticated surrogate models to reduce the number of evaluations for instance reduction algorithms has been neglected.…”

Section: Computational Cost Of the Objective Functionmentioning

confidence: 99%

A Local Search with a Surrogate Assisted Option for Instance Reduction

Neri

Triguero

2020

Applications of Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

In data mining, instance reduction is a key data pre-processing step that simplifies and cleans raw data, by either selecting or creating new samples, before applying a learning algorithm. This usually yields to a complex large scale and computationally expensive optimisation problem which has been typically tackled by sophisticated population-based metaheuristics. Unlike the recent literature, in order to accomplish this target, this article proposes the use of a simple local search algorithm and its integration with an optional surrogate assisted model. This local search, in accordance with variable decomposition techniques for large scale problems, perturbs an n-dimensional vector along the directions identified by its design variables one by one. Empirical results in 40 small data sets show that, despite its simplicity, the proposed baseline local search on its own is competitive with more complex algorithms representing the state-of-the-art for instance reduction in classification problems. The use of the proposed local surrogate model enables a reduction of the computationally expensive objective function calls with accuracy test results overall comparable with respect to its baseline counterpart.

show abstract

“…This approach is different from the partitioning of the data considered in MapReduce because all the information of the training set is available, but only a part of it is used in each iteration. This technique has already been extended to other evolutionary algorithms used in data mining tasks [15].…”

Section: B a Two-level Parallelization Model For Eusmentioning

confidence: 99%

“…To do so, we consider the MapReduce algorithm, which splits the data in different chunks that are processed in different computing nodes (mappers), such that EUS can be applied concurrently. Moreover, the time required by EUS is further reduced by considering a windowing scheme inside the algorithm [14], [15]. This is specifically designed for the class imbalance problem.…”

Section: Introductionmentioning

confidence: 99%

Evolutionary undersampling for imbalanced big data classification

Triguero¹,

Galar

Vluymans³

et al. 2015

2015 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Classification techniques in the big data scenario are in high demand in a wide variety of applications. The huge increment of available data may limit the applicability of most of the standard techniques. This problem becomes even more difficult when the class distribution is skewed, the topic known as imbalanced big data classification. Evolutionary undersampling techniques have shown to be a very promising solution to deal with the class imbalance problem. However, their practical application is limited to problems with no more than tens of thousands of instances.In this contribution we design a parallel model to enable evolutionary undersampling methods to deal with large-scale problems. To do this, we rely on a MapReduce scheme that distributes the functioning of these kinds of algorithms in a cluster of computing elements. Moreover, we develop a windowing approach for class imbalance data in order to speed up the undersampling process without losing accuracy. In our experiments we test the capabilities of the proposed scheme with several data sets with up to 4 million instances. The results show promising scalability abilities for evolutionary undersampling within the proposed framework.

show abstract

A combined MapReduce-windowing two-level parallel scheme for evolutionary prototype generation

Cited by 6 publications

References 20 publications

Big data preprocessing: methods and prospects

Big data preprocessing: methods and prospects

A Local Search with a Surrogate Assisted Option for Instance Reduction

Evolutionary undersampling for imbalanced big data classification

Contact Info

Product

Resources

About