Evolutionary Feature Selection for Big Data Classification: A MapReduce Approach

Peralta, Daniel; Río, Sara del; Ramírez-Gallego, Sergio; Triguero, Isaac; Benitez, Jose M.; Herrera, Francisco

doi:10.1155/2015/246139

Cited by 120 publications

(62 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Each row of this matrix shows the sequence of dataset samples on the machine corresponding to that row. Each column of this matrix shows the sum of characteristics in big dataset sample classification or prediction process.…”

Section: Proposed Methodologymentioning

confidence: 99%

A hybrid multi‐objective firefly and simulated annealing based algorithm for big data classification

Devi¹,

Sabrigiriraj

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Efficient management of big data becomes challenging in recent decades. Online Feature Selection (OFS) is one type of online learning in contrast to batch learning, allowing a classifier to have small and fixed number of features. The major aim of this work is to introduce an OFS algorithm supported on meta‐heuristic algorithm that exploits the MapReduce paradigm. A novel Hybrid Multi‐Objective Firefly and Simulated Annealing (HMOFSA) algorithm is proposed to select optimal set of features. Therefore, as a first step, the original big dataset is decomposed into blocks of examples in the map phase. Subsequently, HMOFSA algorithm is employed to choose the selected features from examples. After that, the attained partial outcomes will be combined into a final vector of features in the reduce phase and evaluated using Kernel Support Vector Machine (KSVM) classifier. The mentioned OFS approach is analyzed with the help of the well‐known classifiers (Logistic Regression, KSVM and Naïve Bayes) developed within the Spark framework. Experiments were conducted on big datasets, containing 66 million samples and 2000 attributes that confirm the proficiency of proposed work. The proposed KSVM classifier results are measured in terms of the metrics like Precision, Recall, Geometric‐mean (G‐mean), F‐measure, and accuracy.

show abstract

Section: Proposed Methodologymentioning

confidence: 99%

A hybrid multi‐objective firefly and simulated annealing based algorithm for big data classification

Devi¹,

Sabrigiriraj

2018

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…• Peralta et al [28] used the MapReduce model to implement a wrapperbased evolutionary search FS method. The dataset was split by instances and the FS method was applied to each resulting subset.…”

Section: Background and Related Workmentioning

confidence: 99%

Distributed correlation-based feature selection in spark

Palma-Mendoza

de‐Marcos

Rodríguez

et al. 2019

Information Sciences

View full text Add to dashboard Cite

Feature selection (FS) is a key preprocessing step in data mining. CFS (Correlation-Based Feature Selection) is an FS algorithm that has been successfully applied to classification problems in many domains. We describe Distributed CFS (DiCFS) as a completely redesigned, scalable, parallel and distributed version of the CFS algorithm, capable of dealing with the large volumes of data typical of big data applications. Two versions of the algorithm were implemented and compared using the Apache Spark cluster computing model, currently gaining popularity due to its much faster processing times than Hadoop's MapReduce model. We tested our algorithms on four publicly available datasets, each consisting of a large number of instances and two also consisting of a large number of features. The results show that our algorithms were superior in terms of both time-efficiency and scalability. In leveraging a computer cluster, they were able to handle larger datasets than the non-distributed WEKA version while maintaining the quality of the results, i.e., exactly the same features were returned by our algorithms when compared to the original algorithm available in WEKA.

show abstract

“…For this reason, several approaches have been proposed to enable data reduction techniques to tackle big datasets based on Hadoop MapReduce. Concretely, based on k-NN, we can find an approach in [24] to perform feature selection on big datasets using the k-NN rule within a evolutionary mode. And in [25], a framework named MRPR was designed to enable instance reduction techniques to be applied on big datasets.…”

Section: B Data Reduction With the K-nn Algorithmmentioning

confidence: 99%

From Big Data to Smart Data with the K-Nearest Neighbours Algorithm

Triguero

Maillo

Luengo

et al. 2016

2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and I

Self Cite

View full text Add to dashboard Cite

Abstract-The k-nearest neighbours algorithm is one of the most widely used data mining models because of its simplicity and accurate results. However, when it comes to deal with big datasets, with potentially noisy and missing information, this technique becomes ineffective and inefficient. Due to its drawbacks to tackle large amounts of imperfect data, plenty of research has aimed at improving this algorithm by means of data preprocessing techniques. These weaknesses have turned out as strengths and the k-nearest neighbours rule has become a core model to actually detect and correct imperfect data, eliminating noisy and redundant data, as well as correcting missing values.In this work, we delve into the role of the k nearest neighbour algorithm to come up with smart data from big datasets. We analyse how this model is affected by the big data problem, but at the same time, how it can be used to transform raw data into useful data. Concretely, we discuss the benefits of recent big data technologies (Hadoop and Spark) to enable this model to address large amounts of data, as well as the usefulness of prototype reduction and missing values imputation techniques based on it. As a result, guidelines on the use of the k-nearest neighbour to obtain Smart data are provided and new potential research trends are drawn.

show abstract

Evolutionary Feature Selection for Big Data Classification: A MapReduce Approach

Cited by 120 publications

References 30 publications

A hybrid multi‐objective firefly and simulated annealing based algorithm for big data classification

A hybrid multi‐objective firefly and simulated annealing based algorithm for big data classification

Distributed correlation-based feature selection in spark

From Big Data to Smart Data with the K-Nearest Neighbours Algorithm

Contact Info

Product

Resources

About