Distributed Feature Selection for Efficient Economic Big Data Analysis

Abstract

Abstract-With the rapidly increasing popularity of economic activities, a large amount of economic data is being collected. Although such data offers super opportunities for economic analysis, its low-quality, high-dimensionality and huge-volume pose great challenges on efficient analysis of economic big data. The existing methods have primarily analyzed economic data from the perspective of econometrics, which involves limited indicators and demands prior knowledge of economists. When embracing large varieties of economic factors, these methods tend to yield unsatisfactory performance. To address the challenges, this paper presents a new framework for efficient analysis of high-dimensional economic big data based on innovative distributed feature selection. Specifically, the framework combines the methods of economic feature selection and econometric model construction to reveal the hidden patterns for economic development. The functionality rests on three pillars: (i) novel data pre-processing techniques to prepare high-quality economic data, (ii) an innovative distributed feature identification solution to locate important and representative economic indicators from multidimensional data sets, and (iii) new econometric models to capture the hidden patterns for economic development. The experimental results on the economic data collected in Dalian, China, demonstrate that our proposed framework and methods have superior performance in analyzing enormous economic data.

Opportunities and Challenges of Feature Selection Methods for High Dimensional Data: A Review

“…Zhao et al [52] introduced a new technique for an efficient analysis of the economic bigdata. First, the technique preprocesses the data and then it finds the best feature subsets as the economic indicators.…”

Feature Selection using Stochastic Diffusion Search Algorithm in Big Data Analysis

“…In these experiments, there were 67 million instances and 2000 attributes thus proving this to be a framework that was well-suited in dealing with problems in Big Data. Zhao et al, [11] accessed a new context that had an efficient analysis of high dimensional feature selection that was original and distributed. This outline had a combination of variations in economic selection of features and the construction of econometric models that show the hidden patterns in economic development.…”

MRCS: Map Reduce based Algorithm for Identifying Important Features from Big Data using Chi-Square Test

“…Liang Zhao et al, [5] addressed the challenges of efficient analyzing of the high dimensional of economic data based on distributed selection of feature. The goal is to extract the features for generating rules from economic big data.…”