Online Feature Selection and Its Applications

Abstract

Abstract-Feature selection is an important technique for data mining. Despite its importance, most studies of feature selection are restricted to batch learning. Unlike traditional batch learning methods, online learning represents a promising family of efficient and scalable machine learning algorithms for large-scale applications. Most existing studies of online learning require accessing all the attributes/features of training instances. Such a classical setting is not always appropriate for real-world applications when data instances are of high dimensionality or it is expensive to acquire the full set of attributes/features. To address this limitation, we investigate the problem of Online Feature Selection (OFS) in which an online learner is only allowed to maintain a classifier involved only a small and fixed number of features. The key challenge of Online Feature Selection is how to make accurate prediction for an instance using a small number of active features. This is in contrast to the classical setup of online learning where all the features can be used for prediction. We attempt to tackle this challenge by studying sparsity regularization and truncation techniques. Specifically, this article addresses two different tasks of online feature selection: (1) learning with full input where an learner is allowed to access all the features to decide the subset of active features, and (2) learning with partial input where only a limited number of features is allowed to be accessed for each instance by the learner. We present novel algorithms to solve each of the two problems and give their performance analysis. We evaluate the performance of the proposed algorithms for online feature selection on several public datasets, and demonstrate their applications to real-world problems including image classification in computer vision and microarray gene expression analysis in bioinformatics. The encouraging results of our experiments validate the efficacy and efficiency of the proposed techniques.

Big Data and Its Applications in Smart Real Estate and the Disaster Management Life Cycle: A Systematic Analysis

“…Volume Parallel computing [116] Cloud computing [40] Variety Data integration; deep learning methods; dimensionality reduction [117] Velocity Extreme learning machine (ELM) [118] Online learning [119] Value Knowledge discovery in databases (KDD); data mining technologies [120] Uncertainty and incompleteness Matrix completion [121] AI and machine learning methods are being increasingly integrated in systems dealing with a wide variety of issues related to disasters. This includes disaster prediction, risk assessment, detection, susceptibility mapping, and disaster response activities such as damage assessment after the occurrence of a disaster.…”

A Correlation-Change Based Feature Selection Method for IoT Equipment Anomaly Detection

“…Compared to the other dimension reduction methods, we believe the advantage of our method is that our offline learning procedure shrinks the range of dimension reduction computation. The existing works [17][18][19][20][21][22][23] conduct dimension reduction over all the sensors. However, our method focuses on the dimension reduction for each cluster, i.e., we ignore the possible dimension reduction for the uncorrelated sensor pairs.…”

“…One assumes that the number of features is fixed while the number of data points changes over time [17]. Wang et al [18] proposed an Online Feature Selection (OFS) method, which assumes data instances are sequentially presented, and performs feature selection upon each data instance's arrival. Wu et al [19] presented a simple but smart second-order online feature selection algorithm that is extremely efficient, scalable to large scale and ultra-high dimensionality.…”

Supervised feature selection: A tutorial

“…Feature selection methods for dynamically increasing and incomplete data sets have also been proposed. [69,70] 7 Guidelines for applying feature selection methods…”