Feature selection using Deep Neural Networks

Abstract

Feature descriptors involved in video processing are generally high dimensional in nature. Even though the extracted features are high dimensional, many a times the task at hand depends only on a small subset of these features. For example, if two actions like running and walking have to be identified, extracting features related to the leg movement of the person is enough. Since, this subset is not known apriori, we tend to use all the features, irrespective of the complexity of the task at hand. Selecting task-aware features may not only improve the efficiency but also the accuracy of the system. In this work, we propose a supervised approach for task-aware selection of features using Deep Neural Networks (DNN) in the context of action recognition. The activation potentials contributed by each of the individual input dimensions at the first hidden layer are used for selecting the most appropriate features. The selected features are found to give better classification performance than the original high-dimensional features. It is also shown that the classification performance of the proposed feature selection technique is superior to the low-dimensional representation obtained by principal component analysis (PCA).

Phylogenetic convolutional neural networks in metagenomics

“…Results are promising, both in terms of learning performance and biomarkers detection Extensions of the Ph-CNN architecture are addressing the testing of different tree distances, optimization of neighbours detection and of the number of Phylo-Conv layers. Further, different feature selection algorithms, either generic or DLspecific can be adopted [53,54,55]. Improvements are expected on the transfer learning and domain adaptation procedures, such as learning on synthetic data and testing on metagenomics, and applying to larger datasets.…”

AFS: An Attention-Based Mechanism for Supervised Feature Selection

“…As the network weights are directly used as the feature weights, it cannot handle situations where inputs have outliers or noise. Towards this end, Roy et al(2015) use the activation potentials contributed by each of the individual input dimensions, as the metric for feature selection. However, this work relies on the specific DNN structure and the ReLU activation function which might not be so suitable in many learning tasks.…”

“… RF(Random Forest), a tree-based feature selection method provided by scikit-learn package.  Roy et al (2015): a DNN-based feature selection method, reproduced according to the paper Parameter Settings. Model parameters are initialized with truncated normal distribution with a mean of 0 and standard deviation of 0.1.…”

Identifying factors associated with opioid cessation in a biracial sample using machine learning

“…Several methods have been proposed for feature selection using DNN, with the focus on reducing input dimensionality, such as sparse one-to-one, dropout feature ranking, and activation potential based (15)(16)(17). We used the activation potential based method because of its proven performance in reducing the number of features, to obviate application of another filtering method coupled with DNN, and its simplicity and intuitiveness for selecting the number of important variables.…”

“…We used the activation potential based method because of its proven performance in reducing the number of features, to obviate application of another filtering method coupled with DNN, and its simplicity and intuitiveness for selecting the number of important variables. Feature selection was performed according to the method proposed by Roy et.al (17). Briefly, we computed the activation potential of each input feature connected to each of the hidden nodes in the first layer before applying Relu.…”