Gradient boosted feature selection

Xu, Zhixiang; Huang, Gao; Weinberger, Kilian Q.; Zheng, Alice X.

doi:10.1145/2623330.2623635

Cited by 148 publications

(64 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The basic idea is to reduce the dimensionality of a large dataset by selecting a subset of representative features without substantial loss of information [5,7,24,39]. This problem has attracted substantial attention in the so-called high-dimensional regime, where it is typically assumed that only a small subset of features are relevant to a response.…”

Section: Related Workmentioning

confidence: 99%

Variable selection in heterogeneous datasets: A truncated-rank sparse linear mixed model with applications to genome-wide association studies

Wang

Aragam

Xing

2018

Methods

View full text Add to dashboard Cite

A fundamental and important challenge in modern datasets of ever increasing dimensionality is variable selection, which has taken on renewed interest recently due to the growth of biological and medical datasets with complex, non-i.i.d. structures. Naïvely applying classical variable selection methods such as the Lasso to such datasets may lead to a large number of false discoveries. Motivated by genome-wide association studies in genetics, we study the problem of variable selection for datasets arising from multiple subpopulations, when this underlying population structure is unknown to the researcher. We propose a unified framework for sparse variable selection that adaptively corrects for population structure via a low-rank linear mixed model. Most importantly, the proposed method does not require prior knowledge of individual relationships in the data and adaptively selects a covariance structure of the correct complexity. Through extensive experiments, we illustrate the effectiveness of this framework over existing methods. Further, we test our method on three different genomic datasets from plants, mice, and humans, and discuss the knowledge we discover with our model.

show abstract

Section: Related Workmentioning

confidence: 99%

Variable selection in heterogeneous datasets: A truncated-rank sparse linear mixed model with applications to genome-wide association studies

Wang

Aragam

Xing

2018

Methods

View full text Add to dashboard Cite

show abstract

“…Recently, Xu et al [57] proposed a gradient boosted feature selection approach that is able to consider predefined group feature structures. However, the proposed approach employs the group structure information for controlled boosting only and, thus, components from the same multi-dimensional feature are, in general, preferred for selection.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised group feature selection for media classification

Zaharieva

Breiteneder

Hudec

2017

Int J Multimed Info Retr

View full text Add to dashboard Cite

The selection of an appropriate feature set is crucial for the efficient analysis of any media collection. In general, feature selection strongly depends on the data and commonly requires expert knowledge and previous experiments in related application scenarios. Current unsupervised feature selection methods usually ignore existing relationships among components of multi-dimensional features (group features) and operate on single feature components. In most applications, features carry little semantics. Thus, it is less relevant if a feature set consists of complete features or a selection of single feature components. However, in some domains, such as content-based audio retrieval, features are designed in a way that they, as a whole, have considerable semantic meaning. The disruption of a group feature in such application scenarios impedes the interpretability of the results. In this paper, we propose an unsupervised group feature selection algorithm based on canonical correlation analysis (CCA). Experiments with different audio and video classification scenarios demonstrate the outstanding performance of the proposed approach and its robustness across different datasets.

show abstract

“…Lasso is a widely used technique in different models, including logistic regression, support vector machines, and deep neural networks [92]. More recently, a feature selection based on gradient boosting [148] was proposed. Another feature selection method is based on the dependency between features and class labels.…”

Section: Fisher Kernel Approach Via Generative Modelsmentioning

confidence: 99%

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Liu

Kirchhoff

2016

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Graph-based semi-supervised learning (SSL) is a widely used semi-supervised learning method in which the labeled data and unlabeled data are jointly represented as a weighted graph, and the information is propagated from the labeled data to the unlabeled data. The key assumption that graph-based SSL makes is that data samples lie on a low dimensional manifold, where samples that are close to each other are expected to have the same class label. More importantly, by exploiting the relationship between training and test samples, graph-based SSL implicitly adapts to the test data.In this thesis, we address several key challenges in applying graph-based SSL to acoustic modeling. We first investigate and compare several state-of-the-art graph-based SSL algorithms on a benchmark dataset. In addition, we propose novel graph construction methods that allow graph-based SSL to handle variable-length input features. We next investigate the efficacy of graph-based SSL in context of a fully-fledged DNN-based ASR system. We compare two different integration frameworks for graph-based learning. First, we propose a lattice-based late integration framework that combines graph-based SSL with the DNN-based acoustic modeling and evaluate the framework on continuous word recognition tasks. Second, we propose an early integration framework using neural graph embeddings and compare two different neural graph embedding features that capture the information of the manifold at different levels. The embedding features are used as input to a DNN system and are shown to outperform the conventional acoustic feature inputs on several medium-to-large vocabulary conversational speech recognition tasks.

show abstract

Gradient boosted feature selection

Cited by 148 publications

References 31 publications

Variable selection in heterogeneous datasets: A truncated-rank sparse linear mixed model with applications to genome-wide association studies

Variable selection in heterogeneous datasets: A truncated-rank sparse linear mixed model with applications to genome-wide association studies

Unsupervised group feature selection for media classification

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Contact Info

Product

Resources

About