Incremental relevance sample-feature machine: A fast marginal likelihood maximization approach for joint feature selection and classification

Mohsenzadeh, Yalda; Sheikhzadeh, Hamid; Nazari, Sobhan

doi:10.1016/j.patcog.2016.06.028

Cited by 21 publications

(17 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The presence of 19 noise features in the Waveform dataset increases the hardness of the classification problem. Ideal feature selection methods should select the relevant features (features 1-21) and simultaneously remove the irrelevant noise features (features [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40]. To evaluate the stability and robustness of feature selection of PFCVM LP with noise features, we choose wave 1 vs. wave 2 from the Waveform as the experimental data, which includes 3,345 samples.…”

Section: Waveform Dataset: Stability and Robustness Against Noisementioning

confidence: 99%

“…Feature selection, as a dimensionality reduction technique, has been extensively studied in machine learning and data mining, and various feature selection methods have been proposed [5, 26, 29, 34, 36-38, 40, 47, 48, 50, 52-57]. Feature selection methods can be divided into three groups: filter methods [20,37,38,40], wrapper methods [50], and embedded methods [5,26,29,33,34,36]. Filter methods independently select the subset of features from the classifier learning.…”

Section: Introductionmentioning

confidence: 99%

“…Wrapper methods consider all possible feature subsets and then select a specific subset based on its predictive power. Therefore, the feature selection stage and classification model are separated and independent in the filter and wrapper methods, and the wrapper methods might suffer from high computational complexity especially for high-dimensional data [33]. Embedded methods embed feature selection in the training process, which aims to combine the advantages of the filter and wrapper methods.…”

Section: Introductionmentioning

confidence: 99%

“…To select relevant features, JCFO introduces hierarchical sparseness promoting priors on feature weights and then employs EM and gradient-based methods to optimize the feature weights. In order to simultaneously select relevance samples and features, Mohsenzadeh et al [34] extend the standard RVM and then design the relevance sample feature machine (RSFM) and an incrementally learning version (IRSFM) [33], that scales the basis parameters in RVM to a vector and applies zero-mean Gaussian priors on feature weights to generate sparsity in the feature space. Li and Chen [29] propose an EM algorithm based joint feature selection strategy for PCVM (denoted as PFCVM EM ), in which they add truncated Gaussian priors to features to enable PCVM to jointly select relevant samples and features.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Probabilistic Feature Selection and Classification Vector Machine

Jiang

Chang

Rijke

et al. 2019

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

Sparse Bayesian learning is a state-of-the-art supervised learning algorithm that can choose a subset of relevant samples from the input data and make reliable probabilistic predictions. However, in the presence of highdimensional data with irrelevant features, traditional sparse Bayesian classifiers suffer from performance degradation and low efficiency by failing to eliminate irrelevant features. To tackle this problem, we propose a novel sparse Bayesian embedded feature selection method that adopts truncated Gaussian distributions as both sample and feature priors. The proposed method, called probabilistic feature selection and classification vector machine (PFCVM LP ), is able to simultaneously select relevant features and samples for classification tasks. In order to derive the analytical solutions, Laplace approximation is applied to compute approximate posteriors and marginal likelihoods. Finally, parameters and hyperparameters are optimized by the type-II maximum likelihood method. Experiments on three datasets validate the performance of PFCVM LP along two dimensions: classification performance and effectiveness for feature selection. Finally, we analyze the generalization performance and derive a generalization error bound for PFCVM LP . By tightening the bound, the importance of feature selection is demonstrated.

show abstract

Section: Waveform Dataset: Stability and Robustness Against Noisementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic Feature Selection and Classification Vector Machine

Jiang

Chang

Rijke

et al. 2019

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

show abstract

“…SVM is one of the most popular classifiers used for decoding analysis. 7,9,15,22,23 In spite of using kernel method to increase data separability and its reasonable performance in neural decoding, it remains susceptible to the curse of dimensionality. [24][25][26][27] Researchers have applied both supervised and unsupervised dimensionality reduction methods to map neuroimaging data into a lower feature space prior to the classification phase.…”

Section: Introductionmentioning

confidence: 99%

Enhancing multivariate pattern analysis for magnetoencephalography through relevant sensor selection

Hatamimajoumerd

Talebpour

Mohsenzadeh

2020

Int J Imaging Syst Tech

Self Cite

View full text Add to dashboard Cite

While the popularity of multivariate pattern classification is growing rapidly in magnetoencephalography (MEG) data analysis, the analysis pipelines used by the neuroscience community are still missing some fundamental machinelearning techniques and principles that would increase their effectiveness.Here, we show that MEG decoding accuracy improves significantly with the addition of feature selection methods to the analysis pipeline. We compare one unsupervised and two supervised feature reduction methods in the current study. Our results show that supervised feature selection methods like statistical dependency and mutual information improve decoding performance and attain higher session-to-session reliability compared to unsupervised dimensionality reduction methods like principal component analysis. Furthermore, we demonstrate that the selected sensors in the data related to a visual task at each time point are consistent with the pattern reflecting the sweep of information in the ventral visual pathway. K E Y W O R D Sdimensionality reduction, feature selection, MEG analysis, MEG sensor selection, multivariate pattern analysis, mutual information, statistical dependency

show abstract

Sparse Bayesian learning using TMB (Template Model Builder)

Helgøy,

Skaug,

2024

Stat Comput

View full text Add to dashboard Cite

Sparse Bayesian Learning, and more specifically the Relevance Vector Machine (RVM), can be used in supervised learning for both classification and regression problems. Such methods are particularly useful when applied to big data in order to find a sparse (in weight space) representation of the model. This paper demonstrates that the Template Model Builder (TMB) is an accurate and flexible computational framework for implementation of sparse Bayesian learning methods.The user of TMB is only required to specify the joint likelihood of the weights and the data, while the Laplace approximation of the marginal likelihood is automatically evaluated to numerical precision. This approximation is in turn used to estimate hyperparameters by maximum marginal likelihood. In order to reduce the computational cost of the Laplace approximation we introduce the notion of an “active set” of weights, and we devise an algorithm for dynamically updating this set until convergence, similar to what is done in other RVM type methods. We implement two different methods using TMB; the RVM and the Probabilistic Feature Selection and Classification Vector Machine method, where the latter also performs feature selection. Experiments based on benchmark data show that our TMB implementation performs comparable to that of the original implementation, but at a lower implementation cost. TMB can also calculate model and prediction uncertainty, by including estimation uncertainty from both latent variables and the hyperparameters. In conclusion, we find that TMB is a flexible tool that facilitates implementation and prototyping of sparse Bayesian methods.

show abstract

Incremental relevance sample-feature machine: A fast marginal likelihood maximization approach for joint feature selection and classification

Cited by 21 publications

References 34 publications

Probabilistic Feature Selection and Classification Vector Machine

Probabilistic Feature Selection and Classification Vector Machine

Enhancing multivariate pattern analysis for magnetoencephalography through relevant sensor selection

Sparse Bayesian learning using TMB (Template Model Builder)

Contact Info

Product

Resources

About