The -OMP Algorithm for Feature Selection With Application to Gene Expression Data

Abstract

Feature selection for predictive analytics is the problem of identifying a minimal-size subset of features that is maximally predictive of an outcome of interest. To apply to molecular data, feature selection algorithms need to be scalable to tens of thousands of features. In this paper, we propose γ-OMP, a generalisation of the highly-scalable Orthogonal Matching Pursuit feature selection algorithm. γ-OMP can handle (a) various types of outcomes, such as continuous, binary, nominal, time-to-event, (b) discrete (categorical) features, (c) different statistical-based stopping criteria, (d) several predictive models (e.g., linear or logistic regression), (e) various types of residuals, and (f) different types of association. We compare γ-OMP against LASSO, a prototypical, widely used algorithm for high-dimensional data. On both simulated data and several real gene expression datasets, γ-OMP is on par, or outperforms LASSO in binary classification (case-control data), regression (quantified outcomes), and time-to-event data (censored survival times). γ-OMP is based on simple statistical ideas, it is easy to implement and to extend, and our extensive evaluation shows that it is also effective in bioinformatics analysis settings.

Automated machine learning for genome wide association studies

“…εpilogi is a greedy feature selection algorithm based on the generalization of the Orthogonal Matching Pursuit algorithm ( Pati et al 1993 ) called the γ-OMP ( Tsagris et al 2022 ). γ-OMP generalizes the standard OMP to any type of outcome, any type of predictor feature, metric for measuring residuals, and predictive model used internally by the algorithm.…”

“…using logistic regression) with the selected features and computes the residuals of the model (e.g. deviance residuals or raw residuals) ( Tsagris et al 2022 ). Next, εpilogi selects as the next-best feature to include the one that is mostly correlated with the residuals.…”

“…In this work, we customized JADBio to include a feature selection algorithm named εpilogi (standing for selection in Greek), a variant of γ-OMP ( Tsagris et al 2022 ) that returns multiple feature subsets that are equally predictive. While for prediction purposes all these subsets are equivalent, it is important to inform the user of their presence for knowledge discovery purposes.…”

A comprehensive country-based day-ahead wind power generation forecast model by coupling numerical weather prediction data and CatBoost with feature selection methods for Turkey

“…gOMP is a faster and more flexible implementation of the OMP with the options for using different models and stopping criteria (Tsagris et al, 2020). After standardizing the data, covariances of the response and the predictor variables are calculated and then the feature with the maximum covariance is chosen.…”

“…In other words, the gOMP algorithm continues to select features since the criterion of minimum 2 BIC difference is met. Rfast library has been used for the gOMP implementation (Tsagris et al, 2020).…”