A Semidefinite Programming Based Search Strategy for Feature Selection with Mutual Information Measure

Abstract

Abstract-Feature subset selection, as a special case of the general subset selection problem, has been the topic of a considerable number of studies due to the growing importance of data-mining applications. In the feature subset selection problem there are two main issues that need to be addressed: (i) Finding an appropriate measure function than can be fairly fast and robustly computed for high-dimensional data. (ii) A search strategy to optimize the measure over the subset space in a reasonable amount of time. In this article mutual information between features and class labels is considered to be the measure function. Two series expansions for mutual information are proposed, and it is shown that most heuristic criteria suggested in the literature are truncated approximations of these expansions. It is well-known that searching the whole subset space is an NP-hard problem. Here, instead of the conventional sequential search algorithms, we suggest a parallel search strategy based on semidefinite programming (SDP) that can search through the subset space in polynomial time. By exploiting the similarities between the proposed algorithm and an instance of the maximumcut problem in graph theory, the approximation ratio of this algorithm is derived and is compared with the approximation ratio of the backward elimination method. The experiments show that it can be misleading to judge the quality of a measure solely based on the classification accuracy, without taking the effect of the non-optimum search strategy into account.

Exploring diagnosis and imaging biomarkers of Parkinson’s disease via iterative canonical correlation analysis based feature selection

“…RLDA can be considered as a special case of robust regression (Huang et al, 2016). As also confirmed in the literature, regression methods can be extended for the challenging classification tasks (Naghibi et al, 2015; Wang et al, 2010; Huang et al, 2011). Nevertheless, the lack of robustness against noise is one of the major drawbacks of most existing regression approaches, especially when the outliers affect the normal distribution of subjects within high-dimensional feature space (Huang et al, 2016).…”

“…In (Senawi et al, 2017), Senawi et al proposed a maximum relevance-minimum multicollinearity (MRmMC) method in which relevant features are measured by correlation characteristics based on conditional variance while redundancy elimination is achieved according to multiple correlation assessment using orthogonal projection scheme. In (Naghibi et al, 2015), Naghibi et al proposed a parallel search strategy on semidefinite programming, which can search through the subset space in polynomial time, with mutual information between features and class labels considered as measure function. In (Zhu et al, 2014), Zhu et al proposed a novel canonical feature selection method, which efficiently integrates the correlation information between structural and functional neuroimaging data into a sparse multi-task learning framework.…”

Estimating Feature-Label Dependence Using Gini Distance Statistics

“…In [62], Sindhwani et al presented a method that maximizes the mutual information between the class labels and classifier output. Naghibi et al [52] considered mutual information between features and class labels in a parallel search algorithm.…”

Use of relevancy and complementary information for discriminatory gene selection from high-dimensional cancer data

“…The 19 activity and/or expression level of a gene affects the synthesis of downstream proteins 20 that dictate the functionality of a cell. Therefore, the properties as well as the 21 expression levels of a particular set of genes are responsible for a particular phenotype 22 such as disease or tissue morphology. Those genes which are able to differentiate 23 between different states (such as normal vs diseased, quiescent vs proliferating, adult vs 24 stem cells, etc.)…”