$\ell_P$ Norm Independently Interpretable Regularization Based Sparse Coding for Highly Correlated Data

Abstract

Sparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, is a well-established signal processing methodology and has good efficiency in various areas. The varying sparse constraint can influence the performances of sparse coding algorithms greatly. However, commonly used sparse regularization may not be robust in high-coherence condition. In this paper, inspired from independently interpretable lasso (IILasso), which considers the coherence of sensing matrix columns in constraint to implement the strategy of selecting uncorrelated variables, we propose a new regularization by introducing p norm (0 < p < 1) into the regularization part of IILasso. The new regularization can efficiently enhance the performances in obtaining sparse and accurate coefficient. To solve the optimization problem with the new regularization, we propose to use the coordinate descent algorithm with weighted 1 norm, named independently interpretable weighted lasso (IIWLasso), and the proximal operator, named independently interpretable iterative shrinkage thresholding algorithm (II-ISTA) and independently interpretable proximal operator for 2 3 norm regularization (II2/3PO). We present synthetic data experiments and gene expression data experiments to validate the performance of our proposed algorithms. The experiment results show that all independently interpretable algorithms can perform better than their original ones in different coherence conditions. Among them, IIWLasso can obtain relatively best performance both in relative norm error and support error of synthetic data and misclassification error of tenfold cross-validating gene expression data. INDEX TERMS Sparse coding, highly correlated data, p norm, weighted 1 norm, independently interpretable weighted lasso.

Correlated Feature Selection with Extended Exclusive Group Lasso

“…Although the properties of exclusive group Lasso have been investigated in several studies, these have focused either on examples where group structure was known, or where the informative features were uncorrelated, or where an proper group number was required (Kong et al, 2014;Huai et al, 2018;Manabe et al, 2018;Huang & Liu, 2018;Ming et al, 2019;Zhao et al, 2019). In this paper we examine the performance of exclusive group Lasso in the accurate selection of synthetic and real-world correlated features, and introduce new methods for relaxing its limitations.…”