Sparse Inverse Covariance Estimates for Hyperspectral Image Classification

Abstract

Abstract-Classification of remotely sensed hyperspectral images calls for a classifier that gracefully handles high-dimensional data, where the amount of samples available for training might be very low relative to the dimension. Even when using simple parametric classifiers such as the Gaussian maximum-likelihood rule, the large number of bands leads to copious amounts of parameters to estimate. Most of these parameters are measures of correlations between features. The covariance structure of a multivariate normal population can be simplified by setting elements of the inverse covariance matrix to zero. Well-known results from time series analysis relates the estimation of the inverse covariance matrix to a sequence of regressions by using the Cholesky decomposition. We observe that discriminant analysis can be performed without inverting the covariance matrix. We propose defining a sparsity pattern on the lower triangular matrix resulting from the Cholesky decomposition, and develop a simple search algorithm for choosing this sparsity. The resulting classifier is used on four different hyperspectral images, and compared with conventional approaches such as support vector machines, with encouraging results.

Improving Hyperspectral Classifiers: The Difference Between Reducing Data Dimensionality and Reducing Classifier Parameter Complexity

“…Constraints such as the assumption that features are uncorrelated, well known as a naïve bayes classifier, reduces the number of parameters to estimate for each distribution down to the dimensionality. We recently [2] proposed an approach for reducing the number of parameters needed to estimate when designing classifiers in high dimensional feature spaces, sparse cholesky triangle inverse covariance (STIC) estimates. The method is based on time series theory regarding the Cholesky decomposition of the inverse covari-…”

“…The search is guided by ten-fold cross-validation (10-CV) as a performance measure. Further details of this approach can be found in [2].…”

“…For decades, the common perception in the literature has been that the solution to this has been to reduce data dimensionality. However, as can be seen from a result by Cover [1], reducing dimensionality increases the risk of making the classification problem more complex.Using the simple GML classifier we compare state of the art dimensionality reduction strategies with a recently proposed strategy for sparsing of parameter estimates in full dimension [2]. Results show that reducing parameter estimation complexity by fitting sparse models in full dimension have a slight edge on the common approaches.…”

The incredible shrinking covariance estimator

“…While shrinkage is the most common regularization scheme, sparse 15,16 and sparse transform [17][18][19][20] methods have also been proposed. To deal with the non-Gaussian nature of most data, both robust 21 and anti-robust 22 estimators have been proposed.…”

Mining brain region connectivity for alzheimer's disease study via sparse inverse covariance estimation

“…These observations in theory and practice motivate the wide use of the sparse inverse covariance estimation. It has been shown to be useful in various applications, including evaluating patterns of association among variables [9], exploration of genetic networks [17], senator voting records analysis [2], hyperspectral image classification [4], and speech recognition [5].…”