Regularized matrix data clustering and its application to image analysis

Abstract

We propose a novel regularized mixture model for clustering matrix-valued data. The proposed method assumes a separable covariance structure for each cluster and imposes a sparsity structure (e.g., low rankness, spatial sparsity) for the mean signal of each cluster. We formulate the problem as a finite mixture model of matrix-normal distributions with regularization terms, and then develop an EM-type of algorithm for efficient computation. In theory, we show that the proposed estimators are strongly consistent for various choices of penalty functions. Simulation and two applications on brain signal studies confirm the excellent performance of the proposed method including a better prediction accuracy than the competitors and the scientific interpretability of the solution.

A Penalized Matrix Normal Mixture Model for Clustering Matrix Data

“…Ref. [9] suggested a penalized matrix normal mixture model by imposing a penalty for the mean matrix. This method can capture the row-wise and column-wise correlation simultaneously and has the ability to find the sparsity nature that is inherent in the signals and images.…”

“…This method can capture the row-wise and column-wise correlation simultaneously and has the ability to find the sparsity nature that is inherent in the signals and images. However, the approach of [9] shows weakness for high dimension/low sample size data because it cannot reduce the number of precision parameters to estimate.…”

“…In this paper, the work of [9] is extended by further regularizing the precision matrix as in the Gaussian graphical model, to cope better with the curse of dimensionality for large size images. Specifically, we propose a penalized matrix normal mixture model with lasso-type penalty terms in both mean and covariance matrices, and then an expectation maximization algorithm is developed to estimate the parameters.…”

“…(1) Since the data type for the Gaussian mixture model is a vector and K-means and spectral clustering methods use dissimilarity and similarity between the observations as input, these methods do not have the modeling ability for the row-wise/columnwise covariance structure of matrix data (images). On the other hand, both [9] and the proposed model can capture the row-wise and column-wise correlation of image pixels simultaneously by estimating the row covariance U and column covariance V, where the covariance matrix Σ is the Kronecker product of U and V. This capability of modeling separable row-wise/column-wise covariance structure helps for better clustering of image data. (2) Because the proposed model is a model-based clustering method, it has the sound mathematical basis and the interpretability of results with the estimated parameters, whereas centroid-based and similarity-based methods do not.…”

“…Given p × q, random matrix data X, and the number of clusters G, the conventional Gaussian mixture model needs to estimate a total of Gpq(pq + 1)/2 + Gpq + G − 1 parameters because it converts the data into a vector of pq dimension. The method [9] is regularized on the mean matrix, and thus it needs to estimate Gp(p + 1)/2 + Gq(q + 1)/2 + Gpq + G − 1 − r parameters, where r is the number of the mean parameters values that are shrunken to zero. Since the proposed method is regularized further on the precision matrix, that is, on the row and column covariances, U and V, with shrunken elements, it needs to estimate…”