Joint Independent Subspace Analysis: A Quasi-Newton Algorithm

Abstract

In this paper, we present a quasi-Newton (QN) algorithm for joint independent subspace analysis (JISA). JISA is a recently proposed generalization of independent vector analysis (IVA). JISA extends classical blind source separation (BSS) to jointly resolve several BSS problems by exploiting statistical dependence between latent sources across mixtures, as well as relaxing the assumption of statistical independence within each mixture. Algebraically, JISA based on second-order statistics amounts to coupled block diagonalization of a set of covariance and crosscovariance matrices, as well as block diagonalization of a single permuted covariance matrix. The proposed QN algorithm achieves asymptotically the minimal mean square error (MMSE) in the separation of multidimensional Gaussian components. Numerical experiments demonstrate convergence and source separation properties of the proposed algorithm.

Blind Source Separation for Unimodal and Multimodal Brain Networks: A Unifying Framework for Subspace Modeling

“…We propose a novel taxonomy to define four general BSS subproblems, as follows: Single dataset unidimensional (SDU): x consists of a single dataset whose sources are not grouped, e.g., independent component analysis (ICA) [19]–[21] and second order blind identification (SOBI) [22], [23], as in section III-A1;Multiple dataset unidimensional (MDU): x consists of one or more datasets but no multidimensional sources occur within any of the datasets, although multidimensional sources containing a single source from each dataset may occur, e.g., canonical correlation analysis (CCA) [24], partial least squares (PLS) [25], and independent vector analysis (IVA) [26], [27], discussed in III-A2;Single dataset multidimensional (SDM): x consists of a single dataset with one or more multidimensional sources, e.g., multidimensional independent component analysis (MICA) [28], [29] and independent subspace analysis (ISA) [30], [31], discussed in III-A3;Multiple dataset multidimensional (MDM): x contains one or more datasets, each containing one or more multidimensional sources that may group further with single or multidimensional sources from the remaining datasets, e.g., multidataset independent subspace analysis (MISA) [32], [33] and joint independent subspace analysis (JISA) [34], discussed in III-A4.…”

“…Multiple dataset multidimensional (MDM): x contains one or more datasets, each containing one or more multidimensional sources that may group further with single or multidimensional sources from the remaining datasets, e.g., multidataset independent subspace analysis (MISA) [32], [33] and joint independent subspace analysis (JISA) [34], discussed in III-A4.…”

“…Thus, $pfalse(\mathrm{bolds}false)=true{\prod }_{k=1}^{K}pfalse({\mathrm{s}}_{k}false)$, and minimization of (6) with user-defined source groupings for s k follows. Variations include models that either exploit a) only SOS, like joint independent subspace analysis (JISA) [34], b) only HOS, like multidataset independent subspace analysis (MISA) [32], [33] (emphasizing uncorrelation within subspaces), or c) both, like MISA with SOS [137]. …”

Joint, Partially-joint, and Individual Independent Component Analysis in Multi-Subject fMRI Data

“…Hereafter, this source model is referred to as joint/partiallyjoint/individual MDM (JpJI-MDM). [1] -Independent component analysis (ICA) [1] -Second order blind identification (SOBI) methods [1] Multiple dataset unidimensional (MDU) -K > 1 -Independent sources in each dataset -Exactly one dependent source between datasets -Canonical correlation analysis (CCA) [5] -Common feature analysis method [6] -Cross cumulant tensor block diagonalization [7] -Group information guided ICA (GIG-ICA) [8] -Independent vector analysis (IVA) [9] Single dataset multi-dimensional (SDM) -K = 1 -One or more group of sources -Dependent sources in each group -Multidimensional independent component analysis (MICA) [10] -Independent subspace analysis (ISA) [11] Multiple dataset multi-dimensional (MDM) -K > 1 -One or more group of sources in each dataset -Dependent sources in each group -Joint group of sources across all, some or just one of datasets -Joint and individual variation explained (JIVE) [12] -Common orthogonal basis extraction (COBE) and common nonnegative features extraction (CNFE) [3] -Joint/Individual Thin ICA (JI-ThICA) [4] -Multi-dataset independent subspace analysis (MISA) [13] -Joint independent subspace analysis (JISA) [14] The JpJI-MDM source model has wide applications in many studies analyzing multi-subject recordings from healthy and disease subjects. In these studies, all subjects may have some joint sources indicating common conditions of all subjects, some partially-joint sources which depend only on the conditions of the group of disease subjects (or healthy subjects), and some individual sources which are appeared due to the independent conditions of each subject.…”

Joint Independent Subspace Analysis Using Second-Order Statistics

“…The JISA model, which is the core of this paper, and a SOS-based relative gradient (RG) algorithm that achieves the optimal separation in the presence of real Gaussian data, were first presented in [6]. A Newton-based algorithm that is based on the error analysis in this paper has recently been presented in [39]. A gradient algorithm that performs JISA based on the multivariate Laplace distribution has recently been proposed in [40].…”