Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis

Abstract

SummaryThe widespread use of multi-sensor technology and the emergence of big datasets has highlighted the limitations of standard flat-view matrix models and the necessity to move towards more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift towards models that are essentially polynomial and whose uniqueness, unlike the matrix methods, is guaranteed under very mild and natural conditions. Benefiting from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints that match data properties, and to find more general latent components in the data than matrix-based methods. A comprehensive introduction to tensor decompositions is provided from a signal processing perspective, starting from the algebraic foundations, via basic Canonical Polyadic and Tucker models, through to advanced cause-effect and multi-view data analysis schemes. We show that tensor decompositions enable natural generalizations of some commonly used signal processing paradigms, such as canonical correlation and subspace techniques, signal separation, linear regression, feature extraction and classification. We also cover computational aspects, and point out how ideas from compressed sensing and scientific computing may be used for addressing the otherwise unmanageable storage and manipulation problems associated with big datasets. The concepts are supported by illustrative real world case studies illuminating the benefits of the tensor framework, as efficient and promising tools for modern signal processing, data analysis and machine learning applications; these benefits also extend to vector/matrix data through tensorization.

My Multiway Analysis: From Jan de Leeuw toTWPackand Back

“…It is in this field that there is a great upsurge of applications. Much about these developments can be found in the Handbook of Blind Source Separation (Comon and Jutten 2010, Chapter 13), the review paper by Cichocki, Mandic, De Lathauwer, Zhou, Zhao, Caiafa, and Phan (2015), and the book by Cichocki, Zdunek, Phan, and Amari (2009).…”

A novel deterministic method for large-scale blind source separation

“…A mode-n vector of a tensor is defined by fixing every index except the nth and is a natural extension of the rows and columns of a matrix. The mode-n unfolding of A is a matrix A (n) with the mode-n vectors of A as its columns (see [12,13] for formal definitions). The vectorization of A maps each element a i1i2···i N onto vec(A) j with j = 1 +…”

“…, R. The PD is called canonical (CPD) when R is equal to the rank of A. The CPD is a powerful model for several applications within signal processing, biomedical sciences, computer vision, machine learning and data mining [12,13]. The decomposition is essentially unique, i.e., up to trivial permutations of the rank-1 terms and scalings of the factors in the same term, under rather mild conditions [14][15][16].…”

Flexible fusion of electroencephalography and functional magnetic resonance imaging: Revealing neural-hemodynamic coupling through structured matrix-tensor factorization

“…Traditional BSS methods such as independent component analysis [9] or canonical correlation analysis [11] rely mostly on matrix factorization techniques, which operate on data in a two-way format. Over the past years however, tensor-based techniques which factorize third or higher order arrays have been increasingly used [12], [13], also in EEG-fMRI data analysis [14]- [16]. They offer the advantage that the inherent multidimensional nature of datasets (such as those in brain analytics) is respected, and most importantly, that the factorizations of tensors can be unique under mild conditions, as opposed to matrix factorization [12].…”

“…Over the past years however, tensor-based techniques which factorize third or higher order arrays have been increasingly used [12], [13], also in EEG-fMRI data analysis [14]- [16]. They offer the advantage that the inherent multidimensional nature of datasets (such as those in brain analytics) is respected, and most importantly, that the factorizations of tensors can be unique under mild conditions, as opposed to matrix factorization [12]. However, when applied to recordings of brain activity, these data-driven techniques exploit (by definition) no model of the neurovascular coupling, and might thus neglect prior (approximate) knowledge on the joint 'behavior' of the datasets.…”