Bayesian Nonparametric Models for Multiway Data Analysis

Abstract

Tensor decomposition is a powerful computational tool for multiway data analysis. Many popular tensor decomposition approaches-such as the Tucker decomposition and CANDECOMP/PARAFAC (CP)-amount to multi-linear factorization. They are insufficient to model (i) complex interactions between data entities, (ii) various data types (e.g., missing data and binary data), and (iii) noisy observations and outliers. To address these issues, we propose tensor-variate latent nonparametric Bayesian models for multiway data analysis. We name these models InfTucker. These new models essentially conduct Tucker decomposition in an infinite feature space. Unlike classical tensor decomposition models, our new approaches handle both continuous and binary data in a probabilistic framework. Unlike previous Bayesian models on matrices and tensors, our models are based on latent Gaussian or t processes with nonlinear covariance functions. Moreover, on network data, our models reduce to nonparametric stochastic blockmodels and can be used to discover latent groups and predict missing interactions. To learn the models efficiently from data, we develop a variational inference technique and explore properties of the Kronecker product for computational efficiency. Compared with a classical variational implementation, this technique reduces both time and space complexities by several orders of magnitude. On real multiway and network data, our new models achieved significantly higher prediction accuracy than state-of-art tensor decomposition methods and blockmodels.

Survey on Probabilistic Models of Low-Rank Matrix Factorizations

“…Tucker decomposition and CP decomposition are two popular tensor decomposition approaches. The probabilistic Tucker decomposition models mainly include probabilistic Tucker decomposition [34], exponential family tensor factorization [35] and InfTucker model [36]. Probabilistic Tucker decomposition was closely related to probabilistic PCA.…”

“…Among them, the TT format is a special case of the HT and the tensor tree structure [33]. The probabilistic models of the Tucker were presented in [34][35][36] and that of the CP were developed in [37][38][39][40][41][42][43][44][45][46][47][48].…”

Tensor-Based Nonlinear Classifier for High-Order Data Analysis

“…Tensor subspace learning methods, such as HOSVD, Tucker decomposition and CANDECOMP [3], MPCA [4] and probabilistic decompositions [5,6,7] have been proposed to tackle the dimensionality problem. These methods project the raw data to a lower dimensional space, in which the projected data can be considered as highly descriptive features of the raw information.…”

Recursive Variational Bayesian Inference to Simultaneous Registration and Fusion

“…VB has low computational costs in comparison with the sampling methods, such as Markov Chain Monte Carlo (MCMC). It has widespread use in machine learning, signal processing and many other fields, such as the state space model [10,11], time series [12,13], Mixture Models [14], filter [15][16][17], image [18][19][20], communication [21,22], speech recognition [23] and graphical models [24,25].…”