Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination

Abstract

Abstract-CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem especially for CP rank. In addition, existing approaches do not take into account uncertainty information of latent factors, as well as missing entries. To address these issues, we formulate CP factorization using a hierarchical probabilistic model and employ a fully Bayesian treatment by incorporating a sparsity-inducing prior over multiple latent factors and the appropriate hyperpriors over all hyperparameters, resulting in automatic rank determination. To learn the model, we develop an efficient deterministic Bayesian inference algorithm, which scales linearly with data size. Our method is characterized as a tuning parameter-free approach, which can effectively infer underlying multilinear factors with a low-rank constraint, while also providing predictive distributions over missing entries. Extensive simulations on synthetic data illustrate the intrinsic capability of our method to recover the ground-truth of CP rank and prevent the overfitting problem, even when a large amount of entries are missing. Moreover, the results from real-world applications, including image inpainting and facial image synthesis, demonstrate that our method outperforms state-of-the-art approaches for both tensor factorization and tensor completion in terms of predictive performance.

Nonlinear least squares updating of the canonical polyadic decomposition

“…Determining when the rank should change, is more difficult, however. See [20] for an extended discussion. …”

Survey on Probabilistic Models of Low-Rank Matrix Factorizations

“…A Bayesian robust tensor factorization [47] was proposed and it is the extension of probabilistic stable robust PCA. And in [48], the CP factorization was formulated by a hierarchical probabilistic model.…”

“…Among them, the TT format is a special case of the HT and the tensor tree structure [33]. The probabilistic models of the Tucker were presented in [34][35][36] and that of the CP were developed in [37][38][39][40][41][42][43][44][45][46][47][48].…”

Removal of EEG artifacts for BCI applications using fully Bayesian tensor completion

“…The posterior distribution of Ψ can be estimated by a deterministic approximate inference under variational Bayesian framework [16,20]. The predictive distribution over missing entries Y Y Y \Ω is then inferred as…”

“…Tensor factorization of incomplete data provides a powerful approach to estimate the latent factors from partially observed entries by typically exploiting a CANDE-COMP/PARAFAC (CP) multilinear model with a predefined rank [14]. Recently, tensor completion has attracted increasing research interest and been successfully applied to visual data analysis [15,16]. In EEG recording, it is actually quite impossible that all of the segments are contaminated by severe artefacts.…”