Sparse Semi-supervised Heterogeneous Interbattery Bayesian Analysis

Abstract

The Bayesian approach to feature extraction, known as factor analysis (FA), has been widely studied in machine learning to obtain a latent representation of the data. An adequate selection of the probabilities and priors of these bayesian models allows the model to better adapt to the data nature (i.e. heterogeneity, sparsity), obtaining a more representative latent space.The objective of this article is to propose a general FA framework capable of modelling any problem. To do so, we start from the Bayesian Inter-Battery Factor Analysis (BIBFA) model, enhancing it with new functionalities to be able to work with heterogeneous data, include feature selection, and handle missing values as well as semisupervised problems.The performance of the proposed model, Sparse Semi-supervised Heterogeneous Interbattery Bayesian Analysis (SSHIBA) has been tested on 4 different scenarios to evaluate each one of its novelties, showing not only a great versatility and an interpretability gain, but also outperforming most of the state-of-the-art algorithms.

Bayesian Sparse Factor Analysis with Kernelized Observations

“…We propose a novel probabilistic latent variable model to generate kernel relationships, instead of data observations, based on a linear generative model. We introduce this model using the Bayesian inter-battery factor analysis approach proposed in [14] to show its capabilities to efficiently face semisupervised heterogeneous multi-view problems combining linear and non-linear data representations. Besides, we extend the model formulation to provide the automatic selection of SVs, obtaining scalable solutions, as well as include an ARD prior over the kernel to obtain feature selection capabilities.…”

“…1 Whereas kernel LVMs obtain this latent representation as a linear combination, by some dual variables, of the kernel representation of the n-th data, here we propose to reformulate this idea from a generative point of view. In particular, we start from the SSHIBA algorithm formulation [14] and consider that there exist some latent variables z n,: ∼ N (0, I Kc ) which are linearly combined with a set of dual variables A (m) ∈ R N ×Kc to generate a kernel vector, k (m) n,: , as:…”

“…N,: )]. The dual variable matrix A (m) plays the role of the linear projection matrix and it is defined using the same structured ARD prior considered in both [13] and [14]. Namely, an ARD prior that promotes that full rows of this matrix are cancelled, i.e.…”

“…Among their many variants, such as Probabilistic PCA [10], Supervised PCA [11], Bayesian Factor Regression [12] or Bayesian CCA [13], Inter-Battery FA models [13] stand out for their capability of handling not only latent variables associated to the common information among all the views, but also for being able to model the intra-view information. This model has been recently extended in [14], named as Sparse Semi-supervised Inter-Battery Bayesian Analysis (SSHIBA), to incorporate both missing attributes, feature sparsity/selection, and the ability to handle heterogeneous data such as categorical or multidimensional binary data.…”

“…sample latent representations and project on an N -dimensional space (being N the number of points) using a weight matrix representing the dual parameters. We apply this trick over the SSHIBA formulation [14] to exploit their functionalities over this kernelized formulation. Thanks to that, we can efficiently face semisupervised heterogeneous multi-view problems combining linear and non-linear data representations; in this way, one can combine kernelized views to deal with non-linear relationships with linearly kernelized to work with high dimensional problems.…”