Bayesian Analysis of Mixtures of Factor Analyzers

Utsugi, Akio; Kumagai, Toru

doi:10.1162/08997660151134299

Cited by 33 publications

(28 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative way of proceeding is to adopt some prior distribution for the D i as in the Bayesian approaches of FokouÃ e and Titterington (2000), Ghahramani and Beal (2000) and Utsugi and Kumagai (2001). The mixture of probabilistic component analyzers (PCAs) model, as proposed by Tipping and Bishop (1997), has form (6) with each D i now having the isotropic structure are given explicitly by an eigenvalue decomposition of the current value of V i .…”

Section: Maximum Likelihood Estimation Of Mixture Of Factor Analyzersmentioning

confidence: 99%

Modelling high-dimensional data by mixtures of factor analyzers

McLachlan

Peel

Bean

2003

Computational Statistics & Data Analysis

249

174

View full text Add to dashboard Cite

We focus on mixtures of factor analyzers from the perspective of a method for model-based density estimation from high-dimensional data, and hence for the clustering of such data. This approach enables a normal mixture model to be ÿtted to a sample of n data points of dimension p, where p is large relative to n. The number of free parameters is controlled through the dimension of the latent factor space. By working in this reduced space, it allows a model for each component-covariance matrix with complexity lying between that of the isotropic and full covariance structure models. We shall illustrate the use of mixtures of factor analyzers in a practical example that considers the clustering of cell lines on the basis of gene expressions from microarray experiments.

show abstract

Section: Maximum Likelihood Estimation Of Mixture Of Factor Analyzersmentioning

confidence: 99%

Modelling high-dimensional data by mixtures of factor analyzers

McLachlan

Peel

Bean

2003

Computational Statistics & Data Analysis

249

174

View full text Add to dashboard Cite

show abstract

“…We believe that a careful study of the limitations noticed so far would lead to better sampling schemes that would then be fully applicable to truly high-dimensional Machine Learning tasks. Finally, we acknowledge the very recent appearance of Utsugi and Kumagai (2001), who set out basic MCMC principles for MFAs along the lines also reported by Fokoué (2000). …”

Section: Discussionmentioning

confidence: 79%

Untitled

Fokoué

Titterington

2003

Machine Learning

View full text Add to dashboard Cite

Abstract. Factor Analysis (FA) is a well established probabilistic approach to unsupervised learning for complex systems involving correlated variables in high-dimensional spaces. FA aims principally to reduce the dimensionality of the data by projecting high-dimensional vectors on to lower-dimensional spaces. However, because of its inherent linearity, the generic FA model is essentially unable to capture data complexity when the input space is nonhomogeneous. A finite Mixture of Factor Analysers (MFA) is a globally nonlinear and therefore more flexible extension of the basic FA model that overcomes the above limitation by combining the local factor analysers of each cluster of the heterogeneous input space. The structure of the MFA model offers the potential to model the density of high-dimensional observations adequately while also allowing both clustering and local dimensionality reduction. Many aspects of the MFA model have recently come under close scrutiny, from both the likelihood-based and the Bayesian perspectives. In this paper, we adopt a Bayesian approach, and more specifically a treatment that bases estimation and inference on the stochastic simulation of the posterior distributions of interest. We first treat the case where the number of mixture components and the number of common factors are known and fixed, and we derive an efficient Markov Chain Monte Carlo (MCMC) algorithm based on Data Augmentation to perform inference and estimation. We also consider the more general setting where there is uncertainty about the dimensionalities of the latent spaces (number of mixture components and number of common factors unknown), and we estimate the complexity of the model by using the sample paths of an ergodic Markov chain obtained through the simulation of a continuous-time stochastic birth-and-death point process. The main strengths of our algorithms are that they are both efficient (our algorithms are all based on familiar and standard distributions that are easy to sample from, and many characteristics of interest are by-products of the same process) and easy to interpret. Moreover, they are straightforward to implement and offer the possibility of assessing the goodness of the results obtained. Experimental results on both artificial and real data reveal that our approach performs well, and can therefore be envisaged as an alternative to the other approaches used for this model.

show abstract

“…1, namely, a mixture of factor analysis, or local FA (including local PCA or local subspaces [1,[32][33][34][35][36]48,[57][58][59][60][61][62][63][64][65]). The implementation can be handled in either of the following two choices:…”

Section: (92)mentioning

confidence: 99%

“…In other words, the role Ψ x applies to a standard local FA or a mixture of FA models [63][64][65][66][67].…”

Section: (92)mentioning

confidence: 99%

“…With η a = η a (ζ, Φ), the de-noise nature of the above new parametrization is also embedded to a Gaussian mixture within a dimension reduced subspace and further to local FA [57][58][59][60][61][62][63][64][65][66][67] as illustrated in the Box 9 . With η y = η y (ε, B), we are further led to a co-dim matrix pair based generalization of temporal FA and structural state space model [68][69][70][71][72][73][74][75], as illustrated in the Boxes 11 and 12 .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Codimensional matrix pairing perspective of BYY harmony learning: hierarchy of bilinear systems, joint decomposition of data-covariance, and applications of network biology

2011

Front. Electr. Electron. Eng. China

View full text Add to dashboard Cite

One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang (BYY) harmony learning from a perspective of problem solving, parameter learning, and model selection. In a complementary role, the paper provides further insights from another perspective that a co-dimensional matrix pair (shortly co-dim matrix pair) forms a building unit and a hierarchy of such building units sets up the BYY system. The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair, which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning. Besides updating typical algorithms of factor analysis (FA), binary FA (BFA), binary matrix factorization (BMF), and nonnegative matrix factorization (NMF) to share such a mechanism, we are also led to (a) a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA (LFA); (b) an alternative formulation of graph Laplacian based linear manifold learning; (c) a codecomposition of data and covariance for learning regularization and data integration; and (d) a co-dim matrix pair based generalization of temporal FA and state space model. Moreover, with help of a co-dim matrix pair in Hadamard product, we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model. Furthermore, we address that these advances provide with new tools for network biology studies, including learning transcriptional regulatory, Protein-Protein Interaction network alignment, and network integration.

show abstract

Bayesian Analysis of Mixtures of Factor Analyzers

Cited by 33 publications

References 7 publications

Modelling high-dimensional data by mixtures of factor analyzers

Modelling high-dimensional data by mixtures of factor analyzers

Untitled

Codimensional matrix pairing perspective of BYY harmony learning: hierarchy of bilinear systems, joint decomposition of data-covariance, and applications of network biology

Contact Info

Product

Resources

About