Bayesian Selection on the Number of Factors in a Factor Analysis Model

Sparse Bayesian infinite factor models

“…Lopes & West (2004) proposed a reversible jump Markov chain Monte Carlo algorithm to allow for uncertainty in the number of factors. Lee & Song (2002) developed a path sampling approach instead. A more recent method infers the number of factors by zeroing a subset of the loading elements using Bayesian variable selection priors (Lucas et al, 2006;Carvalho et al, 2008); see also the 2009 discussion paper from the University of Chicago Booth School of Business by Schnatter and Lopes.…”

Bayesian Model Selection in Factor Analytic Models

“…It is well known that Bayes factors tend to be sensitive to the prior, motivating a rich literature on objective Bayes methods (Berger and Pericchi 1996;Berger and Pericchi 2001). Lee and Song (2002) rely on highly-informative priors in implementing Bayesian model selection for factor analysis, an approach which is only reliable when substantial prior knowledge is available allowing one to concisely guess a narrow range of plausible values for all of the parameters in the model. Such knowledge is often lacking.…”

“…We will refer to this approach as Importance Sampling with Parameter Expansion (IS-PX). Lee and Song (2002) use the path sampling approach of Gelman and Meng (1998) for estimating log Bayes factors. They construct a path using a scalar t∈ [0, 1] to link two models M 0 and M 1 .…”

“…Motivated by this problem, Carvalho et al (2008) proposed an evolutionary search algorithm, which provides a useful approach for searching for good factor models in high dimensions. Lee and Song (2002) developed a method for estimating Bayes factors for selecting the number of factors in a factor analysis model using the idea of path sampling (Gelman and Meng, 1998). They proposed a procedure that is simple to implement and tends to have good performance in terms of accuracy.…”

“…In addition, even if this was not an issue, high variance priors tend to systematically favor small models, so that one will tend to select a one factor model if the prior variance is extremely large. In implementing their approach, Lee and Song (2002) Ghosh and Dunson (2007) proposed an approach for simultaneously addressing the issues of efficient computation and prior specification in latent factor models through the use of a parameter-expansion approach. This approach conveys a dramatic improvement in MCMC efficiency in many cases, while inducing heavy-tailed priors that can be used for robust model selection via the path sampling approach.…”