Generalized infinite factorization models

Abstract

Summary Factorization models express a statistical object of interest in terms of a collection of simpler objects. For example, a matrix or tensor can be expressed as a sum of rank-one components. However, in practice, it can be challenging to infer the relative impact of the different components as well as the number of components. A popular idea is to include infinitely many components having impact decreasing with the component index. This article is motivated by two limitations of existing methods: (i) the lack of careful consideration of the within component sparsity structure; and (ii) no accommodation for grouped variables and other non-exchangeable structures. We propose a general class of infinite factorization models that address these limitations. Theoretical support is provided, practical gains are shown in simulation studies, and an ecology application focusing on modelling bird species occurrence is discussed.

Powering Research through Innovative Methods for Mixtures in Epidemiology (PRIME) Program: Novel and Expanded Statistical Methods

“…Roy et al included a comparison to Bayesian multi-study factor analysis [52], which can be implemented via the MSFA package [53]. The generalized infinite factor model (GIF-SIS) allows for the inclusion of such information in exploratory FA, while learning the number of factors and the loadings structure flexibly [13]. Schiavon et al [13] compared performance with popular Bayesian FA methods based on the multiplicative gamma process [54], as implemented in the hmsc package [55].…”

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis

“…. , m. In recent work by Schiavon et al [25], the CUSP prior is extended to generalized infinite factorization models, where the factor loadings β ih are allowed to be exact zeros, see also [9]. For illustration, we consider here a finite generalized CUSP prior on the column-specific shrinkage parameter θ h in a finite overfitting model with H ≤ H max , where H max = (m − 1)/2 is equal to the upper bound of Anderson & Rubin [26], ensuring econometric identification.…”

Normalized Latent Measure Factor Models

“…from some probability distribution with support on R + , such as the gamma distribution. We find it more convenient to specify a shrinkage prior on Λ, to automatically select the number of latent factors H. This approach has received considerable attention in Gaussian latent factor models, see, for instance, Bhattacharya and Dunson (2011); Legramanti et al (2020); Schiavon et al (2022). In our example, we consider Λ distributed as a multiplicative gamma process (Bhattacharya and Dunson, 2011),…”

“…In that context, a common practice to recover identifiability is to constrain the matrix Λ to be lower triangular with positive entries on the diagonal (Geweke and Zhou, 2015). More recently, it has been proposed to ignore the identifiability issue and obtain a pointestimate of the posterior distribution either by post-processing the MCMC chains (see Papastamoulis and Ntzoufras, 2022;Poworoznek et al, 2021, and the references therein) or by choosing the maximum a posteriori (Schiavon et al, 2022). In particular, Poworoznek et al (2021) propose to orthogonalize each posterior sample of Λ and then solve the sign ambiguity and label switching via a greedy matching algorithm.…”