Two Expectation-Maximization algorithms for Boolean Factor Analysis

Frolov, Alexander A.; Húsek, Dušan; Polyakov, Pavel

doi:10.1016/j.neucom.2012.02.055

Cited by 8 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [17] the authors use a Bayesian framework to consider the BMF problem as a maximum log likelihood problem and use a message passing procedure to approximate the MAP assignment. Along the line with using the statistical procedures, the authors in [18]- [20] proposed algorithms based on expectation maximization to solve the Boolean factor analysis problem. The extracted factors are thought to represent the hidden cause for the observed binary data.…”

Section: Related Workmentioning

confidence: 99%

Boolean Matrix Factorization via Nonnegative Auxiliary Optimization

et al. 2021

View full text Add to dashboard Cite

A novel approach to Boolean matrix factorization (BMF) is presented. Instead of solving the BMF problem directly, this approach solves a nonnegative optimization problem with an additional constraint over an auxiliary matrix whose Boolean structure is identical to the initial Boolean data. This additional auxiliary matrix constraint forces the support of the NMF solution to adhere to that of a BMF solution. The solution of the nonnegative auxiliary optimization problem is thresholded to provide a solution for the BMF problem. We provide the proofs for the equivalencies of the two solution spaces under the existence of an exact solution. Moreover, the nonincreasing property of the algorithm is also proven. Experiments on synthetic and real datasets are conducted to show the effectiveness and complexity of the algorithm compared to other current methods. INDEX TERMS Boolean matrix factorization, nonnegative matrix factorization

show abstract

Section: Related Workmentioning

confidence: 99%

Boolean Matrix Factorization via Nonnegative Auxiliary Optimization

et al. 2021

View full text Add to dashboard Cite

show abstract

“…To solve the above inference problem, they introduce a graphical model for the posterior and use belief propagation to find the matrices B and C which maximize the posterior. Frolov et al (2014) and Liang and Lu (2019) used a similar generative model and expectation maximization instead of belief propagation. Rukat et al (2017a) used a Metropolised Gibbs sampler for the posterior inference.…”

Section: Continuous Optimizationmentioning

confidence: 99%

Recent Developments in Boolean Matrix Factorization

Miettinen

Neumann

2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

The goal of Boolean Matrix Factorization (BMF) is to approximate a given binary matrix as the product of two low-rank binary factor matrices, where the product of the factor matrices is computed under the Boolean algebra. While the problem is computationally hard, it is also attractive because the binary nature of the factor matrices makes them highly interpretable. In the last decade, BMF has received a considerable amount of attention in the data mining and formal concept analysis communities and, more recently, the machine learning and the theory communities also started studying BMF. In this survey, we give a concise summary of the efforts of all of these communities and raise some open questions which in our opinion require further investigation.

show abstract

“…Similarly, inference and learning for sparse coding models that replace the linear combination by nonlinear ones have been investigated. Hidden causes models with nonlinearly interacting signal sources include the noisy-or combination rule [ 25 – 30 ], exclusive causes [ 31 ] or a maximum superposition [ 19 , 20 , 32 ]. Also a combination of linear superposition followed by a sigmoidal nonlinearity (post-linear nonlinearities) have been investigated (nonlinear ICA [ 33 ], sigmoid belief networks [ 34 ]).…”

Section: Model: Nonlinear Spike-and-slab Sparse Codingmentioning

confidence: 99%

Nonlinear Spike-And-Slab Sparse Coding for Interpretable Image Encoding

et al. 2015

View full text Add to dashboard Cite

Sparse coding is a popular approach to model natural images but has faced two main challenges: modelling low-level image components (such as edge-like structures and their occlusions) and modelling varying pixel intensities. Traditionally, images are modelled as a sparse linear superposition of dictionary elements, where the probabilistic view of this problem is that the coefficients follow a Laplace or Cauchy prior distribution. We propose a novel model that instead uses a spike-and-slab prior and nonlinear combination of components. With the prior, our model can easily represent exact zeros for e.g. the absence of an image component, such as an edge, and a distribution over non-zero pixel intensities. With the nonlinearity (the nonlinear max combination rule), the idea is to target occlusions; dictionary elements correspond to image components that can occlude each other. There are major consequences of the model assumptions made by both (non)linear approaches, thus the main goal of this paper is to isolate and highlight differences between them. Parameter optimization is analytically and computationally intractable in our model, thus as a main contribution we design an exact Gibbs sampler for efficient inference which we can apply to higher dimensional data using latent variable preselection. Results on natural and artificial occlusion-rich data with controlled forms of sparse structure show that our model can extract a sparse set of edge-like components that closely match the generating process, which we refer to as interpretable components. Furthermore, the sparseness of the solution closely follows the ground-truth number of components/edges in the images. The linear model did not learn such edge-like components with any level of sparsity. This suggests that our model can adaptively well-approximate and characterize the meaningful generation process.

show abstract

Two Expectation-Maximization algorithms for Boolean Factor Analysis

Cited by 8 publications

References 31 publications

Boolean Matrix Factorization via Nonnegative Auxiliary Optimization

Boolean Matrix Factorization via Nonnegative Auxiliary Optimization

Recent Developments in Boolean Matrix Factorization

Nonlinear Spike-And-Slab Sparse Coding for Interpretable Image Encoding

Contact Info

Product

Resources

About