Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

Kim, Jingu; He, Yunlong; Park, Haesun

doi:10.1007/s10898-013-0035-4

Cited by 319 publications

(276 citation statements)

References 70 publications

Supporting

Mentioning

274

Contrasting

Order By: Relevance

“…Kim, He and Park [13] prove that equation (6) satisfies the formulation of BCD method. Equation (6) when extended with the matrix M becomes equation (8).…”

Section: B Bounded Matrix Low Rank Approximationmentioning

confidence: 94%

See 1 more Smart Citation

Bounded Matrix Low Rank Approximation

Kannan¹,

Ishteva²,

Drake³

et al. 2015

Signals and Communication Technology

Self Cite

View full text Add to dashboard Cite

Abstract-Matrix lower rank approximations such as nonnegative matrix factorization (NMF) have been successfully used to solve many data mining tasks. In this paper, we propose a new matrix lower rank approximation called Bounded Matrix Low Rank Approximation (BMA) which imposes a lower and an upper bound on every element of a lower rank matrix that best approximates a given matrix with missing elements. This new approximation models many real world problems, such as recommender systems, and performs better than other methods, such as singular value decompositions (SVD) or NMF. We present an efficient algorithm to solve BMA based on coordinate descent method. BMA is different from NMF as it imposes bounds on the approximation itself rather than on each of the low rank factors. We show that our algorithm is scalable for large matrices with missing elements on multi core systems with low memory. We present substantial experimental results illustrating that the proposed method outperforms the state of the art algorithms for recommender systems such as Stochastic Gradient Descent, Alternating Least Squares with regularization, SVD++, Bias-SVD on real world data sets such as Jester, Movielens, Book crossing, Online dating and Netflix.Keywords-low rank approximation, recommender systems, bound constraints, matrix factorization, block coordinate descent method, scalable algorithm, block I. MOTIVATIONIn a matrix low rank approximation, given a matrix R ∈ R n×m , and a lower rank k < rank(R), we find two matrices P ∈ R n×k and Q ∈ R k×m such that R is well approximated by PQ, i.e., R ≈ PQ. Low rank approximations vary depending on the constraints imposed on the factors as well as the measure for the difference between R and PQ. Low rank approximation draws huge interest in the data mining and machine learning community, for it's ability to address many foundational challenges in this area. A few prominent techniques of machine learning that use low rank approximation are principal component analysis, factor analysis, latent semantic analysis, non-negative matrix factorization, etc.One of the most important low rank approximation is based on singular value decompositions (SVD) [9]. Low rank approximation using SVD has many applications. For example, an image can be compressed by taking the low row rank approximation of its matrix representation using SVD. Similarly, in text mining -latent semantic indexing, is for document retrieval/dimensionality reduction of a termdocument matrix using SVD [6]. The other applications include event detection in streaming data, visualization of document corpus etc.Over the last decade, NMF has emerged as another important low rank approximation technique, where the low-rank factor matrices are constrained to have only non-negative elements. It has received enormous attention and has been successfully applied to a broad range of important problems in areas including text mining, computer vision, community detection in social networks, visualization, bioinformatics etc [18] [11].In this pape...

show abstract

“…Kim, He and Park [13] prove that equation (6) satisfies the formulation of BCD method. Equation (6) when extended with the matrix M becomes equation (8).…”

Section: B Bounded Matrix Low Rank Approximationmentioning

confidence: 94%

“…2) Reduced Rank k: In the case of regular low rank approximation with all known elements, the higher the k; the closer the low rank approximation to the input matrix [13]. However, in the case of predicting with the low rank factors, a good k depends on the nature of the dataset.…”

Section: B Parameter Tuningmentioning

confidence: 99%

Bounded Matrix Low Rank Approximation

Kannan¹,

Ishteva²,

Drake³

et al. 2015

Signals and Communication Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…until convergence problem, it is impractical to solve (4) directly. However, it can be tackled by applying the block coordinate descent (BCD) with two matrix blocks [12]. The framework of minimizing over one matrix while keeping the other matrix fixed is described in Algorithm 1.…”

Section: Algorithm 1: Solving the Sicmf Problemmentioning

confidence: 99%

“…In the proposed model, a complexvalued matrix will be decomposed into two matrices of complex bases and real coefficients-the solutions of a constraint complex optimization problem. real-valued cost function with respect to complex parameters was treated wisely by using the block coordinate descent [12] and the gradient descent method [13]. Without limiting the sign of data, the proposed method, siCMF, can yield extension on real-world applications, particularly the field of complex-valued data processing, such as communication, acoustic, optic, and electromagnetics.…”

Section: Introductionmentioning

confidence: 99%

A New Approach of Matrix Factorization on Complex Domain for Data Representation

Duong

Bui

Ding

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis work presents a new approach which derives a learned data representation method through matrix factorization on the complex domain. In particular, we introduce an encoding matrix-a new representation of data-that satisfies the simplicial constraint of the projective basis matrix on the field of complex numbers. A complex optimization framework is provided. It employs the gradient descent method and computes the derivative of the cost function based on Wirtinger's calculus.

show abstract

“…This MU algorithm is one of the efficient algorithms for NMF proposed in the early stage, and thus many extensions followed (e.g., Cai et al 2011;). However, from the viewpoint of convergence, they were not sufficient (Kim et al 2014). Lin (2007) proposed a Project Gradient Descent (PGD) algorithm for NMF.…”

Section: Matrix-wise Update Algorithmsmentioning

confidence: 99%

A column-wise update algorithm for nonnegative matrix factorization in Bregman divergence with an orthogonal constraint

2016

View full text Add to dashboard Cite

Recently orthogonal nonnegative matrix factorization (ONMF), imposing an orthogonal constraint into NMF, has been attracting a great deal of attention. ONMF is more appropriate than standard NMF for a clustering task because the constrained matrix can be considered as an indicator matrix. Several iterative ONMF algorithms have been proposed, but they suffer from slow convergence because of their matrix-wise updating. In this paper, therefore, a column-wise update algorithm is proposed for speeding up ONMF. To make the idea possible, we transform the matrix-based orthogonal constraint into a set of column-wise orthogonal constraints. The algorithm is stated first with the Frobenius norm and then with Bregman divergence, both for measuring the degree of approximation. Experiments on one artificial and six real-life datasets showed that the proposed algorithms converge faster than the other conventional ONMF algorithms, more than four times in the best cases, due to their smaller numbers of iterations.

show abstract

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

Cited by 319 publications

References 70 publications

Bounded Matrix Low Rank Approximation

Bounded Matrix Low Rank Approximation

A New Approach of Matrix Factorization on Complex Domain for Data Representation

A column-wise update algorithm for nonnegative matrix factorization in Bregman divergence with an orthogonal constraint

Contact Info

Product

Resources

About