Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method

Abstract

Abstract.The non-negative matrix factorization (NMF) determines a lower rank approximation of a matrixis given and nonnegativity is imposed on all components of the factors £ 7 ¥ 8 § © @ 9and £ 7 ¥ A 9 B © @. The NMF has attracted much attention for over a decade and has been successfully applied to numerous data analysis problems. In applications where the components of the data are necessarily nonnegative such as chemical concentrations in experimental results or pixels in digital images, the NMF provides a more relevant interpretation of the results since it gives non-subtractive combinations of non-negative basis vectors. In this paper, we introduce an algorithm for the NMF based on alternating non-negativity constrained least squares (NMF/ANLS) and the active set based fast algorithm for non-negativity constrained least squares with multiple right hand side vectors, and discuss its convergence properties and a rigorous convergence criterion based on the Karush-Kuhn-Tucker (KKT) conditions. In addition, we also describe algorithms for sparse NMFs and regularized NMF. We show how we impose a sparsity constraint on one of the factors by C E D -norm minimization and discuss its convergence properties. Our algorithms are compared to other commonly used NMF algorithms in the literature on several test data sets in terms of their convergence behavior.

Real-Time Fuzzy Analysis of Machine Driven Tunneling

“…Similar to the POD approach, W and A + are denoted as the basis matrix and coefficient matrix, respectively. The alternating non-negative least squares algorithm proposed in [15], which ensures the convergence of the minimization problem, is implemented in this paper. The prediction for non-negative outputs S a is computed similar to the POD-RBF method as S a ⇡ W · B + · F a .…”

A convergent algorithm for orthogonal nonnegative matrix factorization

“…Maximum number of iteration is very crucial in MU and AU algorithms since these algorithms are known to be very slow [2,7,8,9,10,16,18,20,21,22,23]. As shown by Lin [23], LS is very fast to minimize the objective for some first iterations, but then tends to become slower.…”

“…As shown in ref. [8,9,18], sparse NMF usually can give good results if α and/or β are rather small positive numbers.…”

“…And because the orthogonality constraints cannot be recast into alternating nonnegativity least square (ANLS) framework (see [8,18] for discussion on ANLS), converged algorithms for the standard NMF, e.g., [20,21,2,23,18,22], cannot be utilized for solving orthogonal NMF problems. Thus, there is still no converged algorithm for orthogonal NMFs.…”

Fast Nonnegative Tensor Factorization with an Active-Set-Like Method

“…A classical method for solving the NNLS problem is the active set method of Lawson and Hanson [20]; however, applying Lawson and Hanson's method directly to NNCP is extremely slow. Bro and De Jong [4] suggested an improved active-set method to solve the NNLS problems, and Ven Benthem and Keenan [28] further accelerated the active-set method, which was later utilized in NMF [14] and NNCP [15]. In Friedlander and Hatz [10], the NNCP subproblems are solved by a two-metric projected gradient descent method.…”