Non-monotone projection gradient method for non-negative matrix factorization

Abstract

Since Non-negative Matrix Factorization (NMF) was first proposed over a decade ago, it has attracted much attention, particularly when applied to numerous data analysis problems. Most of the existing algorithms for NMF are based on multiplicative iterative and alternating least squares algorithms. However, algorithms based on the optimization method are few, especially in the case where two variables are derived at the same time. In this paper, we propose a non-monotone projection gradient method for NMF and establish the convergence results of our algorithm. Experimental results show that our algorithm converges to better solutions than popular multiplicative update-based algorithms.

An Efficient Algorithm for Solving the Matrix Optimization Problem in the Unsupervised Feature Selection

“…which is a contradiction of (26); hence, (25) holds. ( 24) and ( 25) imply that Y * satisfies the KKT conditions (5). In a similar way, we can prove that X * satisfies the KKT conditions (5).…”

“…Now we will prove the point {(X * , Y * )} is the stationary point of Problem 1, that is, we will prove that {(X * , Y * )} satisfies the KKT conditions (5). We first prove…”

“…The alternating nonnegative least squares method is used to solve nonnegative subproblems. Many numerical methods, such as the active method [3], the projected gradient method [4,5], the projected Barzilai-Borwein method [6,7], the projected Newton method [8] and the projected quasi-Newton method [9,10], have been designed to solve these subproblems.…”

Spectral Projected Gradient Methods: Review and Perspectives

“…Ruggiero, Martínez, and Santos 2004;Gomes-Ruggiero, Martínez, and Santos 2009), nonnegative matrix factorization (Li, Liu, and Zheng 2012), and topology optimization (Tavakoli and Zhang 2012). Moreover, alternative choices of the spectral step length have been considered and analyzed for solving some related nonlinear problems.…”

Spectral residual methods with two new non-monotone line searches for large-scale nonlinear systems of equations

“…This nonmonotone technique, which comes from [9], chooses where m(k) is adjusted in each iteration. However, this non-monotone method also has several shortcomings [10]. For instance, the non-monotone skill required a maximum function, which may result in lost information of good functions.…”