Sequential Coordinate-Wise Algorithm for the Non-negative Least Squares Problem

Franc, Vojtĕch; Hlaváč, Václav; Navara, Mirko

doi:10.1007/11556121_50

Cited by 74 publications

(61 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bellavia et al [2] have studied an interior point Newton-like method, and Franc et al [31] presented a sequential coordinate-wise method. Some observations about the NMF computation based on these methods as well as other methods are offered in Cichocki et al [22].…”

Section: Discussion and Other Methodsmentioning

confidence: 99%

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

2013

View full text Add to dashboard Cite

We review algorithms developed for nonnegative matrix factorization (NMF) and nonnegative tensor factorization (NTF) from a unified view based on the block coordinate descent (BCD) framework. NMF and NTF are low-rank approximation methods for matrices and tensors in which the low-rank factors are constrained to have only nonnegative elements. The nonnegativity constraints have been shown to enable natural interpretations and allow better solutions in numerous applications including text analysis, computer vision, and bioinformatics. However, the computation of NMF and NTF remains challenging and expensive due the constraints. Numerous algorithmic approaches have been proposed to efficiently compute NMF and NTF. The BCD framework in constrained non-linear optimization readily explains the theoretical convergence properties of several efficient NMF and NTF algorithms, which are consistent with experimental observations reported in literature. In addition, we discuss algorithms that do not fit in the BCD framework contrasting them from those based on the BCD framework. With insights acquired from the unified perspective, we also propose efficient algorithms for updating NMF when there is a small change in the reduced dimension or in the data. The effectiveness of the proposed updating algorithms are validated experimentally with synthetic and real-world data sets.

show abstract

Section: Discussion and Other Methodsmentioning

confidence: 99%

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

2013

View full text Add to dashboard Cite

show abstract

“…More recently, Franc et al [7] proposed a cycle block coordinate descent method having fast convergence in practice with low complexity of each iteration, but it still has been not theoretically guaranteed. Subsequently, Vamsi [19] suggested three modifications of random permutations [17], shrinking, and random projections to speed up NNLS for the case that the matrix A is not thin (d ≤ n).…”

Section: Related Workmentioning

confidence: 99%

“…#(nd): nonzero number of matrix having size nd d dimension of data, n number of variables, μ convex parameter, L Lipschitz constant, ELS exact line search, Coord greedy block coordinate descent [7], Accer accelerated method [10], fast active set methods according to Bro R., de [2] Nm non-monotonic fast method [13], frugal frugal coordinate descent [19], Antilop: the proposed method ✓ satisfied (positive), ✗ unsatisfied, ? unknown In summary, active set methods and iterative methods are two major approaches in solving NNLS.…”

Section: Related Workmentioning

confidence: 99%

“…-Coord: this is a cycle block coordinate descent algorithm [7] with fast convergence in practice [17]. -Accer: this is a Nesterov accelerated method with convergence rate O( L k 2 ) [10].…”

Section: Comparison With State-of-the-art Algorithmsmentioning

confidence: 99%

“…However, these approaches have two main drawbacks: invertibility of A T A and its heavy computation, especially for the methods recomputing (A T A) −1 several times for different passive sets. Hence, first-order methods [7,13,19] can be more effective for large-scale least squares problems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Accelerated anti-lopsided algorithm for nonnegative least squares

Nguyen

2016

Int J Data Sci Anal

View full text Add to dashboard Cite

Nonnegative least squares (NNLS) problem has been widely used in scientific computation and data modeling, especially for low-rank representation such as nonnegative matrix and tensor factorization. When applied to large-scale datasets, first-order methods are preferred to provide fast flexible computation for regularized NNLS variants, but they still have the limitations of performance and convergence as key challenges. In this paper, we propose an = 1 for all variables x i to implicitly exploit the second-order derivative, and to guarantee that μ and L are always bounded in order to achieve overbounded convergence of the algorithm, and to enhance the performance of internal processes based on exact line search, greedy coordinate descent methods, and accelerated search. The experiments on large matrices and real applications of nonnegative matrix factorization clearly show the higher performance of the proposed algorithm in comparison with the state-of-the-art algorithms.

show abstract