Randomized nonnegative matrix factorization

Erichson, N. Benjamin; Mendible, Ariana; Wihlborn, Sophie; Kutz, J. Nathan

doi:10.1016/j.patrec.2018.01.007

Cited by 46 publications

(45 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then B = Q * c A is formed with the last view. Erichson et al [14] used such a matrix B, generated by standard subspace iteration, to initialize their proposed method for generating a nonnegative matrix factorization. For more flexibility, the half subspace iteration approach could also be used within their scheme.…”

Section: Other Matrix Factorization Methodsmentioning

confidence: 99%

Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

Bjarkason¹

2019

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

This paper describes practical randomized algorithms for low-rank matrix approximation that accommodate any budget for the number of views of the matrix. The presented algorithms, which are aimed at being as pass efficient as needed, expand and improve on popular randomized algorithms targeting efficient low-rank reconstructions. First, a more flexible subspace iteration algorithm is presented that works for any views v ≥ 2, instead of only allowing an even v. Secondly, we propose more general and more accurate singlepass algorithms. In particular, we propose a more accurate memory efficient single-pass method and a more general single-pass algorithm which, unlike previous methods, does not require prior information to assure near peak performance. Thirdly, combining ideas from subspace and single-pass algorithms, we present a more passefficient randomized block Krylov algorithm, which can achieve a desired accuracy using considerably fewer views than that needed by a subspace or previously studied block Krylov methods. However, the proposed accuracy enhanced block Krylov method is restricted to large matrices that are either accessed a few columns or rows at a time. Recommendations are also given on how to apply the subspace and block Krylov algorithms when estimating either the dominant left or right singular subspace of a matrix, or when estimating a normal matrix, such as those appearing in inverse problems. Computational experiments are carried out that demonstrate the applicability and effectiveness of the presented algorithms. Motivation.The primary motivation for this study was to develop algorithms to speed up inverse methods used to estimate parameters in models describing subsurface flow in geothermal reservoirs [36,3]. Inverting models describing complex geophysical processes, such as fluid flow in the subsurface, frequently involves matching a large data set using highly parameterized computational models. Running the model commonly involves solving an expensive and nonlinear forward problem. Despite the possible nonlinearity of the forward problem, the link between the model parameters and simulated observations is often described in terms of a Jacobian matrix J ∈ R N d ×N m , which locally linearizes the relationship between the parameters and observations. The size of J is therefore determined by the (large) parameter and observation spaces. In this case, explicitly forming J is out of the question since at best it involves solving N m direct problems (linearized forward simulations) or N d adjoint problems (linearized backward simulations) [6,23,35,38]. Nevertheless, the information contained in J can be helpful for the purpose of inverting the model using nonlinear inversion methods such as a Gauss-Newton or Levenberg-Marquardt approach, and for quantifying uncertainty.Using adjoint simulation, direct simulation and randomized algorithms, the necessary information can be extracted from J without ever explicitly forming the large matrix J . Bjarkason et al. [3] showed that inversion of a nonlinea...

show abstract

Section: Other Matrix Factorization Methodsmentioning

confidence: 99%

Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

Bjarkason¹

2019

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…For further details, such as initialization techniques, stopping criterion, and variants of randomized HALS we refer to Erichson et al (2018a). For the absolute chemistry concentration data matrix, the columns of the factor W are the spatial modes while those of factor H are the temporal modes.…”

Section: Randomized Nonnegative Matrix Factorizationmentioning

confidence: 99%

Scalable diagnostics for global atmospheric chemistry using Ristretto library (version 1.0)

et al. 2019

Self Cite

View full text Add to dashboard Cite

Abstract. We introduce a new set of algorithmic tools capable of producing scalable, low-rank decompositions of global spatiotemporal atmospheric chemistry data. By exploiting emerging randomized linear algebra algorithms, a suite of decompositions are proposed that extract the dominant features from big data sets (i.e., global atmospheric chemistry at longitude, latitude, and elevation) with improved interpretability. Importantly, our proposed algorithms scale with the intrinsic rank of the global chemistry space rather than the ever increasing spatiotemporal measurement space, thus allowing for the efficient representation and compression of the data. In addition to scalability, two additional innovations are proposed for improved interpretability: (i) a nonnegative decomposition of the data for improved interpretability by constraining the chemical space to have only positive expression values (unlike PCA analysis); and (ii) sparse matrix decompositions, which threshold small weights to zero, thus highlighting the dominant, localized spatial activity (again unlike PCA analysis). Our methods are demonstrated on a full year of global chemistry dynamics data, showing the significant improvement in computational speed and interpretability. We show that the decomposition methods presented here successfully extract known major features of atmospheric chemistry, such as summertime surface pollution and biomass burning activities.

show abstract

“…12 The effect of parallel update will be predominantly visible when one has to decompose a multidimensional array into matrices of varied dimensions. This problem is called NonNegative Tensor Factorization (NTF) [40], [41], [42], [43]. A brief overview of NTF is given below.…”

Section: Simulationsmentioning

confidence: 99%

Novel Algorithms Based on Majorization Minimization for Nonnegative Matrix Factorization

2019

View full text Add to dashboard Cite

Matrix decomposition is ubiquitous and has applications in various fields like speech processing, data mining and image processing to name a few. Under matrix decomposition, nonnegative matrix factorization is used to decompose a nonnegative matrix into a product of two nonnegative matrices which gives some meaningful interpretation of the data. Thus, nonnegative matrix factorization has an edge over the other decomposition techniques. In this paper, we propose two novel iterative algorithms based on Majorization Minimization (MM)-in which we formulate a novel upper bound and minimize it to get a closed form solution at every iteration. Since the algorithms are based on MM, it is ensured that the proposed methods will be monotonic. The proposed algorithms differ in the updating approach of the two nonnegative matrices. The first algorithm-Iterative Nonnegative Matrix Factorization (INOM) sequentially updates the two nonnegative matrices while the second algorithm-Parallel Iterative Nonnegative Matrix Factorization (PARINOM) parallely updates them. We also prove that the proposed algorithms converge to the stationary point of the problem. Simulations were conducted to compare the proposed methods with the existing ones and was found that the proposed algorithms performs better than the existing ones in terms of computational speed and convergence.

show abstract

Randomized nonnegative matrix factorization

Cited by 46 publications

References 44 publications

Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

Scalable diagnostics for global atmospheric chemistry using Ristretto library (version 1.0)

Novel Algorithms Based on Majorization Minimization for Nonnegative Matrix Factorization

Contact Info

Product

Resources

About