An Alternating Rank-<i>k</i> Nonnegative Least Squares Framework (ARkNLS) for Nonnegative Matrix Factorization

Chu, Delin; Shi, Wenya; Eswar, Srinivas; Park, Haesun

doi:10.1137/20m1352405

Cited by 11 publications

(3 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2e, g, 3b, and 4h) at the level of individual neurons and trials (Fig. 3c, 4e), we used non-negative TCA 68 with a framework of alternating rank-k non-negativity constrained least squares based on block principal pivoting method 69 . Our analysis was focused on rank 4 as a higher number of latents overfitted the data.…”

Section: Saturating Activation Ratiosmentioning

confidence: 99%

Competition of cortical clusters during on-demand visual learning in immersive virtual reality

Rozsa,

Judák,

Dobos

et al. 2023

Preprint

View full text Add to dashboard Cite

Neural circuits support rapid visual learning. However, due to technical roadblocks, it is not known how visual circuits represent multiple features or how behaviorally relevant representations are selected for long-term memory. Here we developed Moculus, a head-mounted virtual reality platform for mice that covers the entire visual field, and allows binocular depth perception and full immersion. This controllable environment, combined with fast acousto-optical imaging, affords rapid visual learning and the uncovering of novel circuit substrates: both the control and reinforcement-associated visual cue coding neuronal assemblies are extended transiently to a near-saturating level. They formed partially orthogonal and overlapping clusters centered around hub cells with higher and earlier ramp-like responses, as well as locally increased connectivity. This temporally maximizes computational capability and allows competition between assemblies that encode behaviorally relevant information by stochastic fluctuation from trial-to-trial. The coding competition is driven by reinforcement feedback at the level of individual neurons.

show abstract

Section: Saturating Activation Ratiosmentioning

confidence: 99%

Competition of cortical clusters during on-demand visual learning in immersive virtual reality

Rozsa,

Judák,

Dobos

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In this paper, we combine these approaches to tackle the HPO related to Matrix Decomposition (MDs) in an unsupervised scenario. In particular, we focused our attention on Nonnegative Matrix Factorizations (NMF) and its sparseness constrained variants [8,6,12,19,23,25,34,35,39]. We regard these problems as penalized optimization tasks with penalization coefficients to be considered HPs, and we focus on their proper choice on HPO issue.…”

Section: Introductionmentioning

confidence: 99%

Bi-level Optimization for hyperparameters in Nonnegative Matrix Factorizations

Buono¹,

Esposito²,

Selicato³

et al. 2022

Preprint

View full text Add to dashboard Cite

Hyperparameters (HPs) Optimization in machine learning algorithms represents an open problem with a direct impact on algorithm performances and on the knowledge extraction process from data. Matrix Decompositions (MDs) has recently gained more attention in data science as mathematical techniques able to capture latent information embedded in large datasets. MDs can be formalized as penalized optimization problems in which the tuning of the penalization HPs represents an issue. Current literature panorama does not provide any general framework addressing optimally conditions for the best configuration of HPs. In this work, we focus on the role the HPs play in the penalized Nonnegative Matrix Factorizations (NMF) context and on the importance of their correct selection. Using a bi-level optimization approach, HPs selection is considered from an optimization point of view and their choice is directly incorporated in the unsupervised algorithm as a part of the updating process. Either theorems of existence and convergence of numerical solutions, under appropriate assumptions, are studied and a novel algorithm, namely AltBi-Alternate bi-level Algorithm, is proposed to tune the penalization HP in NMF problems.

show abstract

“…Large scale applications in which (NNLS) appears include NMR relaxometry [62], imaging deblurring [4], biometric templates [40], transcriptomics [42], magnetic microscopy [50], sparse hyperspectral unmixing [26], and system identification [14]. The formulation in (NNLS) is also closely related to non-negative matrix factorization [19] and supervised learning methods such as support vector machines [67]. We refer the interested reader to [13] for a survey about the development of algorithms that enforce non-negativity.…”

Section: Introductionmentioning

confidence: 99%

Non-negative Least Squares via Overparametrization

Chou¹,

Maly²,

Verdun³

2022

Preprint

View full text Add to dashboard Cite

In many applications, solutions of numerical problems are required to be non-negative, e.g., when retrieving pixel intensity values or physical densities of a substance. In this context, nonnegative least squares (NNLS) is a ubiquitous tool, especially when seeking sparse solutions of high-dimensional statistical problems. Despite vast efforts since the seminal work of Lawson and Hanson in the '70s, the non-negativity assumption is still an obstacle for the theoretical analysis and scalability of many off-the-shelf solvers. In the different context of deep neural networks, we recently started to see that the training of overparametrized models via gradient descent leads to surprising generalization properties and the retrieval of regularized solutions. In this paper, we prove that, by using an overparametrized formulation, NNLS solutions can reliably be approximated via vanilla gradient flow. We furthermore establish stability of the method against negative perturbations of the ground-truth. Our simulations confirm that this allows to use vanilla gradient descent as a novel and scalable numerical solver for NNLS.

show abstract

An Alternating Rank-k Nonnegative Least Squares Framework (ARkNLS) for Nonnegative Matrix Factorization

Cited by 11 publications

References 40 publications

Competition of cortical clusters during on-demand visual learning in immersive virtual reality

Competition of cortical clusters during on-demand visual learning in immersive virtual reality

Bi-level Optimization for hyperparameters in Nonnegative Matrix Factorizations

Non-negative Least Squares via Overparametrization

Contact Info

Product

Resources

About