Energy dissipating flows for solving nonlinear eigenpair problems

Cohen, I. Bernard; Gilboa, Guy

doi:10.1016/j.jcp.2018.09.012

Cited by 16 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relation between gradient flows and eigenvectors of general subdifferential operators was studied in Bungert and Burger (2020), Bungert, Burger, Chambolle, and Novaga (2020), Bungert, Burger, and Tenbrinck (2019), Hynd and Lindgren (2017), and Varvaruca (2004), see also . Rayleigh quotient minimizing flows were investigated in Feld, Aujol, Gilboa, and Papadakis (2019), Aujol, Gilboa, and Papadakis (2018), Cohen and Gilboa (2018), and Nossek and Gilboa (2018), see also Gilboa (2020) and Gilboa (2018). In the following we will review the flows proposed there and discover that they have strong relations, in particular being connected through time reparametrizations and normalizations.…”

Section: Flows For Solving Nonlinear Eigenproblemsmentioning

confidence: 99%

Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions

Bungert,

Burger

2021

Preprint

View full text Add to dashboard Cite

This chapter describes how gradient flows and nonlinear power methods in Banach spaces can be used to solve nonlinear eigenvector-dependent eigenvalue problems, and how these can be approximated using Γ-convergence. We review several flows fromliterature, which were proposed to compute nonlinear eigenfunctions, and show that they all relate to normalized gradient flows. Furthermore, we show that the implicit Euler discretization of gradient flows gives rise to a nonlinear power method of the proximal operator and prove their convergence to nonlinear eigenfunctions. Finally, we prove that Γ-convergence of functionals implies convergence of their ground states.

show abstract

Section: Flows For Solving Nonlinear Eigenproblemsmentioning

confidence: 99%

Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions

Bungert,

Burger

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In [13] a method for solving such problems was proposed, following the flows of [27] and [1]. The basic formulation was to solve the (double) nonlinear problem,…”

Section: Cohen-gilboa (Cg)mentioning

confidence: 99%

“…This combined flow admits (d/dt)J(u) ≤ 0 and (d/dt)E(u) ≤ 0 (for α large enough). Numerically, iterations which follow this flow are provided in [13], using the following adaptive time step for the main flow,…”

Section: Cohen-gilboa (Cg)mentioning

confidence: 99%

See 1 more Smart Citation

Iterative Methods for Computing Eigenvectors of Nonlinear Operators

Gilboa¹

2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this chapter we are examining several iterative methods for solving nonlinear eigenvalue problems. These arise in variational image-processing, graph partition and classification, nonlinear physics and 39 more. The canonical eigenproblem we solve is T (u) = λu, where T : R n → R n is some bounded nonlinear operator. Other variations of eigenvalue problems are also discussed. We present a progression of 5 algorithms, coauthored in recent years by the author and colleagues. Each algorithm attempts to solve a unique problem or to improve the theoretical foundations. The algorithms can be understood as nonlinear PDE's which converge to an eigenfunction in the continuous time domain. This allows a unique view and understanding of the discrete iterative process. Finally, it is shown how to evaluate numerically the results, along with some examples and insights related to priors of nonlinear denoisers, both classical algorithms and ones based on deep networks.

show abstract

“…[38,1,7] suggested nonlinear flows to solve eigenproblems induced by total-variation and one-homogeneous functionals. This flow was later generalized to solve eigenproblems emerging in nonlinear optics [15]. Algorithms to minimize generalized Rayleigh-quotients on grids and graphs were proposed and analyzed in [21].…”

mentioning

confidence: 99%

Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks

Bungert,

Hait-Fraenkel,

Papadakis

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Neural networks have revolutionized the field of data science, yielding remarkable solutions in a datadriven manner. For instance, in the field of mathematical imaging, they have surpassed traditional methods based on convex regularization. However, a fundamental theory supporting the practical applications is still in the early stages of development. We take a fresh look at neural networks and examine them via nonlinear eigenvalue analysis. The field of nonlinear spectral theory is still emerging, providing insights about nonlinear operators and systems. In this paper we view a neural network as a complex nonlinear operator and attempt to find its nonlinear eigenvectors. We first discuss the existence of such eigenvectors and analyze the kernel of ReLU networks. Then we study a nonlinear power method for generic nonlinear operators. For proximal operators associated to absolutely one-homogeneous convex regularization functionals, we can prove convergence of the method to an eigenvector of the proximal operator. This motivates us to apply a nonlinear method to networks which are trained to act similarly as a proximal operator. In order to take the nonhomogeneity of neural networks into account we define a modified version of the power method. We perform extensive experiments on various shallow and deep neural networks designed for image denoising. For simple nets, we observe the influence of training data on the eigenvectors. For stateof-the-art denoising networks, we show that eigenvectors can be interpreted as (un)stable modes of the network, when contaminated with noise or other degradations.

show abstract

Energy dissipating flows for solving nonlinear eigenpair problems

Cited by 16 publications

References 33 publications

Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions

Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions

Iterative Methods for Computing Eigenvectors of Nonlinear Operators

Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks

Contact Info

Product

Resources

About