Computing low‐rank approximations of the Fréchet derivative of a matrix function using Krylov subspace methods

Kandolf, Peter; Koskela, Antti; Relton, Samuel D.; Schweitzer, Marcel

doi:10.1002/nla.2401

Cited by 17 publications

(13 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several directions for future research. On the one hand, the generalization of Krylov subspace methods for the first-order Fréchet derivative with low-rank direction term [12,29] to the higher-order case would be interesting to further reduce computational complexity in this setting. On the other hand, it is worth investigating whether the proposed integral representations of the second-order Fréchet derivative also allow to find formulas or at least bounds for level-2 condition numbers of other classes of functions and/or matrices then what we have considered in the present work.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Integral representations for higher-order Fréchet derivatives of matrix functions: Quadrature algorithms and new results on the level-2 condition number

Schweitzer¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We propose an integral representation for the higher-order Fréchet derivative of analytic matrix functions f (A) which unifies known results for the first-order Fréchet derivative of general analytic matrix functions and for higher-order Fréchet derivatives of A −1 . We highlight two applications of this integral representation: On the one hand, it allows to find the exact value of the level-2 condition number (i.e., the condition number of the condition number) of f (A) for a large class of functions f when A is Hermitian. On the other hand, it also allows to use numerical quadrature methods to approximate higher-order Fréchet derivatives. We demonstrate that in certain situations-in particular when the derivative order k is moderate and the direction terms in the derivative have low-rank structure-the resulting algorithm can outperform established methods from the literature by a large margin.

show abstract

Section: Discussionmentioning

confidence: 99%

“…where I denotes the n × n identity matrix. It is known [9,11,12] that in this case, the first-order Fréchet derivative is given by…”

Section: Analytic Functions Represented By the Cauchy Integral Formulamentioning

confidence: 99%

Integral representations for higher-order Fréchet derivatives of matrix functions: Quadrature algorithms and new results on the level-2 condition number

Schweitzer¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This defines an enormous computational task and explains why automatic differentiation and corresponding software tools are almost exclusively applied. Alternative dedicated recent methods like [16] focus on a special problem structure, viz. the action of the differential of the matrix exponential on a rank-one matrix.…”

Section: Related Workmentioning

confidence: 99%

Learning Linearized Assignment Flows for Image Labeling

2023

View full text Add to dashboard Cite

We introduce a novel algorithm for estimating optimal parameters of linearized assignment flows for image labeling. An exact formula is derived for the parameter gradient of any loss function that is constrained by the linear system of ODEs determining the linearized assignment flow. We show how to efficiently evaluate this formula using a Krylov subspace and a low-rank approximation. This enables us to perform parameter learning by Riemannian gradient descent in the parameter space, without the need to backpropagate errors or to solve an adjoint equation. Experiments demonstrate that our method performs as good as highly-tuned machine learning software using automatic differentiation. Unlike methods employing automatic differentiation, our approach yields a low-dimensional representation of internal parameters and their dynamics which helps to understand how assignment flows and more generally neural networks work and perform.

show abstract

“…where Γ is again a path that winds around spec(A) exactly once; see, e.g., [12,18]. In addition to being of theoretical interest, the integral representation also forms the basis of efficient computational methods for approximating L f (A, E), in particular when E is of low rank; see [17,18,21], as well as [29] for an extension to higher-order Fréchet derivatives. Related is the Gâteaux (or directional) derivative of f at A, defined as…”

Section: The Fréchet Derivativementioning

confidence: 99%

The Fréchet derivative of the tensor t-function

Lund¹,

Schweitzer²

2023

Preprint

View full text Add to dashboard Cite

The tensor t-function, a formalism that generalizes the well-known concept of matrix functions to third-order tensors, is introduced in [K. Lund, The tensor t-function: a definition for functions of third-order tensors, Numer. Linear Algebra Appl. 27 (3), e2288]. In this work, we investigate properties of the Fréchet derivative of the tensor t-function and derive algorithms for its efficient numerical computation. Applications in condition number estimation and nuclear norm minimization are explored. Numerical experiments implemented by the t-Frechet toolbox hosted at https://gitlab.com/katlund/t-frechet illustrate properties of the t-function Fréchet derivative as well as the efficiency and accuracy of the proposed algorithms.

show abstract

Computing low‐rank approximations of the Fréchet derivative of a matrix function using Krylov subspace methods

Cited by 17 publications

References 52 publications

Integral representations for higher-order Fréchet derivatives of matrix functions: Quadrature algorithms and new results on the level-2 condition number

Integral representations for higher-order Fréchet derivatives of matrix functions: Quadrature algorithms and new results on the level-2 condition number

Learning Linearized Assignment Flows for Image Labeling

The Fréchet derivative of the tensor t-function

Contact Info

Product

Resources

About