Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse

Wu, Lingfei; Laeuchli, Jesse; Kalantzis, Vassilis; Stathopoulos, Andreas; Gallopoulos, Efstratios

doi:10.1016/j.jcp.2016.09.001

Cited by 34 publications

(30 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hence, to efficiently compute the quantity m , it suffices to insert a few lines related to (12) and (13) into the existing Lanczos iteration (1). Then, with m , the incremental error is computed in a straightforward manner by using (11). We now put back the index k and summarize this computation in Algorithm 1.…”

Section: Iterative Algorithm For Computing the Incremental Errormentioning

confidence: 99%

“…The trace of a function of a matrix, tr( f(A)), occurs in diverse areas including scientific computing, statistics, and machine learning. [1][2][3][4][5][6][7][8][9][10][11][12] Often in applications, the matrix A is so large that explicitly forming f(A) is not a practically viable option. In this work, we focus on the case when A ∈ R n×n is symmetric positive-definite, so that it admits a spectral decomposition Q T AQ = diag( 1 , … , n ), where the i are real positive eigenvalues and Q is the matrix of normalized eigenvectors.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A posteriori error estimate for computing tr(f(A)) by using the Lanczos method

Chen

Saad

2018

Numerical Linear Algebra App

View full text Add to dashboard Cite

Summary An outstanding problem when computing a function of a matrix, f(A), by using a Krylov method is to accurately estimate errors when convergence is slow. Apart from the case of the exponential function that has been extensively studied in the past, there are no well‐established solutions to the problem. Often, the quantity of interest in applications is not the matrix f(A) itself but rather the matrix–vector products or bilinear forms. When the computation related to f(A) is a building block of a larger problem (e.g., approximately computing its trace), a consequence of the lack of reliable error estimates is that the accuracy of the computed result is unknown. In this paper, we consider the problem of computing tr(f(A)) for a symmetric positive‐definite matrix A by using the Lanczos method and make two contributions: (a) an error estimate for the bilinear form associated with f(A) and (b) an error estimate for the trace of f(A). We demonstrate the practical usefulness of these estimates for large matrices and, in particular, show that the trace error estimate is indicative of the number of accurate digits. As an application, we compute the log determinant of a covariance matrix in Gaussian process analysis and underline the importance of error tolerance as a stopping criterion as a means of bounding the number of Lanczos steps to achieve a desired accuracy.

show abstract

Section: Iterative Algorithm For Computing the Incremental Errormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A posteriori error estimate for computing tr(f(A)) by using the Lanczos method

Chen

Saad

2018

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

“…There are three main factors contributing to the computational complexity: (i) computation of the K max smallest eigenvectors of L N , (ii) K-means clustering, and (iii) computation of the community detection loss function and the model mismatch metric. The overall computational complexity of each method is summarized in Table I. For (i), computing the K max smallest eigenvectors of L N requires O(K max (m + n)) operations using power iteration techniques [35], [36], [37], [38], [39], where m+n is the number of nonzero entries in L N . For (ii), given any K ≤ K max , K-means clustering on the rows of the K smallest eigenvectors of L N requires O(nK 2 ) operations [40].…”

Section: Computational Complexity Analysismentioning

confidence: 99%

Revisiting Spectral Graph Clustering with Generative Community Models

Chen¹,

2017

2017 IEEE International Conference on Data Mining (ICDM)

Self Cite

View full text Add to dashboard Cite

The methodology of community detection can be divided into two principles: imposing a network model on a given graph, or optimizing a designed objective function. The former provides guarantees on theoretical detectability but falls short when the graph is inconsistent with the underlying model. The latter is model-free but fails to provide quality assurance for the detected communities. In this paper, we propose a novel unified framework to combine the advantages of these two principles. The presented method, SGC-GEN, not only considers the detection error caused by the corresponding model mismatch to a given graph, but also yields a theoretical guarantee on community detectability by analyzing Spectral Graph Clustering (SGC) under GENerative community models (GCMs). SGC-GEN incorporates the predictability on correct community detection with a measure of community fitness to GCMs. It resembles the formulation of supervised learning problems by enabling various community detection loss functions and model mismatch metrics. We further establish a theoretical condition for correct community detection using the normalized graph Laplacian matrix under a GCM, which provides a novel data-driven loss function for SGC-GEN. In addition, we present an effective algorithm to implement SGC-GEN, and show that the computational complexity of SGC-GEN is comparable to the baseline methods. Our experiments on 18 real-world datasets demonstrate that SGC-GEN possesses superior and robust performance compared to 6 baseline methods under 7 representative clustering metrics.

show abstract

“…The variance of this estimator grows with N 2 and can be very large even when the diagonal elements of A −1 have a small variance. Sophisticated versions of this approach can be found in [1,2].…”

Section: Introductionmentioning

confidence: 99%

Multigrid deflation for Lattice QCD

Romero

Stathopoulos

Orginos

2020

Journal of Computational Physics

Self Cite

View full text Add to dashboard Cite

A B S T R A C TComputing the trace of the inverse of large matrices is typically addressed through statistical methods. Deflating out the lowest eigenvectors or singular vectors of the matrix reduces the variance of the trace estimator. This work summarizes our efforts to reduce the computational cost of computing the deflation space while achieving the desired variance reduction for Lattice QCD applications. Previous efforts computed the lower part of the singular spectrum of the Dirac operator by using an eigensolver preconditioned with a multigrid linear system solver. Despite the improvement in performance in those applications, as the problem size grows the runtime and storage demands of this approach will eventually dominate the stochastic estimation part of the computation.In this work, we propose to compute the deflation space in one of the following two ways. First, by using an inexact eigensolver on the Hermitian, but maximally indefinite, operator Aγ 5 . Second, by exploiting the fact that the multigrid prolongator for this operator is rich in components toward the lower part of the singular spectrum. We show experimentally that the inexact eigensolver can approximate the lower part of the spectrum even for ill-conditioned operators. Also, the deflation based on the multigrid prolongator is more efficient to compute and apply, and, despite its limited ability to approximate the fine level spectrum, it obtains similar variance reduction on the trace estimator as deflating with approximate eigenvectors from the fine level operator.

show abstract

Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse

Cited by 34 publications

References 39 publications

A posteriori error estimate for computing tr(f(A)) by using the Lanczos method

A posteriori error estimate for computing tr(f(A)) by using the Lanczos method

Revisiting Spectral Graph Clustering with Generative Community Models

Multigrid deflation for Lattice QCD

Contact Info

Product

Resources

About