Algorithms for Numerical Analysis in High Dimensions

Abstract

Nearly every numerical analysis algorithm has computational complexity that scales exponentially in the underlying physical dimension. The separated representation, introduced previously, allows many operations to be performed with scaling that is formally linear in the dimension. In this paper we further develop this representation by (i) discussing the variety of mechanisms that allow it to be surprisingly efficient;(ii) addressing the issue of conditioning; (iii) presenting algorithms for solving linear systems within this framework; and (iv) demonstrating methods for dealing with antisymmetric functions, as arise in the multiparticle Schrödinger equation in quantum mechanics.Numerical examples are given.1. Introduction. The computational complexity of most algorithms in dimension d grows exponentially in d. Even simply accessing a vector in dimension d requires M d operations, where M is the number of entries in each direction. This effect has been dubbed the curse of dimensionality [1, p. 94], and it is the single greatest impediment to computing in higher dimensions. In [3] we introduced a strategy for performing numerical computations in high dimensions with greatly reduced cost, while maintaining the desired accuracy. In the present work, we extend and develop these techniques in a number of ways. In particular, we address the issue of conditioning, describe how to solve linear systems, and show how to deal with antisymmetric functions. We present numerical examples for each of these algorithms.

A Flexible Generalized Conjugate Residual Method with Inner Orthogonalization and Deflated Restarting

“…These sequences correspond to situations where only the righthand sides are changing for a given dimension d. An efficient solution method is of primary interest in certain applications related to e.g. financial engineering, molecular biology or quantum dynamics [5,6]. In the numerical experiments reported here (performed in Matlab) we have used second order finite difference discretization schemes leading to sparse matrices with at most 2d+ 1 nonzero elements per row.…”

A regularized Newton method for the efficient approximation of tensors represented in the canonical tensor format

“…One of the most popular minimization method for solving the lower rank approximation problem with fixed representation rank is the alternating least squares (ALS) algorithm. In [11,Harshman], the ALS method was applied for principle component analysis of order three tensors and in [1,2,Beylkin,Mohlenkamp] for tensors presented in the canonical format. Furthermore, the minimization problem was also solved by a Gauss-Newton method in [13,14,Paatero] and by Newton method in [12, Oseledets, Savost'yanov].…”

Wave propagation using bases for bandlimited functions

“…This representation is similar to the partitioned singular value decomposition considered in [16,17]. We note that both the separated and PLR representations are interesting on their own, with applications in other areas, e.g., computational quantum mechanics (see [15,18]). …”

“…where the transition matrix between the coefficients s il ands il has the block tridiagonal structure (17) and (18). Following the derivation in [11], we obtain (20) for l = 0, .…”