A Review on Adaptive Low-Rank Approximation Techniques in the Hierarchical Tensor Format

Ballani, Jonas; Grasedyck, Lars; Kluge, Melanie

doi:10.1007/978-3-319-08159-5_10

Cited by 3 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is usually assumed that the tree structure of TTNS is given or assumed a priori, and recent efforts aim to find an optimal tree structure from a subset of tensor entries and without any a priori knowledge of the tree structure. This is achieved using so-called rank-adaptive cross-approximation techniques which approximate a tensor by hierarchical tensor formats [9,10].…”

Section: (P)mentioning

confidence: 99%

See 1 more Smart Citation

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Cichocki

Lee

Oseledets

et al. 2016

FNT in Machine Learning

390

342

View full text Add to dashboard Cite

Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning algorithms typically scale exponentially with data volume and complexity of cross-modal couplings - the so called curse of dimensionality - which is prohibitive to the analysis of large-scale, multi-modal and multi-relational datasets. Given that such data are often efficiently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review low-rank tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization problems. Our particular emphasis is on elucidating that, by virtue of the underlying low-rank approximations, tensor networks have the ability to alleviate the curse of dimensionality in a number of applied areas. In Part 1 of this monograph we provide innovative solutions to low-rank tensor network decompositions and easy to interpret graphical representations of the mathematical operations on tensor networks. Such a conceptual insight allows for seamless migration of ideas from the flat-view matrices to tensor network operations and vice versa, and provides a platform for further developments, practical applications, and non-Euclidean extensions. It also permits the introduction of various tensor network operations without an explicit notion of mathematical expressions, which may be beneficial for many research communities that do not directly rely on multilinear algebra. Our focus is on the Tucker and tensor train (TT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide linearly or even super-linearly (e.g., logarithmically) scalable solutions, as illustrated in detail in Part 2 of this monograph

show abstract

Section: (P)mentioning

confidence: 99%

“…Reshape the matrix B n into the (n + 1)th unfolding matrix M n+1 = reshape B T n , [R n I n+1 , ś N p=n+2 I p ] 8: end for 9: Construct the last core as p X (N) = reshape(M N , [R N´1 , I N , 1]) 10: return TT-cores: xx p X (1) , p X (2) , . .…”

mentioning

confidence: 99%

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Cichocki

Lee

Oseledets

et al. 2016

FNT in Machine Learning

390

342

View full text Add to dashboard Cite

show abstract

“…We repeat qMC experiments using the same generating vector q but S different shifts. Thus we obtain S sets of nodes (7), and use (6) to calculate the estimatorsĨ j , j = 1, . .…”

Section: Monte Carlo and Quasi Monte Carlo Techniquesmentioning

confidence: 99%

Parallel cross interpolation for high-precision calculation of high-dimensional integrals

Dolgov

Savostyanov

2020

Computer Physics Communications

View full text Add to dashboard Cite

We propose a parallel version of the cross interpolation algorithm and apply it to calculate high-dimensional integrals motivated by Ising model in quantum physics. In contrast to mainstream approaches, such as Monte Carlo and quasi Monte Carlo, the samples calculated by our algorithm are neither random nor form a regular lattice. Instead we calculate the given function along individual dimensions (modes) and use this data to reconstruct its behaviour in the whole domain. The positions of the calculated univariate fibers are chosen adaptively for the given function. The required evaluations can be executed in parallel both along each mode (variable) and over all modes.To demonstrate the efficiency of the proposed method, we apply it to compute highdimensional Ising susceptibility integrals, arising from asymptotic expansions for the spontaneous magnetisation in two-dimensional Ising model of ferromagnetism. We observe strong superlinear convergence of the proposed method, while the MC and qMC algorithms converge sublinearly. Using multiple precision arithmetic, we also observed exponential convergence of the proposed algorithm. Combining high-order convergence, almost perfect scalability up to hundreds of processes, and the same flexibility as MC and qMC, the proposed algorithm can be a new method of choice for problems involving high-dimensional integration, e.g. in statistics, probability, and quantum physics. ≈ −1 I I J J I I I J J J Figure 1: Cross interpolation for matrices. The full matrix A is approximated by a low-rank decompositionÃ based on a small number of columns and rows computed in A. Note that the approximation (1) is exact in the positions of computed rows and columns. One of the first examples of the latter was the parallel density matrix renormalization group (DMRG) algorithm [81] for ground state computations in quantum physics. In mathematical community this research direction started with dimension-parallel linear solver [30] and cross algorithms in HT format [44]. The main difficulty of parallelisation over dimension is the need of algorithmic modifications, since state of the arts tensor algorithms were designed in intrinsically sequential way. Ideally, such modifications should not compromise numerical stability or convergence for the sake of parallel efficiency. In this paper we develop a parallel version of the TT cross interpolation algorithm [74]. The parallel algorithm is adaptive and converges with the same rate as the sequential version, but involves only local communications with constant loading of processes, and demonstrates almost perfect scaling up to the ultimate partitioning where each process is responsible for a single direction (mode, variable). Moreover, further speedup can be achieved using OpenMP parallelisation of tensor algebra in each process.The rest of the paper is organised as follows. In Sec. 2 we recall the cross interpolation method for matrices and provide necessary definitions. In Sec. 3 we discuss how the matrix interpolation can be applied for high-dimensi...

show abstract

Uncertainty Quantification andBayesian Inversion

Matthies

2017

Encyclopedia of Computational Mechanics Second Edition

View full text Add to dashboard Cite

Uncertainty estimation arises at least implicitly in any kind of modeling of the real – or phenomenological – world, and it is desirable to actually quantify the uncertainty in probabilistic terms. Here the emphasis is on uncertain systems, typically modeled by partial differential equations, where the randomness is assumed spatial. Traditional computational approaches usually use some form of perturbation or Monte Carlo simulation. Here the emphasis is on recent methods based on stochastic functional or spectral approximations. Quantifying the uncertainty in mathematical models due to uncertainties in their parameters, combined with the traditional ability to predict the system behavior given external excitations or loadings, is called solving the stochastic forward problem . Given this capability, one has the possibility to use this to estimate or identify unknown and hence uncertain parameters in the mathematical model by observing the system response. This is called a solution to the inverse problem , and a short introduction into inverse problems in a Bayesian setting using the functional or spectral approximations is provided.

show abstract

A Review on Adaptive Low-Rank Approximation Techniques in the Hierarchical Tensor Format

Cited by 3 publications

References 34 publications

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Parallel cross interpolation for high-precision calculation of high-dimensional integrals

Uncertainty Quantification andBayesian Inversion

Contact Info

Product

Resources

About