Greedy low-rank approximation in Tucker format of solutions of tensor linear systems

Georgieva, Irina; Hofreither, Clemens

doi:10.1016/j.cam.2019.03.002

Cited by 15 publications

(10 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from these works, our proposed ISLET is a one-step procedure that only involves solving a simple least squares regression after performing dimension reduction on covariates by importance sketching (see Steps 1 and 2 in Section 2.2). Moreover, many prior works mainly focused on computational aspects of their proposed methods [7,13,42,48,51], while we show that ISLET is not only computationally efficient (see more discussion and comparison on computation complexity in Section 2.2 Computation and Implementation part) but also has optimal theoretical guarantees in terms of mean square error under the statistical setting.…”

Section: Related Literaturementioning

confidence: 82%

See 1 more Smart Citation

ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching

Zhang¹,

Luo²,

Raskutti³

et al. 2019

Preprint

View full text Add to dashboard Cite

In this paper, we develop a novel procedure for low-rank tensor regression, namely Importance Sketching Low-rank Estimation for Tensors (ISLET). The central idea behind ISLET is importance sketching, i.e., carefully designed sketches based on both the responses and low-dimensional structure of the parameter of interest. We show that the proposed method is sharply minimax optimal in terms of the mean-squared error under low-rank Tucker assumptions and under randomized Gaussian ensemble design. In addition, if a tensor is low-rank with group sparsity, our procedure also achieves minimax optimality. Further, we show through numerical studies that ISLET achieves comparable or better mean-squared error performance to existing state-of-the-art methods whilst having substantial storage and run-time advantages including capabilities for parallel and distributed computing. In particular, our procedure performs reliable estimation with tensors of dimension p = O(10 8 ) and is 1 or 2 orders of magnitude faster than baseline methods.

show abstract

Section: Related Literaturementioning

confidence: 82%

“…[7,10] proposed iterative projection methods to solve large-scale linear systems with Kroneckerproduct-type design matrices. [48] introduced a greedy approach. [69,70] considered Riemannian optimization methods and tensor Krylov subspace methods, respectively.…”

Section: Related Literaturementioning

confidence: 99%

ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching

Zhang¹,

Luo²,

Raskutti³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…For large-scale problems sparse direct solvers eventually fail due to memory limitations. The methods described in [7] can then be considered.…”

Section: Numerical Resultsmentioning

confidence: 99%

Robust Preconditioners for Multiple Saddle Point Problems and Applications to Optimal Control Problems

Beigl¹,

Sogn²,

Zulehner³

2019

Preprint

View full text Add to dashboard Cite

In this paper we consider multiple saddle point problems with block tridiagonal Hessian in a Hilbert space setting. Well-posedness and the related issue of preconditioning are discussed. We give a characterization of all block structured norms which ensure well-posedness of multiple saddle point problems as a helpful tool for constructing block diagonal preconditioners. We subsequently apply our findings to a general class of PDE-constrained optimal control problems containing a regularization parameter α and derive α-robust preconditioners for the corresponding optimality systems. Finally, we demonstrate the generality of our approach with two optimal control problems related to the heat and the wave equation, respectively. Preliminary numerical experiments support the feasibility of our method.

show abstract

“…For example, Boussé et al (2018) developed the algebraic method and Gauss-Newton method to solve the linear system with a CP low-rank tensor solution. Georgieva and Hofreither (2019) and Kressner et al (2016) respectively introduced a greedy approach and an approximate Riemannian Newton method to approximate the linear system by a low Tucker rank tensor. The readers are also referred to Grasedyck et al (2013) for a recent survey.…”

Section: Related Literaturementioning

confidence: 99%

Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence

Luo

Zhang²

2021

Preprint

View full text Add to dashboard Cite

In this paper, we consider the estimation of a low Tucker rank tensor from a number of noisy linear measurements. The general problem covers many specific examples arising from applications, including tensor regression, tensor completion, and tensor PCA/SVD. We propose a Riemannian Gauss-Newton (RGN) method with fast implementations for low Tucker rank tensor estimation. Different from the generic (super)linear convergence guarantee of RGN in the literature, we prove the first quadratic convergence guarantee of RGN for low-rank tensor estimation under some mild conditions. A deterministic estimation error lower bound, which matches the upper bound, is provided that demonstrates the statistical optimality of RGN. The merit of RGN is illustrated through two machine learning applications: tensor regression and tensor SVD. Finally, we provide the simulation results to corroborate our theoretical findings.

show abstract

Greedy low-rank approximation in Tucker format of solutions of tensor linear systems

Cited by 15 publications

References 37 publications

ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching

ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching

Robust Preconditioners for Multiple Saddle Point Problems and Applications to Optimal Control Problems

Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence

Contact Info

Product

Resources

About