Understanding neural networks with reproducing kernel Banach spaces

Bartolucci, Francesca; Vito, Ernesto De; Rosasco, Lorenzo; Vigogna, Stefano

doi:10.48550/arxiv.2109.09710

Cited by 6 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As second example, we strengthen the representation results for 2-layer ReLU networks established by Parhi and Nowak [18] and Bartolucci et al [1], which builds up on the univariate case investigated in [25]. The involved norm was also studied from a theoretical point of view in [17].…”

Section: Introductionsupporting

confidence: 80%

“…Compared to previous results in the literature [1,18], this leads to a stronger characterization of the solution set S together with a nice and elegant proof.…”

Section: Radon Splinesmentioning

confidence: 69%

“…The native Banach space for interpolation with Radon splines given some order m ∈ N is the space M Rad,m (R d ) introduced in Section 4.2. Due to the form of the function ρ Rad,m , this problem is closely related to approximation with 2-layer neural networks as pointed out in [1,18]. Since the point evaluations are in the predual, applying Theorem 6 together with the explicit representation of elements in M Rad,m (R d ) obtained in Corollary 3 yields the following.…”

Section: Radon Splinesmentioning

confidence: 99%

“…By applying Theorem 3, we can get a representation operator for α > d and α−d / ∈ N: Let ρ Liz,α := (−∆) −α/2 {δ} ∈ M α (R d ) with a concrete continuous representation given by (1). Using this representation, we then obtain ρ Liz,α…”

Section: Example 1: Fractional Laplaciansmentioning

confidence: 99%

“…While both mentioned works rely on the very general result on existence of sparse solutions for variational problems by Bredies and Carioni [4] in their proofs, we are additionally able to identify the whole solution set as the weak* closure of sparse solutions of a given form. Similarly as [1], our construction is related to reproducing kernel Banach spaces [5,14,37], since our representation operator is constructed in terms of a kernel and the point evaluations are continuous. The key ingredient that enables us to strengthen the results of these prior works is that we actually construct a predual space for the optimization domain, which enables us to use our proposed variational framework.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Explicit representations for Banach subspaces of Lizorkin distributions

Neumayer¹,

Unser²

2022

Preprint

View full text Add to dashboard Cite

The Lizorkin space is well-suited for studying various operators; e.g., fractional Laplacians and the Radon transform. In this paper, we show that the space is unfortunately not complemented in the Schwartz space. However, we can show that it is dense in C 0 (R d ), a property that is shared by the larger Schwartz space and that turns out to be useful for applications. Based on this result, we investigate subspaces of Lizorkin distributions that are Banach spaces and for which a continuous representation operator exists. Then, we introduce a variational framework involving these spaces and that makes use of the constructed operator. By investigating two particular cases of this framework, we are able to strengthen existing results for fractional splines and 2-layer ReLU networks.

show abstract

Section: Introductionsupporting

confidence: 80%

“…Compared to previous results in the literature [1,18], this leads to a stronger characterization of the solution set S together with a nice and elegant proof.…”

Section: Radon Splinesmentioning

confidence: 69%

Section: Radon Splinesmentioning

confidence: 99%

Section: Example 1: Fractional Laplaciansmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Explicit representations for Banach subspaces of Lizorkin distributions

Neumayer¹,

Unser²

2022

Preprint

View full text Add to dashboard Cite

show abstract

Stability of Image-Reconstruction Algorithms

Aguila

Neumayer

Unser

2023

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularization strategies ( 2 and 1 regularization) and present novel stability results for p -regularized linear inverse problems for p ∈ (1, ∞). Our results guarantee Lipschitz continuity for small p and Hölder continuity for larger p. They generalize well to the L p (Ω) function spaces.

show abstract

TRF: Learning Kernels with Tuned Random Features

Shilton

Gupta

Rana

et al. 2022

AAAI

View full text Add to dashboard Cite

Random Fourier features (RFF) are a popular set of tools for constructing low-dimensional approximations of translation-invariant kernels, allowing kernel methods to be scaled to big data. Apart from their computational advantages, by working in the spectral domain random Fourier features expose the translation invariant kernel as a density function that may, in principle, be manipulated directly to tune the kernel. In this paper we propose selecting the density function from a reproducing kernel Hilbert space to allow us to search the space of all translation-invariant kernels. Our approach, which we call tuned random features (TRF), achieves this by approximating the density function as the RKHS-norm regularised least-squares best fit to an unknown ``true'' optimal density function, resulting in a RFF formulation where kernel selection is reduced to regularised risk minimisation with a novel regulariser. We derive bounds on the Rademacher complexity for our method showing that our random features approximation method converges to optimal kernel selection in the large N,D limit. Finally, we prove experimental results for a variety of real-world learning problems, demonstrating the performance of our approach compared to comparable methods.

show abstract

Understanding neural networks with reproducing kernel Banach spaces

Cited by 6 publications

References 34 publications

Explicit representations for Banach subspaces of Lizorkin distributions

Explicit representations for Banach subspaces of Lizorkin distributions

Stability of Image-Reconstruction Algorithms

TRF: Learning Kernels with Tuned Random Features

Contact Info

Product

Resources

About