Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Campos, Joaquim; Aziznejad, Shayan; Unser, Michaël

doi:10.1109/ojsp.2021.3136488

Cited by 8 publications

(10 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, HTV regularization allows direct control over the complexity of the learned mapping. This concept was put into practice in [31], albeit in the restricted setting d = 2 (two input variables). The CPWL functions in [31] are parameterized through box splines on a hexagonal lattice.…”

Section: B Variational Learning Of Cpwl Functionsmentioning

confidence: 99%

“…This concept was put into practice in [31], albeit in the restricted setting d = 2 (two input variables). The CPWL functions in [31] are parameterized through box splines on a hexagonal lattice. The extension of this uniformly gridded framework to higher dimensions is computationally restricted due to the exponential growth of the number of grid points with the dimension.…”

Section: B Variational Learning Of Cpwl Functionsmentioning

confidence: 99%

“…In two dimensions, we validate our framework by comparing it to [31] (HTV-Box). We learn a R 2 → R mapping using data from a face-height map 3 corrupted with additive Gaussian noise.…”

Section: B Comparison With [31]mentioning

confidence: 99%

“…DHTV-Hex and HTV-Box have the same triangulation; therefore, they induce the same CPWL parameterization. To solve the learning task for DHTV-Hex and DHTV-Id, we use Algorithm 1 FIGURE 5: Loss of the learning task as implemented through Algorithm 1 versus the scheme used in [31].…”

Section: B Comparison With [31]mentioning

confidence: 99%

See 3 more Smart Citations

Delaunay-Triangulation-Based Learning With Hessian Total-Variation Regularization

Pourya

Goujon

Unser

2023

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

Regression is one of the core problems tackled in supervised learning. Neural networks with rectified linear units generate continuous and piecewise-linear (CPWL) mappings and are the state-of-the-art approach for solving regression problems. In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. In contrast to deep neural networks, our CPWL parameterization guarantees stability and is interpretable. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. The function values at the vertices of the triangulation are our learnable parameters and identify the CPWL function uniquely. Formulating the learning scheme as a variational problem, we use the Hessian total variation (HTV) as a regularizer to favor CPWL functions with few affine pieces. In this way, we control the complexity of our model through a single hyperparameter. By developing a computational framework to compute the HTV of any CPWL function parameterized by a triangulation, we discretize the learning problem as the generalized least absolute shrinkage and selection operator. Our experiments validate the usage of our method in low-dimensional scenarios.

show abstract

Section: B Variational Learning Of Cpwl Functionsmentioning

confidence: 99%

Section: B Variational Learning Of Cpwl Functionsmentioning

confidence: 99%

“…In two dimensions, we validate our framework by comparing it to [31] (HTV-Box). We learn a R 2 → R mapping using data from a face-height map 3 corrupted with additive Gaussian noise.…”

Section: B Comparison With [31]mentioning

confidence: 99%

Section: B Comparison With [31]mentioning

confidence: 99%

See 2 more Smart Citations

Delaunay-Triangulation-Based Learning With Hessian Total-Variation Regularization

Pourya

Goujon

Unser

2023

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Throughout this section, we assume that the vertices coincide with the sites of a lattice Λ. This is natural in some applications, such as image processing [41], but can also be a sensible choice in low dimensional learning problems [42]. While a uniform grid constrains the model and thereby reduces its expressivity, it significantly improves the computational performance.…”

Section: Uniform Settingmentioning

confidence: 99%

Stable Parametrization of Continuous and Piecewise-Linear Functions

Goujon¹,

Campos²,

Unser³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function spaces lacks stability. In this paper, we investigate an alternative representation of CPWL functions that relies on local hat basis functions. It is predicated on the fact that any CPWL function can be specified by a triangulation and its values at the grid points. We give the necessary and sufficient condition on the triangulation (in any number of dimensions) for the hat functions to form a Riesz basis, which ensures that the link between the parameters and the corresponding CPWL function is stable and unique. In addition, we provide an estimate of the 2 → L2 condition number of this local representation. Finally, as a special case of our framework, we focus on a systematic parametrization of R d with control points placed on a uniform grid. In particular, we choose hat basis functions that are shifted replicas of a single linear box spline. In this setting, we prove that our general estimate of the condition number is optimal. We also relate our local representation to a nonlocal one based on shifts of a causal ReLU-like function.

show abstract

Functions with Bounded Hessian–Schatten Variation: Density, Variational, and Extremality Properties

Ambrosio,

Brena,

Conti

2023

Arch Rational Mech Anal

View full text Add to dashboard Cite

In this paper we analyze in detail a few questions related to the theory of functions with bounded p-Hessian–Schatten total variation, which are relevant in connection with the theory of inverse problems and machine learning. We prove an optimal density result, relative to the p-Hessian–Schatten total variation, of continuous piecewise linear (CPWL) functions in any space dimension d, using a construction based on a mesh whose local orientation is adapted to the function to be approximated. We show that not all extremal functions with respect to the p-Hessian–Schatten total variation are CPWL. Finally, we prove the existence of minimizers of certain relevant functionals involving the p-Hessian–Schatten total variation in the critical dimension $$d=2$$ d = 2 .

show abstract

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Cited by 8 publications

References 56 publications

Delaunay-Triangulation-Based Learning With Hessian Total-Variation Regularization

Delaunay-Triangulation-Based Learning With Hessian Total-Variation Regularization

Stable Parametrization of Continuous and Piecewise-Linear Functions

Functions with Bounded Hessian–Schatten Variation: Density, Variational, and Extremality Properties

Contact Info

Product

Resources

About