Delaunay-Triangulation-Based Learning With Hessian Total-Variation Regularization

Abstract

Regression is one of the core problems tackled in supervised learning. Neural networks with rectified linear units generate continuous and piecewise-linear (CPWL) mappings and are the state-of-the-art approach for solving regression problems. In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. In contrast to deep neural networks, our CPWL parameterization guarantees stability and is interpretable. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. The function values at the vertices of the triangulation are our learnable parameters and identify the CPWL function uniquely. Formulating the learning scheme as a variational problem, we use the Hessian total variation (HTV) as a regularizer to favor CPWL functions with few affine pieces. In this way, we control the complexity of our model through a single hyperparameter. By developing a computational framework to compute the HTV of any CPWL function parameterized by a triangulation, we discretize the learning problem as the generalized least absolute shrinkage and selection operator. Our experiments validate the usage of our method in low-dimensional scenarios.

Functions with Bounded Hessian–Schatten Variation: Density, Variational, and Extremality Properties

“…This is a variant of the classical second-order total variation ( [5]). It has been inspired by [6,[14][15][16][17] and used in [10,18]; c) in the critical case d = 2 we consider as linear forward operator the evaluation functional at certain points x 1 , . .…”