Polyhedral DC Decomposition and DCA Optimization of Piecewise Linear Functions

Abstract

For piecewise linear functions f : R n ↦ R we show how their abs-linear representation can be extended to yield simultaneously their decomposition into a convex f and a concave part f ^ , including a pair of generalized gradients g ∈ R n ∋ g ^ . The latter satisfy strict chain rules and can be computed in the reverse mode of algorithmic differentiation, at a small multiple of the cost of evaluating f itself. It is shown how f and f ^ can be expressed as a single maximum and a single minimum of affine functions, respectively. The two subgradients g and − g ^ are then used to drive DCA algorithms, where the (convex) inner problem can be solved in finitely many steps, e.g., by a Simplex variant or the true steepest descent method. Using a reflection technique to update the gradients of the concave part, one can ensure finite convergence to a local minimizer of f, provided the Linear Independence Kink Qualification holds. For piecewise smooth objectives the approach can be used as an inner method for successive piecewise linearization.

Algorithmic Differentiation for Cloud Schemes

“…which was already observed in [GW20]. A multiplication of the inverse with P α from the left thus yields…”

Piecewise linear neural networks and deep learning

“…It is worth mentioning that the "max -min" and the absolute-value operators for PWL nonlinearity have been addressed in mathematical programming and functional analysis. For example, major attention is given to aspects of Lipschitz continuity, nondifferentiability, nonsmoothness and their algorithmic aspects, rather than to PWLNNs and data driven applications [67][68][69][70] . Letting f (x x x) be an arbitrary PWLNN with d distinct linear functions, the Lattice representation is formulated as…”

Special Issue “Nonsmooth Optimization in Honor of the 60th Birthday of Adil M. Bagirov”: Foreword by Guest Editors

“…In the article "Polyhedral DC Decomposition and DCA Optimization of Piecewise Linear Functions" by Andreas Griewank and Andrea Walther [15], the abs-linear representation of the piecewise linear functions is extended, yielding their DC decomposition as well as a pair of generalized gradients that can be computed using the reverse mode of algorithmic differentiation. The DC decomposition and two subgradients are used to drive DCA algorithms where the (convex) inner problem can be solved in a finite many iterations and the gradients of the concave part can be updated using a reflection technique.…”