Bundle methods for dual atomic pursuit

Abstract

The aim of structured optimization is to assemble a solution, using a given set of (possibly uncountably infinite) atoms, to fit a model to data. A two-stage algorithm based on gauge duality and bundle method is proposed. The first stage discovers the optimal atomic support for the primal problem by solving a sequence of approximations of the dual problem using a bundle-type method. The second stage recovers the approximate primal solution using the atoms discovered in the first stage. The overall approach leads to implementable and efficient algorithms for large problems.

Cardinality-constrained structured data-fitting problems

“…Many recent approaches for atomic-sparse optimization problems are based on algorithms [23,29]. These methods, however, still need to retrieve at some point a primal solution x, which may require a prohibitive amount of memory for its storage.…”

“…Dual approaches for nuclear-or trace-norm regularized problems are attractive because they enjoy optimal storage, which means that they have space complexity O(m) instead of O(n 2 ) [23,32]. For example, the bundle method for solving the Lagrangian dual formulation of semi-definite programming [37], and the gauge dual formulation of general atomic sparse optimization problem [29], exhibit promising results in practice. Similarly, there are dual approaches for one-norm regularized problems that enjoy better convergence rates than primal approaches [2,43].…”

The dual approach to non-negative super-resolution: impact on primal reconstruction accuracy

“…Moreover, the constraints λ T Φptq ď 1, @t P I can be handled using an exact penalty approach, i.e. can be reformulated as thus making the problem amenable to non-smooth optimisation algorithms such as bundle methods [13,14].…”

“…As in the noise-free setting, we assume here that the dual solution λ˚forms a dual certificate, namely the function qpsq as defined in (12) satisfies conditions (13) and (14). Then, the subdifferential at λ˚has the form:…”

“…Given the functionF defined in (35), let J˚be its Jacobian with respect to the first variable, evaluated at prλ˚, as T , 0q and σ min pJ˚q its smallest singular value. We also assume that the solution λf orms a dual certificate, namely the function qptq defined in (12) satisfies conditions (13) and (14). If J˚is invertible and }w} 2 ď δ w , then:…”