Composite Optimization by Nonconvex Majorization-Minimization

Geiping, Jonas; Moeller, Michael

doi:10.1137/18m1171989

Cited by 17 publications

(10 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a linearized variant of the parametric majorization bound and as such a nonconvex composite majorizer in the sense of [39], as such a key property of majorizationminimization techniques remains in the parametrized setting, choosingx i = x i (θ k ):…”

Section: Iterative Majorizersmentioning

confidence: 99%

Parametric Majorization for Data-Driven Energy Minimization Methods

Geiping

Moeller

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Energy minimization methods are a classical tool in a multitude of computer vision applications. While they are interpretable and well-studied, their regularity assumptions are difficult to design by hand. Deep learning techniques on the other hand are purely data-driven, often provide excellent results, but are very difficult to constrain to predefined physical or safety-critical models. A possible combination between the two approaches is to design a parametric energy and train the free parameters in such a way that minimizers of the energy correspond to desired solution on a set of training examples. Unfortunately, such formulations typically lead to bi-level optimization problems, on which common optimization algorithms are difficult to scale to modern requirements in data processing and efficiency. In this work, we present a new strategy to optimize these bi-level problems. We investigate surrogate single-level problems that majorize the target problems and can be implemented with existing tools, leading to efficient algorithms without collapse of the energy function. This framework of strategies enables new avenues to the training of parameterized energy minimization models from large data.

show abstract

Section: Iterative Majorizersmentioning

confidence: 99%

Parametric Majorization for Data-Driven Energy Minimization Methods

Geiping

Moeller

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In practice, sequentially minimizing convex problems with weighted-1 priors is indeed much simpler than minimizing a non-convex problem with a log-sum prior. From a convergence point of view, multiple works have recently shown that the set of minimizers resulting from a reweighted-1 procedure coincides with the one obtained by minimizing a problem with log-sum prior (Ochs et al 2015;Geiping & Moeller 2018;Ochs et al 2019;. In the following paragraphs, the SARA (Carrillo et al 2012) and HyperSARA (Abdulaziz et al 2019b) log-sum priors are presented, considered as benchmarks to assess the proposed spatio-spectral faceted prior.…”

Section: State-of-the-art Average Sparsity Priorsmentioning

confidence: 97%

Parallel faceted imaging in radio interferometry via proximal splitting (Faceted HyperSARA): I. Algorithm and simulations

Thouvenin¹,

Abdulaziz²,

Dabbech³

et al. 2020

Preprint

View full text Add to dashboard Cite

Upcoming radio interferometers are aiming to image the sky at new levels of resolution and sensitivity, with wide-band image cubes reaching close to the Petabyte scale for SKA. Modern proximal optimization algorithms have shown a potential to significantly outperform CLEAN thanks to their ability to inject complex image models to regularize the inverse problem for image formation from visibility data. They were also shown to be scalable to large data volumes thanks to a splitting functionality enabling the decomposition of data into blocks, for parallel processing of block-specific data-fidelity terms of the objective function. In this work, the splitting functionality is further exploited to decompose the image cube into spatio-spectral facets, and enable parallel processing of facet-specific regularization terms in the objective. The resulting "Faceted HyperSARA" algorithm is implemented in MATLAB (code available on GitHub). Simulation results on synthetic image cubes confirm that faceting can provide a major increase in scalability at no cost in imaging quality. A proof-of-concept reconstruction of a 15 GB image of Cyg A from 7.4 GB of VLA data, utilizing 496 CPU cores on a HPC system for 68 hours, confirms both scalability and a quantum jump in imaging quality from CLEAN. Assuming slow spectral slope of Cyg A, we also demonstrate that Faceted HyperSARA can be combined with a dimensionality reduction technique, enabling utilizing only 31 CPU cores for 142 hours to form the Cyg A image from the same data, while preserving reconstruction quality. Cyg A reconstructed cubes are available online.

show abstract

“…As proposed by the reweighting framework, problem ( 7) is solved several times, considering different weighting matrices W, in order to mimic the 0 pseudo-norm. Note that the reweighting framework is underpinned by recent theoretical results [47][48][49][50] showing that sequentially minimizing convex problems with weighted-1 priors corresponds to solving for a critical point of a non-convex problem with a log-sum prior, itself a close approximation to the target 0 prior. Considering one weighting cycle, a simple algorithm to solve problem (7) is the FB algorithm [51].…”

Section: Algorithm For Constrained Weighted 1 Minimizationmentioning

confidence: 99%

Fast Fiber Orientation Estimation in Diffusion MRI from kq-Space Sampling and Anatomical Priors

et al. 2021

View full text Add to dashboard Cite

High spatio-angular resolution diffusion MRI (dMRI) has been shown to provide accurate identification of complex neuronal fiber configurations, albeit, at the cost of long acquisition times. We propose a method to recover intra-voxel fiber configurations at high spatio-angular resolution relying on a 3D kq-space under-sampling scheme to enable accelerated acquisitions. The inverse problem for the reconstruction of the fiber orientation distribution (FOD) is regularized by a structured sparsity prior promoting simultaneously voxel-wise sparsity and spatial smoothness of fiber orientation. Prior knowledge of the spatial distribution of white matter, gray matter, and cerebrospinal fluid is also leveraged. A minimization problem is formulated and solved via a stochastic forward–backward algorithm. Simulations and real data analysis suggest that accurate FOD mapping can be achieved from severe kq-space under-sampling regimes potentially enabling high spatio-angular resolution dMRI in the clinical setting.

show abstract

Composite Optimization by Nonconvex Majorization-Minimization

Cited by 17 publications

References 63 publications

Parametric Majorization for Data-Driven Energy Minimization Methods

Parametric Majorization for Data-Driven Energy Minimization Methods

Parallel faceted imaging in radio interferometry via proximal splitting (Faceted HyperSARA): I. Algorithm and simulations

Fast Fiber Orientation Estimation in Diffusion MRI from kq-Space Sampling and Anatomical Priors

Contact Info

Product

Resources

About