Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization

Mukkamala, Mahesh Chandra; Ochs, Peter; Pock, Thomas; Sabach, Shoham

doi:10.48550/arxiv.1904.03537

Cited by 6 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The introduction of the h-smoothness assumption in [3] has provided a way to adapt the Legendre kernel to the geometry of the objective function f and thus extend the domain of application of the Bregman Gradient method. Subsequent work has focused on nonconvex extensions [8], linear convergence rates under additional assumptions [24,2], and inertial variants [20,27].…”

Section: Related Workmentioning

confidence: 99%

Optimal Complexity and Certification of Bregman First-Order Methods

Dragomir,

Taylor,

d'Aspremont

et al. 2019

Preprint

View full text Add to dashboard Cite

We provide a lower bound showing that the O(1/k) convergence rate of the NoLips method (a.k.a. Bregman Gradient or Mirror Descent) is optimal for the class of functions satisfying the h-smoothness assumption. This assumption, also known as relative smoothness, appeared in the recent developments around the Bregman Gradient method, where acceleration remained an open issue.On the way, we show how to constructively obtain the corresponding worst-case functions by extending the computer-assisted performance estimation framework of Drori and Teboulle (Mathematical Programming, 2014) to Bregman first-order methods, and to handle the classes of differentiable and strictly convex functions.

show abstract

Section: Related Workmentioning

confidence: 99%

Optimal Complexity and Certification of Bregman First-Order Methods

Dragomir,

Taylor,

d'Aspremont

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…. , L N are not available, one can retrieve them adaptively by applying a backtracking linesearch starting from a lower estimates; see, e.g., [1,2,44,46,56].…”

Section: Adaptive Bpalmmentioning

confidence: 99%

Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization

Ahookhosh,

Hien,

Gillis

et al. 2019

Preprint

View full text Add to dashboard Cite

We introduce and analyze BPALM and A-BPALM, two multi-block proximal alternating linearized minimization algorithms using Bregman distances for solving structured nonconvex problems. The objective function is the sum of a multi-block relatively smooth function (i.e., relatively smooth by fixing all the blocks except one) and block separable (nonsmooth) nonconvex functions. It turns out that the sequences generated by our algorithms are subsequentially convergent to critical points of the objective function, while they are globally convergent under KL inequality assumption. Further, the rate of convergence is further analyzed for functions satisfying the Łojasiewicz's gradient inequality. We apply this framework to orthogonal nonnegative matrix factorization (ONMF) that satisfies all of our assumptions and the related subproblems are solved in closed forms, where some preliminary numerical results is reported.

show abstract

“…We provided two quartic kernels, including the novel Gram kernel, and demonstrated their benefit on numerical experiments. In future work, performance could be improved further by studying inertial variants [33,21]. New kernels could also be explored beyond the class of quartic functions to tackle other problems with inherent non-Euclidean geometries.…”

Section: Discussionmentioning

confidence: 99%

“…The iteration map is the basic brick for non-Euclidean methods à la Bregman. The simplest method is NoLips [3] and its extension Dyn-NoLips, described in Algorithm 1, but other possibilities exist using momentum and acceleration ideas [2,21,27,33].…”

Section: A Family Of Non-euclidean Geometries For Low Rank Minimizati...mentioning

confidence: 99%

See 1 more Smart Citation

Quartic First-Order Methods for Low-Rank Minimization

Dragomir,

d'Aspremont,

Bolte

2019

Preprint

View full text Add to dashboard Cite

We study a generalized nonconvex Burer-Monteiro formulation for low-rank minimization problems. We use recent results on non-Euclidean first order methods to provide efficient and scalable algorithms. Our approach uses geometries induced by quartic kernels on matrix spaces; for unconstrained cases we introduce a novel family of Gram kernels that considerably improves numerical performances. Numerical experiments for Euclidean distance matrix completion and symmetric nonnegative matrix factorization show that our algorithms scale well and reach state of the art performance when compared to specialized methods. * Last two authors are listed in alphabetical order.Preprint. Under review.

show abstract

Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization

Cited by 6 publications

References 0 publications

Optimal Complexity and Certification of Bregman First-Order Methods

Optimal Complexity and Certification of Bregman First-Order Methods

Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization

Quartic First-Order Methods for Low-Rank Minimization

Contact Info

Product

Resources

About