Using negative curvature in solving nonlinear programs

Goldfarb, Donald; Mu, Cun; Wright, John; Zhou, Chaoxu

doi:10.1007/s10589-017-9925-6

Cited by 13 publications

(14 citation statements)

References 33 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because of this, a small-stepping descent method will not move far away from Σ 4θp0 . For concreteness, we will analyze a variant of the curvilinear search method [Gol80,GMWZ17], which moves in a linear combination of the negative gradient direction −g and a negative curvature direction −v. At the k-th iteration, the algorithm updates a (k+1) as…”

Section: Provable Algorithm For Sas Deconvolutionmentioning

confidence: 99%

Geometry and Symmetry in Short-and-Sparse Deconvolution

Kuo¹,

Zhang²,

Lau³

et al. 2020

SIAM Journal on Mathematics of Data Science

Self Cite

View full text Add to dashboard Cite

We study the Short-and-Sparse (SaS) deconvolution problem of recovering a short signal a0 and a sparse signal x0 from their convolution. We propose a method based on nonconvex optimization, which under certain conditions recovers the target short and sparse signals, up to a signed shift symmetry which is intrinsic to this model. This symmetry plays a central role in shaping the optimization landscape for deconvolution. We give a regional analysis, which characterizes this landscape geometrically, on a union of subspaces. Our geometric characterization holds when the length-p0 short signal a0 has shift coherence µ, and x0 follows a random sparsity model with sparsity rate θ ∈ c 1 p 0 ,Based on this geometry, we give a provable method that successfully solves SaS deconvolution with high probability.Because of this symmetry, we only expect to recover a 0 and x 0 up to a signed shift (see Figure 1). Our problem of interest can be stated more formally as: Problem 1.1 (Short-and-Sparse Deconvolution). Given the cyclic convolution y = a 0 * x 0 ∈ R n of a 0 ∈ R p0 short (p 0 n), and x 0 ∈ R n sparse, recover a 0 and x 0 , up to a scaled shift.Despite a long history and many applications, until recently very little algorithmic theory was available for SaS deconvolution. Much of this difficulty can be attributed to the scale-shift symmetry: natural convex relaxations fail, and nonconvex formulations exhibit a complicated optimization landscape, with many 1 arXiv:1901.00256v2 [eess.SP] 11 Apr 2019 ϕ ρ is strongly convex in this region, and it has a minimizer very close to s [a 0 ].Geometry near a single shift. To gain intuition into the properties of ϕ ρ , we first visualize this function in the vicinity of a single shift s [a 0 ] of the ground truth a 0 . In Figure 3, we plot the function value of ϕ ρ overwhere B 2 ,r (a) is a ball of radius r around a. We make two observations:• The objective function ϕ ρ is strongly convex on this neighborhood of s [a 0 ].• There is a local minimizer very close to s [a 0 ].Write x 0J = ι * J x 0 and (x 0 ) I\J = P I\J x 0 . We can further notice that

show abstract

Section: Provable Algorithm For Sas Deconvolutionmentioning

confidence: 99%

Geometry and Symmetry in Short-and-Sparse Deconvolution

Kuo¹,

Zhang²,

Lau³

et al. 2020

SIAM Journal on Mathematics of Data Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Second-order methods over a Riemannian manifold are known to be able to escape saddle points, for example, the trust region method [60], and the negative curvature method [52]. Recent works proposed to solve dictionary learning [42], and phase retrieval [40] using these methods, without any special initialization schemes.…”

Section: A Guaranteed First-order Optimization Algorithmmentioning

confidence: 99%

Multichannel Sparse Blind Deconvolution on the Sphere

Bresler

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Multichannel blind deconvolution is the problem of recovering an unknown signal f and multiple unknown channels x i from their circular convolution y i = x i f (i = 1, 2, . . . , N ). We consider the case where the x i 's are sparse, and convolution with f is invertible. Our nonconvex optimization formulation solves for a filter h on the unit sphere that produces sparse output y i h. Under some technical assumptions, we show that all local minima of the objective function correspond to the inverse filter of f up to an inherent sign and shift ambiguity, and all saddle points have strictly negative curvatures.This geometric structure allows successful recovery of f and x i using a simple manifold gradient descent (MGD) algorithm. Our theoretical findings are complemented by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods.

show abstract

“…B Giovanni Fasano fasano@unive.it Extended author information available on the last page of the article Despite the use of the term 'minimization' in the last problem, most of the methods proposed in the literature (for its solution) generate a sequence of points {x k }, which is only guaranteed to converge to stationary points. Thus, specific methods need to be applied in case stationary points for the above problem, satisfying also second-order necessary optimality conditions, are sought (see, for instance, the seminal papers [1][2][3][4][5][6][7] in the framework of truncated Newton methods). Observe that additional care when using the latter methods is definitely mandatory, since imposing standard first-order stationarity conditions may not in general ensure convexity of the quadratic model of the objective function, in a neighborhood of the solution points.…”

Section: Introductionmentioning

confidence: 99%

“…Observe that additional care when using the latter methods is definitely mandatory, since imposing standard first-order stationarity conditions may not in general ensure convexity of the quadratic model of the objective function, in a neighborhood of the solution points. In this regard, the computation of so-called negative curvature directions for the objective function is an essential tool (see also the recent papers [4,8]), to guarantee convergence to stationary points which satisfy second-order necessary conditions.…”

Section: Introductionmentioning

confidence: 99%

Iterative Grossone-Based Computation of Negative Curvature Directions in Large-Scale Optimization

Leone

Fasano

Roma

et al. 2020

J Optim Theory Appl

View full text Add to dashboard Cite

We consider an iterative computation of negative curvature directions, in large-scale unconstrained optimization frameworks, needed for ensuring the convergence toward stationary points which satisfy second-order necessary optimality conditions. We show that to the latter purpose, we can fruitfully couple the conjugate gradient (CG) method with a recently introduced approach involving the use of the numeral called Grossone. In particular, recalling that in principle the CG method is well posed only when solving positive definite linear systems, our proposal exploits the use of grossone to enhance the performance of the CG, allowing the computation of negative curvature directions in the indefinite case, too. Our overall method could be used to significantly generalize the theory in state-of-the-art literature. Moreover, it straightforwardly allows the solution of Newton's equation in optimization frameworks, even in nonconvex problems. We remark that our iterative procedure to compute a negative curvature direction does not require the storage of any matrix, simply needing to store a couple of vectors. This definitely represents an advance with respect to current results in the literature.

show abstract

Using negative curvature in solving nonlinear programs

Cited by 13 publications

References 33 publications

Geometry and Symmetry in Short-and-Sparse Deconvolution

Geometry and Symmetry in Short-and-Sparse Deconvolution

Multichannel Sparse Blind Deconvolution on the Sphere

Iterative Grossone-Based Computation of Negative Curvature Directions in Large-Scale Optimization

Contact Info

Product

Resources

About