Worst case complexity of direct search under convexity

Dodangeh, Mahdi; Vicente, Luís Nunes

doi:10.1007/s10107-014-0847-0

Cited by 27 publications

(49 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, one knows that in some convex instances [12] the worst case complexity bound of deterministic direct search can be improved to the order of 1/ (corresponding to a global rate of 1/k). One also knows that some deterministic direct-search methods exhibit an r-linear rate of local convergence [14].…”

Section: Open Issuesmentioning

confidence: 99%

Direct Search Based on Probabilistic Descent

Gratton¹,

Royer²,

Vicente³

et al. 2015

SIAM J. Optim.

Self Cite

126

View full text Add to dashboard Cite

Direct-search methods are a class of popular derivative-free algorithms characterized by evaluating the objective function using a step size and a number of (polling) directions. When applied to the minimization of smooth functions, the polling directions are typically taken from positive spanning sets which in turn must have at least n+1 vectors in an n-dimensional variable space. In addition, to ensure the global convergence of these algorithms, the positive spanning sets used throughout the iterations are required to be uniformly non-degenerate in the sense of having a positive (cosine) measure bounded away from zero.However, recent numerical results indicated that randomly generating the polling directions without imposing the positive spanning property can improve the performance of these methods, especially when the number of directions is chosen considerably less than n + 1.In this paper, we analyze direct-search algorithms when the polling directions are probabilistic descent, meaning that with a certain probability at least one of them is of descent type. Such a framework enjoys almost-sure global convergence. More interestingly, we will show a global decaying rate of 1/ √ k for the gradient size, with overwhelmingly high probability, matching the corresponding rate for the deterministic versions of the gradient method or of direct search. Our analysis helps to understand numerical behavior and the choice of the number of polling directions.

show abstract

Section: Open Issuesmentioning

confidence: 99%

Direct Search Based on Probabilistic Descent

Gratton¹,

Royer²,

Vicente³

et al. 2015

SIAM J. Optim.

Self Cite

126

View full text Add to dashboard Cite

show abstract

“…The first WCC bound, of O( −2 ), was derived by Vicente [32] for smooth functions, and later refined to O( −1 ) when the function is convex by Dodangeh and Vicente [17]. Garmanjani and Vicente [21], using a smoothing approach, have shown a WCC bound of O(| log | −3 ) in the non-smooth case.…”

Section: Worst Case Complexity In Dfomentioning

confidence: 99%

Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case

Garmanjani¹,

Júdice²,

Vicente³

2016

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

Trust-region methods are a broad class of methods for continuous optimization that found application in a variety of problems and contexts. In particular, they have been studied and applied for problems without using derivatives.The analysis of trust-region derivative-free methods has focused on global convergence, and they have been proved to generate a sequence of iterates converging to stationarity independently of the starting point. Most of such an analysis is carried out in the smooth case, and, moreover, little is known about the complexity or global rate of these methods.In this paper, we start by analyzing the worst case complexity of general trust-region derivative-free methods for smooth functions. For the non-smooth case, we propose a smoothing approach, for which we prove global convergence and bound the worst case complexity effort. For the special case of non-smooth functions that result of the composition of smooth and non-smooth/convex components, we show how to improve the existing results of the literature and make them applicable to the general methodology.

show abstract

“…Then the worst case complexity bounds obtained by Vicente [20] and Dodangeh and Vicente [9] can be stated as follows. …”

Section: Worst Case Complexity Of Direct Searchmentioning

confidence: 99%

“…In Section 2, we introduce the algorithmic details of direct search. Section 3 summarizes the WCC bounds established in [20,9]. Our main result is given in Section 4, where we prove the optimality of the factor involving n in the bounds O(n 2 ϵ −a ), with a ∈ {1, 2}.…”

Section: Introductionmentioning

confidence: 99%

“…When the forcing function is the square of the step size, it was proved in [20] that direct search drives the norm of the gradient of f below ϵ ∈ (0, 1) within at most O(n 2 ϵ −2 ) function evaluations. In the convex case, Dodangeh and Vicente [9] improved the bound in [20] to O(n 2 ϵ −1 ). Konečný and Richtárik [16] rederived these bounds, in a simpler way, but for a restricted version of direct search where step size increases are not allowed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the optimal order of worst case complexity of direct search

2015

Self Cite

View full text Add to dashboard Cite

The worst case complexity of direct-search methods has been recently analyzed when they use positive spanning sets and impose a sufficient decrease condition to accept new iterates. For smooth unconstrained optimization, it is now known that such methods require at most O(n 2 ϵ −2 ) function evaluations to compute a gradient of norm below ϵ ∈ (0, 1), where n is the dimension of the problem. Such a maximal effort is reduced to O(n 2 ϵ −1 ) if the function is convex. The factor n 2 has been derived using the positive spanning set formed by the coordinate vectors and their negatives at all iterations.In this paper, we prove that such a factor of n 2 is optimal in these worst case complexity bounds, in the sense that no other positive spanning set will yield a better order of n. The proof is based on an observation that reveals the connection between cosine measure in positive spanning and sphere covering.

show abstract

Worst case complexity of direct search under convexity

Cited by 27 publications

References 17 publications

Direct Search Based on Probabilistic Descent

Direct Search Based on Probabilistic Descent

Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case

On the optimal order of worst case complexity of direct search

Contact Info

Product

Resources

About