A nonmonotone truncated Newton–Krylov method exploiting negative curvature directions, for large scale unconstrained optimization

Fasano, Giovanni; Lucidi, Stefano

doi:10.1007/s11590-009-0132-y

Cited by 28 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the ridge filter, problem (25) was solved by employing the truncated-Newton method reported in [7], terminating the algorithm when the sup-norm of the gradient of the objective function was less than or equal to 10 −5 . Table 1: Comparison between the values of the Adjusted Rand Index obtained by applying Single Linkage (SL), Expectation-Maximization for Gaussian mixture Models (EMGM) and Kernel K-Means (KKM) to the original data and to the filtered data.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Data filtering for cluster analysis by $$\ell _0$$ ℓ 0 -norm regularization

Cristofari

2017

Optim Lett

View full text Add to dashboard Cite

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted ℓ 0 -norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the ℓ 0 -norm. The convergence of the global minimum points of the approximating problems towards global minimum points of the original problem is stated. The proposed method also exploits a suitable technique to choose the penalty parameter. Numerical results on synthetic and real data sets are finally provided, showing how some existing clustering methods can take advantages from the proposed filtering strategy. Keywords. Zero-norm approximation Cluster analysis Nonlinear optimization.AMS subject classifications. 90C30. 62H30. 90C06. 49M15. MotivationCluster analysis is a branch of unsupervised learning, arising in many real-world applications and in different fields, e.g., biology, medicine, marketing, document retrieval, image segmentation and many others. It deals with grouping objects so that "alike" data are in the same clusters and "unlike" data are in different clusters. More formally, given a finite set of vectors X = {x 1 , . . . , x m } ⊂ R n , we want to divide X into k groups (clusters), according to a defined measure of similarity, where k can be either known or unknown.Partitioning X into a fixed number of clusters is known to be an NP-hard problem [9] and many existing clustering models are formulated as non-convex optimization problems. As a result, algorithms can generally find only approximate solutions. Moreover, there is no objectively "right" clustering model and the choice of the most suitable algorithm can strongly depend on the specific data set. So, there is still a great interest in developing new strategies for cluster analysis, also in the field of numerical optimization.Here, we propose a data filtering method based on combining two different techniques. The first one is a reformulation of the clustering problem as a penalized regression problem, proposed in [21,11,14] and further studied in [20,3,18]. Assuming that the number of clusters is unknown, this approach is based on introducing for each observation 1

show abstract

Section: Resultsmentioning

confidence: 99%

“…Since y / ∈ C, ξ ∈ (0, 1], and taking into account (7) of Lemma 1, we obtain that x − y 2 − x −ỹ 2 > 0. Lemma 3.…”

Section: Properties Of the Approximating Problemmentioning

confidence: 92%

Data filtering for cluster analysis by $$\ell _0$$ ℓ 0 -norm regularization

Cristofari

2017

Optim Lett

View full text Add to dashboard Cite

show abstract

“…In 1986, Grippo et al in [15] applied a nonmonotone globalization technique to Newton's method for solving unconstrained optimization problems with some success. Since then, a variety of proposals related to nonmonotone globalization techniques have been reported in the literature, see for example [5,7,18,19,25,26], among others. Nonmonotone filter methods have been used to promote global and fast local convergence for sequential quadratic programming algorithms [9,14].…”

Section: Introductionmentioning

confidence: 99%

Assessing the potential of interior point barrier filter line search methods: nonmonotoneversusmonotone approach

Costa

Fernandes

2011

Optimization

View full text Add to dashboard Cite

In this article, we present a numerical study of three nonmonotone filter line search techniques, as well as a three-dimensional filter approach, when incorporated into the solver IPOPT, a primal-dual barrier method developed by Wa¨chter and Biegler [On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program. 106 (2006), pp. 25-57] for nonlinear programming. Primary assessment of the proposals has been done with sets of small-and medium-scale problems and large-scale problems separately. Results show that the use of nonmonotone globalization strategies improves efficiency.

show abstract

“…Hence, similarly to Proposition 5.1, we can apply without loss of generality the linear transformation in (13) to F, in order to obtain the simplified hypersurfaceF in (14), with centre (x * ,x * 0 ) T in (15). Then, we carry on the proof by induction, recursively defining the linesˆ i , i = 1, .…”

mentioning

confidence: 99%

“…As an example, in [11,12] CG-based methods are used to yield superlinear convergence to an optimal solution of large scale unconstrained minimization problems. Within truncated Newton algorithms CG-based methods are also used to compute negative curvature directions for the objective function [13][14][15]. These directions turn out to be useful in proving the convergence of the algorithm to stationary points, along with the satisfaction of second order optimality conditions.…”

mentioning

confidence: 99%

Conjugate Direction Methods and Polarity for Quadratic Hypersurfaces

Fasano

Pesenti

2017

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

Abstract:We use some results from polarity theory to recast several geometric properties of Conjugate Gradient-based methods, for the solution of nonsingular symmetric linear systems. This approach allows us to pursue three main theoretical objectives. First, we can provide a novel geometric perspective on the generation of conjugate directions, in the context of positive definite systems. Second, we can extend the above geometric perspective to treat the generation of conjugate directions for handling indefinite linear systems. Third, by exploiting the geometric insight suggested by polarity theory, we can easily study the possible degeneracy (pivot breakdown) of Conjugate Gradientbased methods on indefinite linear systems. In particular, we prove that the degeneracy of the standard Conjugate Gradient on nonsingular indefinite linear systems can occur only once in the execution of the Conjugate Gradient. Once again we strongly appreciated the suggestions by the Editorial Board and the anonymous Referees, which definitely contributed to improve and enhance the paper. Hereafter we reply to the observations raised by each of them. We also remark that the paper has been given to a professional proofreader before resubmission: we hope we were able to comply with all the issues raised by the Reviewers. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems CorporationAnswer to the Editor in Chief and the Associate EditorAuthors: The instructions of the Editor/Associate Editor should have been followed, including both structural modifications (e.g. we splitted the former section of Conclusions) and minor suggestions.Answer to the First Referee Authors: All this Reviewer's comments should have been included, following her/his indications. In addition, an English native speaker has contributed with proofreading.Answer to the Second Referee Authors: All the accurate Reviewer's modifications should have been implemented, following her/his indications. Furthermore, we also did our best to revise the entire paper, on the basis of the specific comments of this Reviewer. This has led to some additional small changes, including punctuation, with respect to the previous version of the paper. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 2 Giovanni Fasano, Raffaele Pesenti by exploiting the geometric insight suggested by polarity theory, we can easily study the possible degeneracy (pivot breakdown) of Conjugate Gradient-based methods on indefinite linear systems. In particular, we prove that the degeneracy of the standard Conjugate Gradient on nonsingular indefinite linear systems can occur only once in the execution of the Conjugate Gradient. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation

show abstract

A nonmonotone truncated Newton–Krylov method exploiting negative curvature directions, for large scale unconstrained optimization

Cited by 28 publications

References 20 publications

Data filtering for cluster analysis by $$\ell _0$$ ℓ 0 -norm regularization

Data filtering for cluster analysis by $$\ell _0$$ ℓ 0 -norm regularization

Assessing the potential of interior point barrier filter line search methods: nonmonotoneversusmonotone approach

Conjugate Direction Methods and Polarity for Quadratic Hypersurfaces

Contact Info

Product

Resources

About