Adaptive coordinate descent

Loshchilov, Ilya; Schoenauer, Marc; Sebag, Michèle

doi:10.1145/2001576.2001697

Cited by 46 publications

(44 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case that some other direction is unconditionally required, there is always the possibility to rotate the coordinate system so that one of coordinate axes becomes coincident with the desired direction. This is, in fact, what happens in so-called adaptive coordinate descent method [9]. Hence, arbitrary search directions are possible, but with a more involved computation.…”

Section: Introductionmentioning

confidence: 89%

A Derivative-free Algorithm for Finding Least Squares Solutions of Quasi-linear and Linear Systems

Hlupić

Beroš

Basch

2013

CIT

View full text Add to dashboard Cite

A novel derivative-free algorithm for solving quasilinear systems is presented. It resembles "classical" optimization approach but greatly simplifies computation, resulting in fast execution and numerical stability. Though the global convergence cannot be guaranteed, it turns out that the presented algorithm finds a solution as successfully as other commonly accepted methods. The algorithm is clearly developed and mathematically founded, and its properties are examined by comparisons with other methods.

show abstract

Section: Introductionmentioning

confidence: 89%

A Derivative-free Algorithm for Finding Least Squares Solutions of Quasi-linear and Linear Systems

Hlupić

Beroš

Basch

2013

CIT

View full text Add to dashboard Cite

show abstract

“…An appropriate coordinate system can be used to accelerate the convergence. For example, the adaptive coordinate descent method takes principal component analysis to obtain a new coordinate system with as little correlation as possible between the coordinates [152]. The coordinate descent method still has limitations when performing on the non-smooth objective function, which may fall into a non-stationary point.…”

Section: Derivative-free Optimizationmentioning

confidence: 99%

A Survey of Optimization Methods From a Machine Learning Perspective

Sun

Cao

Han

et al. 2020

IEEE Trans. Cybern.

551

228

View full text Add to dashboard Cite

Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization methods in machine learning face more and more challenges. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and machine learning research. In this paper, we first describe the optimization problems in machine learning. Then, we introduce the principles and progresses of commonly used optimization methods. Next, we summarize the applications and developments of optimization methods in some popular machine learning fields. Finally, we explore and give some challenges and open problems for the optimization in machine learning.

show abstract

“…It is similar to several applications where the estimate of error in the previous iteration is used in deciding the move. Such algorithms are referred to as adaptive algorithms, such as the well-known 'adaptive coordinate descent method' [53] and the 'adaptive algorithms for deterministic and stochastic differential equations' [54].…”

Section: Adaptivitymentioning

confidence: 99%