Provably Faster Gradient Descent via Long Steps

Grimmer, Benjamin

doi:10.1137/23m1588408

SIAM J. Optim.

2024

DOI: 10.1137/23m1588408

|View full text |Cite

Provably Faster Gradient Descent via Long Steps

Benjamin Grimmer

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

2025

Publication Types

Select...

Article3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Acceleration by stepsize hedging: Silver Stepsize Schedule for smooth convex optimization

Altschuler,

Parrilo

2024

Math. Program.

View full text Add to dashboard Cite

We provide a concise, self-contained proof that the Silver Stepsize Schedule proposed in our companion paper directly applies to smooth (non-strongly) convex optimization. Specifically, we show that with these stepsizes, gradient descent computes an $$\varepsilon $$ ε -minimizer in $$O(\varepsilon ^{-\log _{\rho } 2}) = O(\varepsilon ^{-0.7864})$$ O ( ε - log ρ 2 ) = O ( ε - 0.7864 ) iterations, where $$\rho = 1+\sqrt{2}$$ ρ = 1 + 2 is the silver ratio. This is intermediate between the textbook unaccelerated rate $$O(\varepsilon ^{-1})$$ O ( ε - 1 ) and the accelerated rate $$O(\varepsilon ^{-1/2})$$ O ( ε - 1 / 2 ) due to Nesterov in 1983. The Silver Stepsize Schedule is a simple explicit fractal: the i-th stepsize is $$1 + \rho ^{\nu (i)-1}$$ 1 + ρ ν ( i ) - 1 where $$\nu (i)$$ ν ( i ) is the 2-adic valuation of i. The design and analysis are conceptually identical to the strongly convex setting in our companion paper, but simplify remarkably in this specific setting.

show abstract

Acceleration by stepsize hedging: Silver Stepsize Schedule for smooth convex optimization

Altschuler,

Parrilo

2024

Math. Program.

View full text Add to dashboard Cite

show abstract

A review of scalable and privacy-preserving multi-agent frameworks for distributed energy resources

Huo,

Huang,

Davis

et al. 2024

Advances in Applied Energy

View full text Add to dashboard Cite

Algebraic conditions for stability in Runge-Kutta methods and their certification via semidefinite programming

Juhl,

Shirokoff

2025

Applied Numerical Mathematics

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Provably Faster Gradient Descent via Long Steps

Cited by 3 publications

References 28 publications

Acceleration by stepsize hedging: Silver Stepsize Schedule for smooth convex optimization

Acceleration by stepsize hedging: Silver Stepsize Schedule for smooth convex optimization

A review of scalable and privacy-preserving multi-agent frameworks for distributed energy resources

Algebraic conditions for stability in Runge-Kutta methods and their certification via semidefinite programming

Contact Info

Product

Resources

About