Recent Theoretical Advances in Non-Convex Optimization

Danilova, Marina; Dvurechensky, Pavel; Gasnikov, Alexander; Gorbunov, Eduard; Guminov, Sergey; Kamzolov, Dmitry; Shibaev, Innokentiy

doi:10.1007/978-3-031-00832-0_3

Cited by 32 publications

(12 citation statements)

References 111 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the last approximation is valid for n ≫ 1. Thus, the probability to find beneficial moves, P(U i > 0|D) ≃ e −n/N , decreases exponentially with n. Note that if n = 0 (no random trials have been made), equation (8) yields P(U i > 0|D) = N /(N + 1), consistent with the prior probability in equation ( 3) which assigns equal weights to all N + 1 values of U i . Thus, in the beginning the system is very optimistic that a beneficial move will be found.…”

Section: Bayesian Estimation Of the Probability To Find A Novel Benef...supporting

confidence: 57%

“…The number of variables in X may be large in real-world applications and F(X) may be costly to evaluate, making it highly desirable to develop efficient global optimization strategies. In general, global optimization is an NP-hard problem [8] and there is no guarantee of finding the global maximum in a reasonable number of steps. Here, we focus on systems with discrete states (such as spin glasses [9,10]) for which the analytical gradient of the fitness function is not available.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An adaptive Bayesian approach to gradient-free global optimization

Yu,

Morozov

2024

New J. Phys.

View full text Add to dashboard Cite

Many problems in science and technology require finding global minima or maxima of complicated objective functions. The importance of global optimization has inspired the development of numerous heuristic algorithms based on analogies with physical, chemical or biological systems. Here we present a novel algorithm, SmartRunner, which employs a Bayesian probabilistic model informed by the history of accepted and rejected moves to make an informed decision about the next random trial. Thus, SmartRunner intelligently adapts its search strategy to a given objective function and moveset, with the goal of maximizing fitness gain (or energy loss) per function evaluation. Our approach is equivalent to adding a simple adaptive penalty to the original objective function, with SmartRunner performing hill ascent on the modified landscape. The adaptive penalty can be added to many other global optimization schemes, enhancing their ability to find high-quality solutions. We have explored SmartRunner's performance on a standard set of test functions, the Sherrington-Kirkpatrick (SK) spin glass model, and Kauffman's NK fitness model, finding that it compares favorably with several widely-used alternative approaches to gradient-free optimization.

show abstract

Section: Bayesian Estimation Of the Probability To Find A Novel Benef...supporting

confidence: 57%

Section: Introductionmentioning

confidence: 99%

An adaptive Bayesian approach to gradient-free global optimization

Yu,

Morozov

2024

New J. Phys.

View full text Add to dashboard Cite

show abstract

“…This inequality is sometimes sufficient by itself for convergence analysis of optimization methods and is often exploited, for example, in the context of weakly convex optimization (see, e.g. [2,6] and the references therein). In [19], a natural extension of this inequality to the case of nonsmooth hypodifferentiable convex functions was proposed and studied.…”

Section: Lipschitzian Approximations and Lipschitz Continuous Hypodif...mentioning

confidence: 99%

Hypodifferentials of nonsmooth convex functions and their applications to nonsmooth convex optimization

Долгополик¹

2023

Preprint

View full text Add to dashboard Cite

A hypodifferential is a compact family of affine mappings that defines a local max-type approximation of a nonsmooth convex function. We present a general theory of hypodifferentials of nonsmooth convex functions defined on a Banach space. In particular, we provide a complete characterization of hypodifferentiability and hypodifferentials of nonsmooth convex functions, derive calculus rules for hypodifferentials, and study the Lipschitz continuity/Lipschitz approximation property of hypodifferentials that can be viewed as a natural extension of the Lipschitz continuity of the gradient to the general nonsmooth setting. As an application of our theoretical results, we study the rate of convergence of several versions of the method of hypodifferential descent for nonsmooth convex optimization and present an accelerated version of this method having the faster rater of convergence O(1/k 2 ).

show abstract

“…However, some of the projections involve nonconvex sets. Hence, the algorithm becomes stuck in local minima 12 . Although other methods have been developed to overcome these limitations by using the tools of modern optimization theory, including semi-definite programming-based (SDP) approaches [13][14][15] , regularization-based methods 16,17 , global optimization methods 18 and Wirtinger flow and its variants 19 , the computational complexity of these PR algorithms is large, and it is time-consuming to converge to a solution with high confidence.…”

Section: Introductionmentioning

confidence: 99%

Ultrafast Bragg Coherent Diffraction Imaging of Epitaxial Thin Films using Deep Complex-valued Neural Networks

Lin

et al. 2023

Preprint

View full text Add to dashboard Cite

Domain wall structures form spontaneously due to epitaxial misfit during thin film growth. Imaging the dynamics of domains and domain walls at ultrafast timescales can provide fundamental clues to features that impact electrical transport in electronic devices. Recently, deep learning-based methods showed promising phase retrieval (PR) performance, allowing intensity-only measurements to be transformed into snapshot real space images. While the Fourier imaging model involves complex-valued quantities, most existing deep learning-based methods solve the PR problem with real-valued-based models, where the connection between amplitude and phase is ignored. To this end, we involve complex numbers operation in the neural network to preserve the amplitude and phase connection. Therefore, we employ the complex-valued neural network for solving the PR problem and evaluate it on Bragg coherent diffraction data streams collected from an epitaxial La2-xSrxCuO4 (LSCO) thin film using an X-ray Free Electron Laser (XFEL). Our proposed complex-valued neural network-based approach outperforms the traditional real-valued neural network methods in both supervised and unsupervised learning manner. Phase domains are also observed from the LSCO thin film at an ultrafast timescale using the complex-valued neural network.

show abstract

Recent Theoretical Advances in Non-Convex Optimization

Cited by 32 publications

References 111 publications

An adaptive Bayesian approach to gradient-free global optimization

An adaptive Bayesian approach to gradient-free global optimization

Hypodifferentials of nonsmooth convex functions and their applications to nonsmooth convex optimization

Ultrafast Bragg Coherent Diffraction Imaging of Epitaxial Thin Films using Deep Complex-valued Neural Networks

Contact Info

Product

Resources

About