Introduction to Online Convex Optimization

Abstract

Lagrangian relaxation and approximate optimization algorithms have received much attention in the last two decades. Typically, the running time of these methods to obtain a ε approximate solution is proportional to 1 ε 2 . Recently, Bienstock and Iyengar, following Nesterov, gave an algorithm for fractional packing linear programs which runs in 1 ε iterations. The latter algorithm requires to solve a convex quadratic program every iteration -an optimization subroutine which dominates the theoretical running time.We give an algorithm for convex programs with strictly convex constraints which runs in time proportional to 1 ε . The algorithm does not require to solve any quadratic program, but uses gradient steps and elementary operations only. Problems which have strictly convex constraints include maximum entropy frequency estimation, portfolio optimization with loss risk constraints, and various computational problems in signal processing.As a side product, we also obtain a simpler version of Bienstock and Iyengar's result for general linear programming, with similar running time.We derive these algorithms using a new framework for deriving convex optimization algorithms from online game playing algorithms, which may be of independent interest.

Introduction to Multi-Armed Bandits

“…To keep the book manageable, and also more accessible, we chose not to dwell on the deep connections to online convex optimization. A modern treatment of this fascinating subject can be found, e.g., in the recent textbook by Hazan (2015). Likewise, we chose not venture into a much more general problem space of reinforcement learning, a subject of many graduate courses and textbooks such as Sutton and Barto (1998) and Szepesvári (2010).…”

Kernel-Based Aggregating Learning System for Online Portfolio Optimization

“…en, online convex optimization theory, in references [26][27][28], is applied to calculate coefficients of the proposed prediction model. Secondly, an enhanced learning system is built to optimize the portfolio by maximizing future wealth with a kernel-based increasing factor.…”

Efficient Online Laplacian Eigenmap Computation for Dimensionality Reduction in Molecular Phylogeny via Optimisation on the Sphere

“…One main drawback of such methods is that all pairwise distances between genomes are implicitely assumed available, which, due to the computational burden of estimating the alignments, is a very complicated issue that hinders the wider application of such refined methods. On the other hand, Laplacian eigenmap computation being as simple to perform as the PCA, online approaches [11] that only need a small proportion of the pairwise distances have a great potential for overcoming these computational issues. Such online algorithms progressively estimate the principal eigenvectors without having to wait for the full matrix to be known.…”