Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond

Yuan, Jianjun; Lamperski, Andrew

doi:10.1609/aaai.v34i04.6149

Cited by 8 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Online control: Online regret analysis is an extensively studied topic in online learning problems (Hazan, 2019;Shalev-Shwartz et al, 2011;Yuan and Lamperski, 2020). Recently, many papers studied the online regret performance in control problems with general time-varying costs, disturbances and known system model (Abbasi-Yadkori et al, 2014;Cohen et al, 2018;Agarwal et al, 2019a,b;Goel and Wierman, 2019).…”

Section: Related Workmentioning

confidence: 99%

Online Learning for Predictive Control with Provable Regret Guarantees

Muthirayan¹,

Yuan²,

Kalathil³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We address the problem of learning to control an unknown linear dynamical system with time varying cost functions through the framework of online Receding Horizon Control (RHC). We consider the setting where the control algorithm does not know the true system model and has only access to a fixedlength (that does not grow with the control horizon) preview of the future cost functions. We characterize the performance of an algorithm using the metric of dynamic regret, which is defined as the difference between the cumulative cost incurred by the algorithm and that of the best sequence of actions in hindsight. We propose two different online RHC algorithms to address this problem, namely Certainty Equivalence RHC (CE-RHC) algorithm and Optimistic RHC (O-RHC) algorithm. We show that under the standard stability assumption for the model estimate, the CE-RHC algorithm achieves O(T 2/3 ) dynamic regret. We then extend this result to the setting where the stability assumption hold only for the true system model by proposing the O-RHC algorithm. We show that O-RHC algorithm achieves O(T 2/3 ) dynamic regret but with some additional computation.

show abstract

Section: Related Workmentioning

confidence: 99%

Online Learning for Predictive Control with Provable Regret Guarantees

Muthirayan¹,

Yuan²,

Kalathil³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…By hedging over a collection of OGD algorithms defined by exponential grid of step sizes, (Zhang et al, 2018a) proposes an algorithm that achieves a faster rate of O( n(1 + P n )) which is shown to be minimax optimal when the loss functions are convex. (Yuan and Lamperski, 2019) proposes strategies that can attain regret rates of…”

Section: Related Workmentioning

confidence: 99%

“…In the OCO setting, when the environment is benign, (Zhao et al, 2020) replaces the √ n dependence in the regret of O( n(1 + P n )) attained by (Zhang et al, 2018b) with problem dependent quantities that could be much smaller than √ n. Although the linear smoother lower bound in Proposition 2 of (Baby and Wang, 2019) would imply that an OEGD (Online Extra Gradient Descent) (Zhao et al, 2020) expert with any learning rate sequences require Ω( √ nP n ) dynamic regret for the 1D-TV-denoising problem. Interestingly in (Yuan and Lamperski, 2019) the authors mention that even in the one-dimensional setting, a lower bound on dynamic regret for strongly convex / exp-concave losses that holds uniformly for the entire range 0 ≤ P n ≤ n is unknown. However, we find that one can combine the existing lower bounds on univariate TV-denoising in a stochastic setting (Donoho and Johnstone, 1998) with the lower bound construction of (Vovk, 2001) (or see Theorem 11.9 in (Cesa-Bianchi and Lugosi, 2006)) for online learning with squared error losses to obtain an Ω(log n ∨ n 1/3 C 2/3 n ) in one dimensions (see Appendix C for details).…”

Section: A More On Related Workmentioning

confidence: 99%

“…The works of (Zhang et al, 2018a) and (Yuan and Lamperski, 2019) attains a dynamic regret of O * ( n(1 + C n )) and O * ( √ nC n ∨ log n) respectively, where O * (•) hides dependence on the dimension and (a ∨ b) = max{a, b}. However, we show a lower bound of Ω * (n 1/3 C 2/3 n ∨ log n) in Proposition 10 applicable to the case when losses are strongly convex / exp-concave.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Dynamic Regret in Exp-Concave Online Learning

Baby,

Wang

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of the Zinkevich (2003)-style dynamic regret minimization in online learning with exp-concave losses. We show that whenever improper learning is allowed, a Strongly Adaptive online learner achieves the dynamic regret of Õ(dwhere Cn is the total variation (a.k.a. path length) of the an arbitrary sequence of comparators that may not be known to the learner ahead of time. Achieving this rate was highly nontrivial even for squared losses in 1D where the best known upper bound was O( √ nCn ∨ log n) (Yuan and Lamperski, 2019). Our new proof techniques make elegant use of the intricate structures of the primal and dual variables imposed by the KKT conditions and could be of independent interest. Finally, we apply our results to the classical statistical problem of locally adaptive non-parametric regression (Mammen, 1991;Donoho and Johnstone, 1998) and obtain a stronger and more flexible algorithm that do not require any statistical assumptions or any hyperparameter tuning.

show abstract

“…which draws considerable attention recently (Besbes et al, 2015;Jadbabaie et al, 2015;Mokhtari et al, 2016;Yang et al, 2016;Wei et al, 2016;Zhang et al, 2017Zhang et al, , 2018aAuer et al, 2019;Baby and Wang, 2019;Yuan and Lamperski, 2020;Zhao et al, 2020a;Zhang et al, 2020a,b;Zhao et al, 2021a;Zhao and Zhang, 2021). The measure is also called the universal dynamic regret (or general dynamic regret), in the sense that it gives a universal guarantee that holds against any comparator sequence.…”

Section: Introductionmentioning

confidence: 99%

Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization

Zhao¹,

Zhang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We investigate online convex optimization in non-stationary environments and choose the dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence. Let T be the time horizon and P T be the path-length that essentially reflects the non-stationarity of environments, the state-of-theart dynamic regret is O( T (1 + P T )). Although this bound is proved to be minimax optimal for convex functions, in this paper, we demonstrate that it is possible to further enhance the guarantee for some easy problem instances, particularly when online functions are smooth. Specifically, we propose novel online algorithms that can leverage smoothness and replace the dependence on T in the dynamic regret by problem-dependent quantities: the variation in gradients of loss functions, the cumulative loss of the comparator sequence, and the minimum of the previous two terms. These quantities are at most O(T ) while could be much smaller in benign environments. Therefore, our results are adaptive to the intrinsic difficulty of the problem, since the bounds are tighter than existing results for easy problems and meanwhile guarantee the same rate in the worst case. Notably, our algorithm requires only one gradient per iteration, which shares the same gradient query complexity with the methods developed for optimizing the static regret. As a further application, we extend the results from the full-information setting to bandit convex optimization with two-point feedback and thereby attain the first problem-dependent dynamic regret for such bandit tasks.

show abstract

Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond

Cited by 8 publications

References 12 publications

Online Learning for Predictive Control with Provable Regret Guarantees

Online Learning for Predictive Control with Provable Regret Guarantees

Optimal Dynamic Regret in Exp-Concave Online Learning

Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization

Contact Info

Product

Resources

About