Online Optimization in Dynamic Environments: A Regret Analysis for Sparse Problems

Abstract

Time-varying systems are a challenge in many scientific and engineering areas. Usually, estimation of time-varying parameters or signals must be performed online, which calls for the development of responsive online algorithms. In this paper, we consider this problem in the context of the sparse optimization; specifically, we consider the Elastic-net model, which promotes parsimonious solutions. Following the rationale in [23], we propose an online algorithm and we theoretically prove that it is successful in terms of dynamic regret. We then show an application to the problem of recursive identification of time-varying autoregressive models, in the case when the number of parameters to be estimated is unknown. Numerical results show the practical efficiency of the proposed method.

Time-Varying Convex Optimization: Time-Structured Algorithms and Applications

“…In [35] and [77], two algorithms are presented, one unstructured using homotopy and one structured building a model based on methods akin to Kalman filters. In [78], unstructured methods for the elastic net are discussed.…”

A linear programming approach to sparse linear regression with quantized data

“…In machine learning and system identification, sparsity is desirable to reduce as much as possible the complexity of the estimated models. In the literature, sparsity is exploited for the identification of linear systems, see, e.g., [3], [4], [5]; non-linear functions in [6]; polynomial models in [7]; time-varying systems in [8], [9].…”

Frequency Hopping Signals Tracking and Sorting Based on Dynamic Programming Modulated Wideband Converters

“…In [26,27], tracking and dynamic filtering of time-varying sparse signals are analyzed under certain assumptions. In [28,29], a series of universal approaches were proposed for online optimization in dynamic environments. These methods transform the CS of time-varying sparse signals to online convex optimization, which proceed from the perspective of regret analysis.…”