On asymptotic properties of hyperparameter estimators for kernel-based regularization methods

Mu, Biqiang; Chen, Tianshi; Ljung, Lennart

doi:10.1016/j.automatica.2018.04.035

Cited by 67 publications

(21 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, when the wireless traffic dataset is large, i.e., N is a large number, the time consumption of each 4 However, for multiple linear kernels, such as the ones proposed in [28], [30], [31], P 0 becomes a difference-of-convex problem and efficient algorithms exist for solving the hyper-parameters.…”

Section: ) Learning Objectivesmentioning

confidence: 99%

Wireless Traffic Prediction With Scalable Gaussian Process: Framework, Algorithms, and Verification

Yin

et al. 2019

IEEE J. Select. Areas Commun.

163

View full text Add to dashboard Cite

The cloud radio access network (C-RAN) is a promising paradigm to meet the stringent requirements of the fifth generation (5G) wireless systems. Meanwhile, wireless traffic prediction is a key enabler for C-RANs to improve both the spectrum efficiency and energy efficiency through load-aware network managements. This paper proposes a scalable Gaussian process (GP) framework as a promising solution to achieve large-scale wireless traffic prediction in a cost-efficient manner. Our contribution is three-fold. First, to the best of our knowledge, this paper is the first to empower GP regression with the alternating direction method of multipliers (ADMM) for parallel hyper-parameter optimization in the training phase, where such a scalable training framework well balances the local estimation in baseband units (BBUs) and information consensus among BBUs in a principled way for large-scale executions.Second, in the prediction phase, we fuse local predictions obtained from the BBUs via a cross-validation based optimal strategy, which demonstrates itself to be reliable and robust for general regression tasks. Moreover, such a cross-validation based optimal fusion strategy is built upon a well acknowledged probabilistic model to retain the valuable closed-form GP inference properties. Third, we propose a C-RAN based scalable wireless prediction architecture, where the prediction accuracy and the time consumption can be balanced by tuning the number of the BBUs according to the real-time system demands. Experimental results show that our proposed scalable GP model can outperform the state-of-the-art approaches considerably, in terms of wireless traffic prediction performance. Index TermsC-RANs, Gaussian processes, parallel processing, ADMM, cross-validation, machine learning, wireless traffic solution to reach such ambitious goals is the adoption of cloud radio access networks (C-RANs) [2], which have attracted intense research interests from both academia and industry in recent years [3]. A C-RAN is composed of two parts: the distributed remote radio heads (RRHs) with basic radio functionalities to provide coverage over a large area, and the centralized baseband units (BBUs) pool with parallel BBUs to support joint processing and cooperative network management. The BBUs can perform dynamic resource allocation in accordance with realtime network demands based on the virtualized resources in cloud computing. One major feature for the C-RANs to enable high energy-efficient services is the fast adaptability to non-uniform traffic variations [1]-[4], e.g., the tidal effects. Consequently, wireless traffic prediction techniques stand out as the key enabler to realize such loadaware management and proactive control in C-RANs, e.g., the load-aware RRH on/off operation [4]. However, the adoption of wireless traffic prediction techniques in C-RANs must satisfy the requirements on prediction accuracy, cost-efficiency, implementability, and scalability for large-scale executions. A. Related WorksIn the literature, many statistical ti...

show abstract

Section: ) Learning Objectivesmentioning

confidence: 99%

Wireless Traffic Prediction With Scalable Gaussian Process: Framework, Algorithms, and Verification

Yin

et al. 2019

IEEE J. Select. Areas Commun.

163

View full text Add to dashboard Cite

show abstract

“…where det(·) is the determinant of a matrix. The EB (18) has the advantage that it is robust Pillonetto & Chiuso (2015), but not asymptotical optimal in the sense of MSE (Mu et al, 2017).…”

Section: Hyperparameter Estimationmentioning

confidence: 99%

“…First, the kernel, through which the regularization is defined, provides a carrier for prior knowledge on the dynamic system to be identified. Second, the model complexity is tuned in a continuous manner through the hyperparameter, which is the parameter vector used to parameterize the kernel, Mu et al (2017); Pillonetto & Chiuso (2015). Third, extensive simulation results show that KRM can have better average accuracy and robustness than ML/PEM for the data that is short and/or has low signal-to-noise ratio, Chen et al (2012); Pillonetto et al (2014), and as a result, algorithms of KRM have been added to the System Identification Toolbox of MATLAB (Ljung et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

“…In Zorzi & Chiuso (2017), a harmonic analysis is first provided for existing kernels including the amplitude modulated locally stationary (AMLS) kernel introduced in and then is shown to be a useful tool to design more general kernels. In contrast, there are few results reported for the issue of hyperparameter estimation Mu et al (2017); Pillonetto & Chiuso (2015). In particular, it was shown in Mu et al (2017) that the Stein's unbiased risk estimator (SURE) is asymptotically optimal but the widely used empirical Bayes estimator is not.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, there are few results reported for the issue of hyperparameter estimation Mu et al (2017); Pillonetto & Chiuso (2015). In particular, it was shown in Mu et al (2017) that the Stein's unbiased risk estimator (SURE) is asymptotically optimal but the widely used empirical Bayes estimator is not.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On input design for regularized LTI system identification: Power-constrained input

Chen

2018

Automatica

Self Cite

View full text Add to dashboard Cite

Input design is an important issue for classical system identification methods but has not been investigated for the kernel-based regularization method (KRM) until very recently. In this paper, we consider in the time domain the input design problem of KRMs for LTI system identification. Different from the recent result, we adopt a Bayesian perspective and in particular make use of scalar measures (e.g., the A-optimality, D-optimality, and E-optimality) of the Bayesian mean square error matrix as the design criteria subject to power-constraint on the input. Instead to solve the optimization problem directly, we propose a two-step procedure. In the first step, by making suitable assumptions on the unknown input, we construct a quadratic map (transformation) of the input such that the transformed input design problems are convex, the number of optimization variables is independent of the number of input data, and their global minima can be found efficiently by applying well-developed convex optimization software packages. In the second step, we derive the expression of the optimal input based on the global minima found in the first step by solving the inverse image of the quadratic map. In addition, we derive analytic results for some special types of fixed kernels, which provide insights on the input design and also its dependence on the kernel structure.

show abstract

Regularization of Linear Regression Models

Pillonetto

Chen

Chiuso

et al. 2022

Regularized System Identification

View full text Add to dashboard Cite

Linear regression models are widely used in statistics, machine learning and system identification. They allow to face many important problems, are easy to fit and enjoy simple analytical properties. The simplest method to fit linear regression models is least squares whose systematic treatment is available in many textbooks, e.g., [35, Chap. 4], [12]. Linear regression models can be fitted also in different way and a class of methods that we will consider in this chapter is the so-called regularized least squares. It is an extension of least squares which minimizes the sum of the square loss function and a regularization term. This latter can take various forms, leading to several variants which have been applied extensively in theory as well as in practical applications. In this chapter, we will focus on these methods and introduce their fundamentals. In the first part of the appendix to this chapter, we also report some basic results of linear algebra useful for the reading.

show abstract

On asymptotic properties of hyperparameter estimators for kernel-based regularization methods

Cited by 67 publications

References 23 publications

Wireless Traffic Prediction With Scalable Gaussian Process: Framework, Algorithms, and Verification

Wireless Traffic Prediction With Scalable Gaussian Process: Framework, Algorithms, and Verification

On input design for regularized LTI system identification: Power-constrained input

Regularization of Linear Regression Models

Contact Info

Product

Resources

About