Non-asymptotic adaptive prediction in functional linear models

Abstract

Functional linear regression has recently attracted considerable interest. Many works focus on asymptotic inference. In this paper we consider in a non asymptotic framework a simple estimation procedure based on functional Principal Regression. It revolves in the minimization of a least square contrast coupled with a classical projection on the space spanned by the m first empirical eigenvectors of the covariance operator of the functional sample. The novelty of our approach is to select automatically the crucial dimension m by minimization of a penalized least square contrast. Our method is based on model selection tools. Yet, since this kind of methods consists usually in projecting onto known non-random spaces, we need to adapt it to empirical eigenbasis made of data-dependent -hence random -vectors. The resulting estimator is fully adaptive and is shown to verify an oracle inequality for the risk associated to the prediction error and to attain optimal minimax rates of convergence over a certain class of ellipsoids. Our strategy of model selection is finally compared numerically with cross-validation.

Lasso in infinite dimension: application to variable selection in functional multivariate linear regression

“…Since the elements of the PCA basis are data-dependent, but depend only on the X i 's, the results of Section 3 hold but not the results of Section 4. Similar results for the PCA basis could be derived from the theory developed in Mas and Ruymgaart (2015); Brunel et al (2016) at the price of further theoretical considerations which are out of the scope of the paper. Depending on the nature of the data, non-random basis, such as Fourier, splines or wavelet basis, could also be considered.…”

“…Hence, the Lasso estimator recovers the true support but gives biased estimators of the coefficients β j , j ∈ J * . For the projected estimator β λ, m , as recommended by Brunel et al (2016), we set the value of the constant κ of criterion (8) to κ = 2. The selected dimensions are plotted in Figure 3.…”

“…where λ = (λ 1 , ..., λ p ) are positive parameters and (H (m) ) m≥1 is a sequence of nested finite-dimensional subspaces of H, to be specified later. A data-driven dimension selection criterion is proposed to select the dimension m, inspired by the works of Barron et al (1999) and their adaptation to the functional linear model by Johannes (2010, 2012); Brunel and Roche (2015); Brunel et al (2016).…”

A one-sample test for normality with kernel methods

“…Finally, we illustrate our test and present a sequential procedure that assesses the rank of a covariance operator. The problem of covariance rank estimation is adressed in several domains: functional regression [9,7], classification [41] and dimension reduction methods such as PCA, Kernel PCA and Non-Gaussian Component Analysis [3,12,13] where the dimension of the kept subspace is a crucial problem. 3 Here is the outline of the paper.…”

Local optimization of black-box functions with high or infinite-dimensional inputs: application to nuclear safety

“…with α ∈ R, β ∈ H and ε ∼ N (0, σ 2 ). If H is a function space, this model is known as functional linear model (Ramsay and Dalzell, 1991;Cardot et al, 1999) and has been widely studied (see Cardot and Sarda 2011 for a recent overview or Brunel et al 2016 for a recent work on this subject). Moreover, it is known that, if H is of high or infinite dimension, least-squares estimators of the slope parameter β are, in general, unstable.…”