Horvitz-Thompson estimators for functional data: asymptotic confidence bands and optimal allocation for stratified sampling

Abstract

When dealing with very large datasets of functional data, survey sampling approaches are useful in order to obtain estimators of simple functional quantities, without being obliged to store all the data. We propose here a Horvitz-Thompson estimator of the mean trajectory. In the context of a superpopulation framework, we prove under mild regularity conditions that we obtain uniformly consistent estimators of the mean function and of its variance function. With additional assumptions on the sampling design we state a functional Central Limit Theorem and deduce asymptotic confidence bands. Stratified sampling is studied in detail, and we also obtain a functional version of the usual optimal allocation rule considering a mean variance criterion. These techniques are illustrated by means of a test population of N = 18902 electricity meters for which we have individual electricity consumption measures every 30 minutes over one week. We show that stratification can substantially improve both the accuracy of the estimators and reduce the width of the global confidence bands compared to simple random sampling without replacement.

Estimating with kernel smoothers the mean of functional data in a finite population setting. A note on variance estimation in presence of partially observed trajectories

“…For example in Cardot and Josserand (2011), the measurements are made every half an hour over a period of two weeks.…”

Using Complex Surveys to Estimate theL1‐Median of a Functional Variable: Application to Electricity Load Curves

“…Usually, one builds the stratification using a variable known on the whole population and strongly correlated with the variable of interest. In our case, we suggest two stratification variables computed using the first week: the first one is the linearized variable u k and the second International Statistical Review (2012), 80, 1, 40 one is the consumption Y k (Cardot & Josserand, 2011). The following two sample allocations are used:…”

“…• the u (1) -optimal allocation (u (1) -OPTIM) as suggested by Cardot & Josserand (2011) and computed here with respect to the variance S 2 u (1) (t),U h of the linearized variable computed during the first week and denoted by u…”

Unexpected properties of bandwidth choice when smoothing discrete data for constructing a functional data classifier

“…For example, in longitudinal data analysis the observation points are often widely spaced and irregularly placed, and substantial smoothing is commonly used to convert discrete data like these to functions. The impact of such smoothing has been addressed in the context of prediction or hypothesis testing for functional data; see, for example, Hall and Van Keilegom (2007), Panaretos, Kraus and Maddocks (2010), Wu and Müller (2011), Benhennia and Degras (2011), Cardot and Josserand (2011) and Cardot, Degras and Josserand (2013). The main conclusion of these papers has been that conventional rules for smoothing discrete data typically apply, and that smoothing parameters of standard size generally are appropriate.…”