Variable selection in classification for multivariate functional data

Abstract

When classification methods are applied to high-dimensional data, selecting a subset of the predictors may lead to an improvement in the predictive ability of the estimated model, in addition to reducing the model complexity. In Functional Data Analysis (FDA), i.e., when data are functions, selecting a subset of predictors corresponds to selecting a subset of individual time instants in the time interval in which the functional data are measured. In this paper, we address the problem of selecting the most informative time instants in multivariate functional data, a case much less studied than its single-variate counterpart. Our proposal allows one to use in a very simple way high-order information of the data, e.g. monotonicity or convexity by means of the functional data derivatives. The aforementioned problem is addressed with tools of Global Optimization in continuous variables: the time instants are selected to maximize the correlation between the class label and the Support Vector Machine score used for classification. The effectiveness of the proposal is shown in univariate and multivariate datasets.

Functional-bandwidth kernel for Support Vector Machine with Functional Data: An alternating optimization algorithm

“…Instead, we propose to maximize the Pearson correlation, R, between the class label Y i of the functional data X i and the score,Ŷ (X i , θ, α) in (1), where all the variables, including the time instants, are treated as continuous variables. Other references in the literature, such as [7,34], have previously used with excellent results the Pearson correlation coefficient. Despite the fact that, when using the Pearson correlation coefficient as a surrogate of accuracy, a linear relationship between the binary label, Y ∈ {−1, 1}, and the real-valued score,Ŷ ∈ R, is implicitly assumed, this coefficient is very fast to compute, and even more important, it also allows us to use gradient-based methodologies since its optimization amounts to solving a continuous optimization problem.…”

“…Support Vector Machine (SVM) [13,14,19,20,39,41,42,56,61] is one of the most used tools in multivariate classification, and it has also been widely applied for functional data. See [7,34,43,44,49,50] among others.…”

“…Consider the family of piecewise constant functions with 3 pieces, in(7). We have thatω (1) (t, θ (1) ) = ω 1 , with θ (1) = ω 1 , ω (2) (t, θ (2) ) = ω 1 I [0,τ1] + ω 2 I (τ1,T ] , with θ (2) = (ω 1 , ω 2 , τ 1 ), and finally ω (3) (t, θ (3) ) = ω 1 I [0,τ1] +ω 2 I (τ1,τ2] + ω 3 I (τ2,T ] , with θ (3) = (ω 1 , ω 2 , ω 3 , τ 1 , τ 2 ).…”

Selection of time instants and intervals with Support Vector Regression for multivariate functional data

“…In this article, we consider the problem of the optimal selection of time intervals (including time instants as a degenerate case) for functional Support Vector Regression. We extend the work done recently for selecting time instants, but not intervals, for the classification, instead of regression, problem (Blanquero et al, 2019b). The use of Support Vector Regression allows us to capture nonlinearities.…”

“…In particular, the feature selection problem in SVR with functional data has not yet been studied in the literature. For the classification problem, an SVM-based method to select time instants has been proposed in Blanquero et al (2019b).…”