Functional Linear Regression with Mixed Predictors

Wang, Daren; Zhao, Zifeng; Yu, Yijun; Willett, Rebecca

doi:10.48550/arxiv.2012.00460

Cited by 5 publications

(5 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results achieve the same problem-dependent scaling as in canonical finite dimensional linear regression [Abbasi-Yadkori et al, 2011b,a, Peña et al, 2008, Hsu et al, 2012b. On the other hand, our results specialize the functional regression setting of [Benatia et al, 2017, Wang et al, 2020 to CDF estimation, where minimal assumptions are made on the data generation process. Moreover, we also derive Ωp a d{nq information theoretic lower bounds for functional linear regression of CDFs.…”

Section: Introductionsupporting

confidence: 72%

Functional Linear Regression of Cumulative Distribution Functions

Zhang¹,

Makur²,

Azizzadenesheli³

2022

Preprint

View full text Add to dashboard Cite

The estimation of cumulative distribution functions (CDF) is an important learning task with a great variety of downstream applications, e.g., risk assessments in predictions and decision making. We study functional regression of contextual CDFs where each data point is sampled from a linear combination of context dependent CDF bases. We propose estimation methods that estimate CDFs accurately everywhere. In particular, given n samples with d bases, we show estimation error upper bounds of r Op a d{nq for fixed design, random design, and adversarial context cases. We also derive matching information theoretic lower bounds, establishing minimax optimality for CDF functional regression. To complete our study, we consider agnostic settings where there is a mismatch in the data generation process. We characterize the error of the proposed estimator in terms of the mismatched error, and show that the estimator is well-behaved under model mismatch.

show abstract

Section: Introductionsupporting

confidence: 72%

Functional Linear Regression of Cumulative Distribution Functions

Zhang¹,

Makur²,

Azizzadenesheli³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Here, λ min /σ √ p 1 ∨ p 2 is required for consistent estimation of the singular vectors a l , b l (also see Cai et al (2019)); while λ min /τ n −α/(2α+1) ∨ (p 1 ∨ p 2 )/n is required for consistent estimation/approximation of the singular functions ξ l and it yields an interesting phase transition: when p 1 ∨ p 2 n 1/(2α+1) , the bound is dominated by the non-parametric rate n −α/(2α+1) ; when p 1 ∨ p 2 n 1/(2α+1) , the bound is dominated by the parametric rate (p 1 ∨ p 2 )/n. We note that the phase transition between parametric and non-parametric rates commonly appear in the study of high-dimensional functional regression with sparsity, e.g., Wang et al (2020b). To the best of our knowledge, we are the first to establish such the phenomenon in the low-rank-based functional data analyses.…”

Section: Local Convergencementioning

confidence: 77%

“…For each l ∈ [r], we sample a l , b l uniformly from the unit spheres S p 1 −1 , S p 2 −1 and generate ξ l from orthonormal basis functions {u i (s)} 10 i=1 ⊂ L 2 ([0, 1]). Following Yuan and Cai (2010); Wang et al (2020b), we set u 1 (s) = 1 and u i (s) = √ 2 cos ((i − 1)πs) for i = 2, . .…”

Section: Simulation Studiesmentioning

confidence: 99%

Guaranteed Functional Tensor Singular Value Decomposition

Han¹,

Shi²,

Zhang³

2021

Preprint

View full text Add to dashboard Cite

This paper introduces the functional tensor singular value decomposition (FTSVD), a novel dimension reduction framework for tensors with one functional mode and several tabular modes. The problem is motivated by high-order longitudinal data analysis. Our model assumes the observed data to be a random realization of an approximate CP low-rank functional tensor measured on a discrete time grid. Incorporating tensor algebra and the theory of Reproducing Kernel Hilbert Space (RKHS), we propose a novel RKHS-based constrained power iteration with spectral initialization. Our method can successfully estimate both singular vectors and functions of the low-rank structure in the observed data. With mild assumptions, we establish the non-asymptotic contractive error bounds for the proposed algorithm. The superiority of the proposed framework is demonstrated via extensive experiments on both simulated and real data.

show abstract

“…Zou et al, 2006), and derived an upper bound in the form of the sum of that on lasso and ridge estimators. In the high-dimensional functional data analysis literature, with potentially many different functional covariates, Wang et al (2020b) showed that the prediction upper bound is the sum of an upper bound related with the smoothness penalty and an upper bound related with the high-dimensionality.…”

Section: A Penalised Estimatormentioning

confidence: 99%

Denoising and change point localisation in piecewise-constant high-dimensional regression coefficients

Wang¹,

Padilla²,

Yu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We study the theoretical properties of the fused lasso procedure originally proposed by Tibshirani et al. (2005) in the context of a linear regression model in which the regression coefficient are totally ordered and assumed to be sparse and piecewise constant. Despite its popularity, to the best of our knowledge, estimation error bounds in high-dimensional settings have only been obtained for the simple case in which the design matrix is the identity matrix. We formulate a novel restricted isometry condition on the design matrix that is tailored to the fused lasso estimator and derive estimation bounds for both the constrained version of the fused lasso assuming dense coefficients and for its penalised version. We observe that the estimation error can be dominated by either the lasso or the fused lasso rate, depending on whether the number of non-zero coefficient is larger than the number of piece-wise constant segments. Finally, we devise a post-processing procedure to recover the piecewise-constant pattern of the coefficients. Extensive numerical experiments support our theoretical findings.

show abstract

Functional Linear Regression with Mixed Predictors

Cited by 5 publications

References 57 publications

Functional Linear Regression of Cumulative Distribution Functions

Functional Linear Regression of Cumulative Distribution Functions

Guaranteed Functional Tensor Singular Value Decomposition

Denoising and change point localisation in piecewise-constant high-dimensional regression coefficients

Contact Info

Product

Resources

About