Stringing High-Dimensional Data for Functional Analysis

Chen, Kun; Chen, Kehui; Müller, Hans‐Georg; Wang, Jane-Ling

doi:10.1198/jasa.2011.tm10314

Cited by 30 publications

(27 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also of special interest are recent developments in the interface of high-dimensional and functional data. These include the following: combining functional elements with high-dimensional covariates, such as modeling predictor times that exercise an individual predictor effect on an outcome that goes beyond the functional linear model (Kneip & Sarda 2011); predictor selection among high-dimensional FPC scores and baseline covariates in functional regression models (Kong et al 2015); or converting high-dimensional data outright into functional data, where the latter has been referred to as stringing (Wu & Müller 2010, K. Chen et al 2011 and is based on a uni-or multidimensional scaling step to order predictors along locations on an interval or low-dimensional domain. The stringing method then assigns the value of the respective predictor to the location of the predictor on the interval, for all predictors.…”

Section: Outlook and Future Perspectivesmentioning

confidence: 99%

Functional Data Analysis

Wang

Chiou

Müller

2016

Annu. Rev. Stat. Appl.

Self Cite

753

450

View full text Add to dashboard Cite

With the advance of modern technology, more and more data are being recorded continuously during a time interval or intermittently at several discrete time points. These are both examples of functional data, which has become a commonly encountered type of data. Functional data analysis (FDA) encompasses the statistical methodology for such data. Broadly interpreted, FDA deals with the analysis and theory of data that are in the form of functions. This paper provides an overview of FDA, starting with simple statistical notions such as mean and covariance functions, then covering some core techniques, the most popular of which is functional principal component analysis (FPCA). FPCA is an important dimension reduction tool, and in sparse data situations it can be used to impute functional data that are sparsely observed. Other dimension reduction approaches are also discussed. In addition, we review another core technique, functional linear regression, as well as clustering and classification of functional data. Beyond linear and single-or multiple-index methods, we touch upon a few nonlinear approaches that are promising for certain applications. They include additive and other nonlinear functional regression models and models that feature time warping, manifold learning, and empirical differential equations. The paper concludes with a brief discussion of future directions.Annu. Rev. Stat. Appl. 2016.3:257-295. Downloaded from www.annualreviews.org Access provided by Academia Sinica -Life Science Library on 06/04/16. For personal use only. Section 2, and several approaches for dimension reduction in functional regression are discussed in Section 3. Clustering and classification of functional data are useful and important tools with wide-ranging applications in FDA. Methods include extensions of classical k-means and hierarchical clustering, Bayesian and model approaches to clustering, and classification via functional regression and functional discriminant analysis. These topics are explored in Section 4. The classical methods for FDA have been predominantly linear, such as functional principal components (FPCs) or the FLM. As more and more functional data are generated, it has emerged that many such data have inherent nonlinear features that make linear methods less effective. Sections 5 reviews some nonlinear approaches to FDA, including time warping, nonlinear manifold modeling, and nonlinear differential equations to model the underlying empirical dynamics.A well-known and well-studied nonlinear effect is time warping, where in addition to the common amplitude variation, one also considers time variation. This creates a basic nonidentifiability problem, and Section 5.1 provides a discussion of these foundational issues. A more general approach to modeling nonlinearity in functional data is to assume that the functional data lie on www.annualreviews.org • Review of Functional Data Analysis 259

show abstract

Section: Outlook and Future Perspectivesmentioning

confidence: 99%

Functional Data Analysis

Wang

Chiou

Müller

2016

Annu. Rev. Stat. Appl.

Self Cite

753

450

View full text Add to dashboard Cite

show abstract

“…The problem of recovering a hidden Hamiltonian cycle (path) in a weighted complete graph falls into a general problem known as data seriation [Ken71] or data stringing [CCMW11]. In particular, we are given a similarity matrix Y for n objects, and are interested in seriating or stringing the data, by ordering the n objects so that similar objects i and j are near each other.…”

Section: Data Seriationmentioning

confidence: 99%

“…In particular, we are given a similarity matrix Y for n objects, and are interested in seriating or stringing the data, by ordering the n objects so that similar objects i and j are near each other. Data seriation has diverse applications ranging from data visualization, DNA sequencing to functional data analysis [CCMW11] and archaeological dating [Rob51]. Most previous work on data seriation focuses on the noiseless case [Rob51,Ken71], where there is an unknown ordering of n objects so that if object j is closer than object k to object i in the ordering, then Y ij ≥ Y ik , i.e., the similarity between i and j is always no less than the similarity between i and k. Such a matrix Y is called Robinson matrix.…”

Section: Data Seriationmentioning

confidence: 99%

Hidden Hamiltonian Cycle Recovery via Linear Programming

Bagaria

Ding

Tse

et al. 2020

Operations Research

121

View full text Add to dashboard Cite

We introduce the problem of hidden Hamiltonian cycle recovery, where there is an unknown Hamiltonian cycle in an n-vertex complete graph that needs to be inferred from noisy edge measurements. The measurements are independent and distributed according to P n for edges in the cycle and Q n otherwise. This formulation is motivated by a problem in genome assembly, where the goal is to order a set of contigs (genome subsequences) according to their positions on the genome using long-range linking measurements between the contigs. Computing the maximum likelihood estimate in this model reduces to a Traveling Salesman Problem (TSP). Despite the NP-hardness of TSP, we show that a simple linear programming (LP) relaxation, namely the fractional 2-factor (F2F) LP, recovers the hidden Hamiltonian cycle with high probability as n → ∞ provided that α n − log n → ∞, where α n −2 log √ dP n dQ n is the Rényi divergence of order 1 2 . This condition is information-theoretically optimal in the sense that, under mild distributional assumptions, α n ≥ (1 + o(1)) log n is necessary for any algorithm to succeed regardless of the computational cost.Departing from the usual proof techniques based on dual witness construction, the analysis relies on the combinatorial characterization (in particular, the half-integrality) of the extreme points of the F2F polytope. Represented as bicolored multi-graphs, these extreme points are further decomposed into simpler "blossom-type" structures for the large deviation analysis and counting arguments. Evaluation of the algorithm on real data shows improvements over existing approaches.

show abstract

“…Indeed, providing novel and general frameworks for the statistical analysis of infinite-dimensional data has been of interest to address a number of issues in the so called "large p small n" problems, related to the curse of dimensionality. For instance, Chen et al (2011) propose to adopt a FDA viewpoint to perform regression in the presence of high-dimensional predictors, and show that their approach offers a sensible alternative to classical regression models, that often fail in this setting. Viceversa, well-known methods that are commonly employed in high-dimensional statistics can be inspiring to address similar problems arising in the FDA setting.…”

Section: Discussion: the Application Viewpointmentioning

confidence: 99%

Kriging for Hilbert-space valued random fields: The operatorial point of view

Menafoglio

Petris

2016

Journal of Multivariate Analysis

View full text Add to dashboard Cite

Please cite this article as: A. Menafoglio, G. Petris, Kriging for Hilbert-space valued random fields: The operatorial point of view, Journal of Multivariate Analysis (2015), http://dx. AbstractWe develop a comprehensive framework for linear spatial prediction in Hilbert spaces. We explore the problem of Best Linear Unbiased (BLU) prediction in Hilbert spaces through an original point of view, based on a new Operatorial definition of Kriging. We ground our developments on the theory of Gaussian processes in function spaces and on the associated notion of measurable linear transformation. We prove that our new setting allows (a) to derive an explicit solution to the problem of Operatorial Ordinary Kriging, and (b) to establish the relation of our novel predictor with the key concept of conditional expectation of a Gaussian measure. Our new theory is posed as a unifying theory for Kriging, which is shown to include the Kriging predictors proposed in the literature on Functional Data through the notion of finite-dimensional approximations. Our original viewpoint to Kriging offers new relevant insights for the geostatistical analysis of either finite-or infinite-dimensional georeferenced dataset.

show abstract

Stringing High-Dimensional Data for Functional Analysis

Cited by 30 publications

References 26 publications

Functional Data Analysis

Functional Data Analysis

Hidden Hamiltonian Cycle Recovery via Linear Programming

Kriging for Hilbert-space valued random fields: The operatorial point of view

Contact Info

Product

Resources

About