Geometric Representation of High Dimension, Low Sample Size Data

Hall, Peter A.; Marron, J. S.; Neeman, Amnon

doi:10.1111/j.1467-9868.2005.00510.x

Cited by 427 publications

(411 citation statements)

References 21 publications

Supporting

Mentioning

392

Contrasting

Order By: Relevance

“…As Hall et al (2005) also show, Gaussian data behave in the same way and a demonstration of this is seen in Table 1. To provide an idea of consensus of these results, the 200,000-dimensional Gaussian was repeated and yielded on successive runs values of the ultrametricity measure of: 0.96, 0.98, 0.96.…”

Section: Quantifying Degree Of Ultrametricitysupporting

confidence: 60%

The Remarkable Simplicity of Very High Dimensional Data: Application of Model-Based Clustering

Murtagh

2009

J Classif

View full text Add to dashboard Cite

An ultrametric topology formalizes the notion of hierarchical structure. An ultrametric embedding, referred to here as ultrametricity, is implied by a hierarchical embedding. Such hierarchical structure can be global in the data set, or local. By quantifying extent or degree of ultrametricity in a data set, we show that ultrametricity becomes pervasive as dimensionality and/or spatial sparsity increases. This leads us to assert that very high dimensional data are of simple structure. We exemplify this finding through a range of simulated data cases. We discuss also application to very high frequency time series segmentation and modeling.

show abstract

Section: Quantifying Degree Of Ultrametricitysupporting

confidence: 60%

The Remarkable Simplicity of Very High Dimensional Data: Application of Model-Based Clustering

Murtagh

2009

J Classif

View full text Add to dashboard Cite

show abstract

“…. , x n ∈ S d with n d, which is frequently referred to as the high-dimension lowsample-size situation (Hall et al, 2005;Dryden, 2005;Ahn et al, 2007). In Euclidean space, the dimension of the data can be reduced to n without losing any information.…”

Section: Computational Algorithmmentioning

confidence: 99%

Analysis of principal nested spheres

Jung¹,

Dryden²,

Marron³

2012

Biometrika

Self Cite

130

135

View full text Add to dashboard Cite

SUMMARYA general framework for a novel non-geodesic decomposition of high-dimensional spheres or high-dimensional shape spaces for planar landmarks is discussed. The decomposition, principal nested spheres, leads to a sequence of submanifolds with decreasing intrinsic dimensions, which can be interpreted as an analogue of principal component analysis. In a number of real datasets, an apparent one-dimensional mode of variation curving through more than one geodesic component is captured in the one-dimensional component of principal nested spheres. While analysis of principal nested spheres provides an intuitive and flexible decomposition of the high-dimensional sphere, an interesting special case of the analysis results in finding principal geodesics, similar to those from previous approaches to manifold principal component analysis. An adaptation of our method to Kendall's shape space is discussed, and a computational algorithm for fitting principal nested spheres is proposed. The result provides a coordinate system to visualize the data structure and an intuitive summary of principal modes of variation, as exemplified by several datasets.

show abstract

“…We consider asymptotics of the method for d → ∞ with the sample size n fixed. Hall et al (2005) first demonstrated the insight available from such asymptotics. They showed that, under some conditions, each data point in a sample of size n tends to lie near a vertex of a regular n-simplex and all the randomness in the data appears in the form of a random rotation of this simplex.…”

Section: ·1 Four Clusters Casementioning

confidence: 99%

“…The regularity conditions for the geometric representation in Hall et al (2005) require that the entries of the data vector satisfy a ρ-mixing condition. Ahn et al (2007) gave a milder condition using asymptotic properties of the sample covariance.…”

Section: ·1 Four Clusters Casementioning

confidence: 99%

See 1 more Smart Citation

Bidirectional discrimination with application to data visualization

Huang¹,

Liu²,

Marron³

2012

Biometrika

Self Cite

View full text Add to dashboard Cite

SUMMARYLinear classifiers are very popular, but can have limitations when classes have distinct subpopulations. General nonlinear kernel classifiers are very flexible, but do not give clear interpretations and may not be efficient in high dimensions. We propose the bidirectional discrimination classification method, which generalizes linear classifiers to two or more hyperplanes. This new family of classification methods gives much of the flexibility of a general nonlinear classifier while maintaining the interpretability, and much of the parsimony, of linear classifiers. They provide a new visualization tool for high-dimensional, low-sample-size data. Although the idea is generally applicable, we focus on the generalization of the support vector machine and distance-weighted discrimination methods. The performance and usefulness of the proposed method are assessed using asymptotics and demonstrated through analysis of simulated and real data. Our method leads to better classification performance in high-dimensional situations where subclusters are present in the data.

show abstract

Geometric Representation of High Dimension, Low Sample Size Data

Cited by 427 publications

References 21 publications

The Remarkable Simplicity of Very High Dimensional Data: Application of Model-Based Clustering

The Remarkable Simplicity of Very High Dimensional Data: Application of Model-Based Clustering

Analysis of principal nested spheres

Bidirectional discrimination with application to data visualization

Contact Info

Product

Resources

About