Determining Intrinsic Dimension and Entropy of High-Dimensional Shape Spaces

Costa, Jose A.; Hero, Alfred O.

doi:10.1007/0-8176-4481-4_9

Cited by 59 publications

(106 citation statements)

References 13 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…Additional degrees of freedoms could be the length of the main line, the angle between the main line and the upper line and the length of the upper line. The intrinsic dimensions of digit 2 and 3 were estimated in (Costa & Hero, 2004) for a subsample of size 1000 as 13 and 12 respectively 12 and 11 depending on the way they build their neighborhood graph. We estimate an intrinsic dimension of 13 for digit 2 and 14 for digit 3.…”

Section: The Artificial 1-digit Datasetmentioning

confidence: 99%

Intrinsic dimensionality estimation of submanifolds in R^d

Hein

Audibert²

2005

Proceedings of the 22nd International Conference on Machine Learning - ICML '05

156

162

View full text Add to dashboard Cite

We present a new method to estimate the intrinsic dimensionality of a submanifold M in R d from random samples. The method is based on the convergence rates of a certain U -statistic on the manifold. We solve at least partially the question of the choice of the scale of the data. Moreover the proposed method is easy to implement, can handle large data sets and performs very well even for small sample sizes. We compare the proposed method to two standard estimators on several artificial as well as real data sets.

show abstract

Section: The Artificial 1-digit Datasetmentioning

confidence: 99%

Intrinsic dimensionality estimation of submanifolds in R^d

Hein

Audibert²

2005

Proceedings of the 22nd International Conference on Machine Learning - ICML '05

156

162

View full text Add to dashboard Cite

show abstract

“…Theorem 4 in [3] ensures that geodetic distances in the infinitesimal ball converge to Euclidean distances with probability 1; moreover, recalling the result reported in Theorem 1, it is possible to notice that, for x i =x, the quantities ρ(x i ) are samples drawn from the pdf reported in Equation (1), where the parameter k is known and the parameter d must be estimated. A simple approach for the estimation of d is the maximization of the log-likelihood function: …”

Section: Maximum Likelihood Approachesmentioning

confidence: 80%

“…This technique is based on the observation that the length function of such graphs, that is the sum of arc weights on the minimal graph that spans all the points in the dataset, is strongly dependent on d. The authors test their method by adopting either the geodesic minimal spanning tree (GMST [4]), where the arc weights are the geodetic distances computed through the ISOMAP [23] algorithm, or the kNN-graph (kNNG [5]), where the arc weights are based on the Euclidean distances, thus requiring a lower computational cost.…”

Section: Related Workmentioning

confidence: 99%

Minimum Neighbor Distance Estimators of Intrinsic Dimension

Lombardi

Rozza

Ceruti

et al. 2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Most of the machine learning techniques suffer the "curse of dimensionality" effect when applied to high dimensional data. To face this limitation, a common preprocessing step consists in employing a dimensionality reduction technique. In literature, a great deal of research work has been devoted to the development of algorithms performing this task. Often, these techniques require as parameter the number of dimensions to be retained; to this aim, they need to estimate the "intrinsic dimensionality" of the given dataset, which refers to the minimum number of degrees of freedom needed to capture all the information carried by the data. Although many estimation techniques have been proposed, most of them fail in case of noisy data or when the intrinsic dimensionality is too high. In this paper we present a family of estimators based on the probability density function of the normalized nearest neighbor distance. We evaluate the proposed techniques on both synthetic and real datasets comparing their performances with those obtained by state of the art algorithms; the achieved results prove that the proposed methods are promising.

show abstract

“…These data sets are often modeled as samples from a probability measure µ concentrated on or around a low-dimensional set embedded in high dimensional space (see for example [1,2,3,4,5,6,7]). While it is often assumed that such low-dimensional sets are in fact low-dimensional smooth manifolds, empirical evidence suggests that this is only a idealized situation: these sets may be not be smooth [8,4,9], they may have a non-differentiable metric tensor, self-intersections, and changes in dimensionality (see [10,11,12,13] and references therein).…”

Section: Introductionmentioning

confidence: 99%

Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature

Little

Maggioni

Rosasco

2017

Applied and Computational Harmonic Analysis

View full text Add to dashboard Cite

Large data sets are often modeled as being noisy samples from probability distributions µ in R D , with D large. It has been noticed that oftentimes the support M of these probability distributions seems to be well-approximated by low-dimensional sets, perhaps even by manifolds. We shall consider sets that are locally well approximated by k-dimensional planes, with k ≪ D, with k-dimensional manifolds isometrically embedded in R D being a special case. Samples from µ are furthermore corrupted by D-dimensional noise. Certain tools from multiscale geometric measure theory and harmonic analysis seem well-suited to be adapted to the study of samples from such probability distributions, in order to yield quantitative geometric information about them. In this paper we introduce and study multiscale covariance matrices, i.e. covariances corresponding to the distribution restricted to a ball of radius r, with a fixed center and varying r, and under rather general geometric assumptions we study how their empirical, noisy counterparts behave. We prove that in the range of scales where these covariance matrices are most informative, the empirical, noisy covariances are close to their expected, noiseless counterparts. In fact, this is true as soon as the number of samples in the balls where the covariance matrices are computed is linear in the intrinsic dimension of M. As an application, we present an algorithm for estimating the intrinsic dimension of M.

show abstract

Determining Intrinsic Dimension and Entropy of High-Dimensional Shape Spaces

Cited by 59 publications

References 13 publications

Intrinsic dimensionality estimation of submanifolds in R^d

Intrinsic dimensionality estimation of submanifolds in R^d

Minimum Neighbor Distance Estimators of Intrinsic Dimension

Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature

Contact Info

Product

Resources

About

Determining Intrinsic Dimension and Entropy of High-Dimensional Shape Spaces

Cited by 59 publications

References 13 publications

Intrinsic dimensionality estimation of submanifolds in Rd

Intrinsic dimensionality estimation of submanifolds in Rd

Minimum Neighbor Distance Estimators of Intrinsic Dimension

Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature

Contact Info

Product

Resources

About

Intrinsic dimensionality estimation of submanifolds in R^d

Intrinsic dimensionality estimation of submanifolds in R^d