Scaffoldings and Spines: Organizing High-Dimensional Data Using Cover Trees, Local Principal Component Analysis, and Persistent Homology

Abstract

We propose a flexible and multi-scale method for organizing, visualizing, and understanding datasets sampled from or near stratified spaces. The first part of the algorithm produces a cover tree using adaptive thresholds based on a combination of multiscale local principal component analysis and topological data analysis. The resulting cover tree nodes consist of points within or near the same stratum of the stratified space. They are then connected to form a scaffolding graph, which is then simplified and collapsed down into a spine graph. From this latter graph the stratified structure becomes apparent. We demonstrate our technique on several synthetic point cloud examples and we use it to understand song structure in musical audio data. IntroductionWe consider point cloud data, modeled as a set of points X = {x 1 , . . . , x n } in a Euclidean space R D . Such clouds are hard to analyze directly when n and/or D is large. Subsampling techniques are often used in the former situation, and dimension reduction in the latter. In both cases, much care has to be taken to ensure that the reduction in the number of points and number of dimensions does not destroy essential features of the data. While theorems exist in both contexts, they tend to make assumptions about parameters (like intrinsic dimension) that may be unknown or that may vary widely across X. The latter problem often occurs when X is not sampled from a manifold, but rather from a stratified space.A stratified space is a topological space that can be decomposed into manifold pieces (called strata), of possibly different dimension, all of which fit together in some uniform fashion. A key distinction is between maximal and non-maximal (also called singular) strata: briefly, a non-maximal stratum occurs where two or more maximal strata meet. See Figure 2 for an example. This paper proposes a novel, fast, and flexible technique for organizing, visualizing, analyzing, and understanding a point cloud that has been sampled from or near a stratified space. The technique uses a data structure called the cover tree [6] and employs techniques derived from multi-scale local principal component analysis (MLPCA, [18]) and topological data analysis (TDA,[12]). Our method summarizes the strata that make up the underlying stratified space, exhibits how the different pieces fit together, and reflects the local geometric and topological properties of the space.Instead of subsampling or reducing dimensions, we derive from X a multi-scale set of graphs, called the scaffoldings and the spine of X. At each fixed scale, a point in X belongs to a unique node in these graphs. A subset of points belongs to a common node only if the local geometry at each of those points is similar enough; that is, only if they belong to a common stratum. We also determine whether a node corresponds to a maximal or non-maximal stratum. In the former case, this suggests that a single dimension reduction technique might be employed on the points in that node. The edges of these graphs give informatio...

Persistent homology of complex networks for dynamic state detection

“…One particularly useful tool for this analysis is 1-dimensional persistent homology [32,33], which encodes how circular structures persist over the course of a filtration in a topological signature called a persistence diagram. This and its variants have been quite successful in applications, particularly for the analysis of periodicity [34][35][36][37][38][39][40][41], including for parameter selection [42,43], data clustering [44], machining dynamics [45][46][47][48][49], gene regulatory systems [50,51], financial data [52][53][54], wheeze detection [55], sonar classification [56], video analysis [57][58][59], and annotation of song structure [60,61].…”

Topological methods for genomics: Present and future directions

“…Zigzag [43,44] and multidimensional [45] persistence are, for instance, promising methods for the analysis of temporal genomic data. Recent advances in dimensional reduction leveraging the modularity of topologically stratified spaces [46] will probably result in valuable tools for the analysis of genomic data. In summary, a rich interplay between formal developments and new applications is expected in upcoming years, which may place TDA in the standard toolbox of computational biology.…”

Heuristic Framework for Multiscale Testing of the Multi-Manifold Hypothesis

“…While the two notions are closely related, the emphasis of stratified spaces is topological. Bendich, Gasparovic, Tralie and Harer [9] used stratified spaces to develop a heuristic approach for partitioning the space. Their approach is both similar and complementary to the partitioning approach used in our methodology.…”