Local-global merge tree computation with local exchanges

Nigmetov, Arnur; Morozov, Dmitriy

doi:10.1145/3295500.3356188

Cited by 10 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It hosts 4,200 KNL compute nodes with the a single-socket 68-core Intel Xeon Phi 7250 processor (i.e., same as Cori's KNL nodes), and 1,736 SKX compute nodes with 48 Intel Xeon Platinum 8160 cores on two sockets. Due to intermittent machine availability, we have performed the comparison between Reeber [1] (a library implementing the local-global representation [16,18]) and the merge forest (Fig. 13) on Stampede 2 and all other experiments on Cori.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, in some cases, an a priori superlevel set is known and data can be thresholded [11]. Then region growing can be resolved until correct [18]. Unfortunately, if the exact threshold is unknown, a conservative lower bound must be used, leading to less scalable execution due to more work at the global reduction phase of the algorithms.…”

Section: Related Workmentioning

confidence: 99%

“…Unfortunately, neither of the PMT nor the Reeber libraries provide a distributed query implementation to compare against (with the exception of halo component extraction using the components algorithm [18] in Reeber). Hence, in order to compare the performance of queries using the different data structures, we implemented Maxima and ComponentMax queries in the publicly available local-global implementation (Reeber).…”

Section: Topological Queries At Scale: Density Clusters In Cosmologymentioning

confidence: 99%

“…The volume of superlevel-set components is used to compute a percolation threshold, which is useful in studying a flow's turbulence and validating normalization schemes [7,10]. Although there exist several approaches to computing merge trees in a distributed setting [11,16,18], they are mainly focused on building a global data structure that is later used to answer topological queries. The construction of a global tree is often a very expensive task and requires a series of collective operations that dramatically affect performance at scale.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Distributed merge forest

Huang

Klacansky

Petruzza

et al. 2021

Proceedings of the ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Topological analysis is used in several domains to identify and characterize important features in scientific data, and is now one of the established classes of techniques of proven practical use in scientific computing. The growth in parallelism and problem size tackled by modern simulations poses a particular challenge for these approaches. Fundamentally, the global encoding of topological features necessitates interprocess communication that limits their scaling. In this paper, we extend a new topological paradigm to the case of distributed computing, where the construction of a global merge tree is replaced by a distributed data structure, the merge forest, trading slower individual queries on the structure for faster end-to-end performance and scaling. Empirically, the queries that are most negatively affected also tend to have limited practical use. Our experimental results demonstrate the scalability of both the merge forest construction and the parallel queries needed in scientific workflows, and contrast this scalability with the two established alternatives that construct variations of a global tree. CCS CONCEPTS• Theory of computation → Distributed algorithms; • Mathematics of computing → Exploratory data analysis.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Topological Queries At Scale: Density Clusters In Cosmologymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Distributed merge forest

Huang

Klacansky

Petruzza

et al. 2021

Proceedings of the ACM International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…The ETL is a function of expected 0D and 1D barcodes. The expected 0D barcode can also be viewed as the expected heights of branching in a random merge tree [90][91][92][93][94][95].…”

Section: Inference On Expected Birth and Death Valuesmentioning

confidence: 99%

Topological data analysis of human brain networks through order statistics

Das

Anand²,

Chung³

2023

PLoS ONE

View full text Add to dashboard Cite

Understanding the common topological characteristics of the human brain network across a population is central to understanding brain functions. The abstraction of human connectome as a graph has been pivotal in gaining insights on the topological properties of the brain network. The development of group-level statistical inference procedures in brain graphs while accounting for the heterogeneity and randomness still remains a difficult task. In this study, we develop a robust statistical framework based on persistent homology using the order statistics for analyzing brain networks. The use of order statistics greatly simplifies the computation of the persistent barcodes. We validate the proposed methods using comprehensive simulation studies and subsequently apply to the resting-state functional magnetic resonance images. We found a statistically significant topological difference between the male and female brain networks.

show abstract

LowFive: In Situ Data Transport for High-Performance Workflows

Peterka

Morozov

Nigmetov

et al. 2023

2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Self Cite

View full text Add to dashboard Cite

We describe LowFive, a new data transport layer based on the HDF5 data model, for in situ workflows. Executables using LowFive can communicate in situ (using in-memory data and MPI message passing), reading and writing traditional HDF5 files to physical storage, and combining the two modes. Minimal and often no source-code modification is needed for programs that already use HDF5. LowFive maintains deep copies or shallow references of datasets, configurable by the user. More than one task can produce (write) data, and more than one task can consume (read) data, accommodating fan-in and fan-out in the workflow task graph. LowFive supports data redistribution from n producer processes to m consumer processes. We demonstrate the above features in a series of experiments featuring both synthetic benchmarks as well as a representative use case from a scientific workflow, and we also compare with other data transport solutions in the literature.

show abstract

Local-global merge tree computation with local exchanges

Cited by 10 publications

References 16 publications

Distributed merge forest

Distributed merge forest

Topological data analysis of human brain networks through order statistics

LowFive: In Situ Data Transport for High-Performance Workflows

Contact Info

Product

Resources

About