Generalized EmbedSOM on quadtree-structured self-organizing maps

Kratochvíl, Miroslav; Koladiya, Abhishek; Vondrášek, Jiřı́

doi:10.12688/f1000research.21642.1

Cited by 9 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The task of the SOM training is to assign values to the neurons so that the training dataset is covered by neighborhoods of the neurons, and, at the same time, that the topology of the neurons is preserved in the trained network. A 2D grid is one of the most commonly used topologies because it simplifies interpretation of the results as neuron values positioned in the 2D space, and related visualization purposes (e.g., EmbedSOM [ 15 ]). At the same time, the trained network can serve as a simple clustering of the input dataset, classifying each data point to its nearest neuron.…”

Section: Methodsmentioning

confidence: 99%

“…To simplify visualization of the results, GigaSOM.jl includes a parallel reimplementation of the EmbedSOM algorithm in Julia [ 15 ], which quickly provides interpretable visualizations of the cell distribution within the datasets. EmbedSOM computes an embedding of the cells to 2D space, similarly as the popular t-SNE or UMAP algorithms [ 31 , 32 ].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

et al. 2020

Self Cite

View full text Add to dashboard Cite

Background The amount of data generated in large clinical and phenotyping studies that use single-cell cytometry is constantly growing. Recent technological advances allow the easy generation of data with hundreds of millions of single-cell data points with >40 parameters, originating from thousands of individual samples. The analysis of that amount of high-dimensional data becomes demanding in both hardware and software of high-performance computational resources. Current software tools often do not scale to the datasets of such size; users are thus forced to downsample the data to bearable sizes, in turn losing accuracy and ability to detect many underlying complex phenomena. Results We present GigaSOM.jl, a fast and scalable implementation of clustering and dimensionality reduction for flow and mass cytometry data. The implementation of GigaSOM.jl in the high-level and high-performance programming language Julia makes it accessible to the scientific community and allows for efficient handling and processing of datasets with billions of data points using distributed computing infrastructures. We describe the design of GigaSOM.jl, measure its performance and horizontal scaling capability, and showcase the functionality on a large dataset from a recent study. Conclusions GigaSOM.jl facilitates the use of commonly available high-performance computing resources to process the largest available datasets within minutes, while producing results of the same quality as the current state-of-art software. Measurements indicate that the performance scales to much larger datasets. The example use on the data from a massive mouse phenotyping effort confirms the applicability of GigaSOM.jl to huge-scale studies.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…A 2-dimensional grid is one of the most commonly used topologies, because it simpli es interpretation of the results as neuron values positioned in the 2-dimensional space, and related visualization purposes (e.g. EmbedSOM [14]). At the same time, the trained network can serve as a simple clustering of the input dataset, classifying each data point to its nearest neuron.…”

Section: Methodsmentioning

confidence: 99%

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

Kratochvíl

Hunewald

Heirendt

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background: The amount of data generated in large clinical and phenotyping studies that use single-cell cytometry is constantly growing. Recent technological advances allow to easily generate data with hundreds of millions of single-cell data points with more than 40 parameters, originating from thousands of individual samples. The analysis of that amount of high-dimensional data becomes demanding in both hardware and software of high-performance computational resources. Current software tools often do not scale to the datasets of such size; users are thus forced to downsample the data to bearable sizes, in turn losing accuracy and ability to detect many underlying complex phenomena. Results: We present GigaSOM.jl, a fast and scalable implementation of clustering and dimensionality-reduction for flow and mass cytometry data. The implementation of GigaSOM.jl in the high-level and high-performance programming language Julia makes it accessible to the scientific community, and allows for efficient handling and processing of datasets with billions of data points using distributed computing infrastructures. We describe the design of GigaSOM.jl, measure its performance and horizontal scaling capability, and showcase the functionality on a large dataset from a recent study. Conclusions: GigaSOM.jl facilitates utilization of the commonly available high-performance computing resources to process the largest available datasets within minutes, while producing results of the same quality as the current state-of-art software. Measurements indicate that the performance scales to much larger datasets. The example use on the data from an massive mouse phenotyping effort confirms the applicability of GigaSOM.jl to huge-scale studies.

show abstract

“…EmbedSOM is a visualization-oriented method of non-linear dimensionality reduction that works by describing a high-dimensional point by its location relative to landmarks equipped with a topology, and reproducing the point in a lowdimensional space using an explicit low-dimensional projection of the landmarks with the same topology [15]. The ability to e ectively work with a simpli ed model of the data di erentiates it from other dimensionality reduction methods; in turn it o ers superior performance by reducing the amount of necessary computation as well as by opening parallelization potential, since the computations of the projections of many individual points are independent.…”

Section: Landmark-directed Dimensionality Reductionmentioning

confidence: 99%

“…The most popular methods, typically based on neighborhood embedding computed by stochastic descent, force-based layouting or neural autoencoders, reach applicability limits when the dataset size is too large. To tackle the limitations, we have previously developed EmbedSOM [15], a dimensionality reduction and visualization algorithm based on self-organizing maps (SOMs) [13]. EmbedSOM provided an order-of-magnitude speedup on datasets typical for the single-cell cytometry data visualization while retaining competitive quality of the results.…”

mentioning

confidence: 99%

Scalable semi-supervised dimensionality reduction with GPU-accelerated EmbedSOM

Šmelko¹,

Molnárová²,

Kratochvíl³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Dimensionality reduction methods have found vast application as visualization tools in diverse areas of science. Although many di erent methods exist, their performance is o en insu cient for providing quick insight into many contemporary datasets, and the unsupervised mode of use prevents the users from utilizing the methods for dataset exploration and netuning the details for improved visualization quality. We present BlosSOM, a high-performance semi-supervised dimensionality reduction so ware for interactive user-steerable visualization of high-dimensional datasets with millions of individual data points. BlosSOM builds on a GPUaccelerated implementation of the EmbedSOM algorithm, complemented by several landmarkbased algorithms for interfacing the unsupervised model learning algorithms with the user supervision. We show the application of BlosSOM on realistic datasets, where it helps to produce high-quality visualizations that incorporate user-speci ed layout and focus on certain features. We believe the semi-supervised dimensionality reduction will improve the data visualization possibilities for science areas such as single-cell cytometry, and provide a fast and e cient base methodology for new directions in dataset exploration and annotation.

show abstract

Generalized EmbedSOM on quadtree-structured self-organizing maps

Cited by 9 publications

References 12 publications

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasets

Scalable semi-supervised dimensionality reduction with GPU-accelerated EmbedSOM

Contact Info

Product

Resources

About