FastPG: Fast clustering of millions of single cells

Abstract

Current single-cell experiments can produce datasets with millions of cells. Unsupervised clustering can be used to identify cell populations in single-cell analysis but often leads to interminable computation time at this scale. This problem has previously been mitigated by subsampling cells, which greatly reduces accuracy. We built on the graph-based algorithm PhenoGraph and developed FastPG which has the same cell assignment accuracy but is on average 27x faster in our tests. FastPG also outperforms two other fast clustering methods, FlowSOM and PARC.Mass cytometry measures proteins abundances at the single-cell level. This technology can measure 100,000-200,000 cells per sample and up to around 40 protein markers simultaneously 1 . Unsupervised clustering is a common task in single-cell data analysis, with the goal of unbiasedly identifying known and unknown cell-types based on protein markers. Clustering cells across multiple samples can be optimal for some experiments to gain a deeper understanding of how a particular cell population changes in a disease state. Most clustering algorithms cannot efficiently handle a very large number of collected cells. These algorithms are severely restricted by computational time and available system memory. Common approaches to mitigate these issues include subsampling and meta-clustering 2-4 . The disadvantage to both is loss of information and lower potential to discover rare cell types 4 .A commonly used and robust method for mass cytometry data is PhenoGraph 5 , which uses a graph-based approach to unbiasedly identify clusters, or cell types. Here we present FastPG (Figure 1), a modified version of PhenoGraph which was developed to maximize computational efficiency with no loss of cell assignment accuracy. PhenoGraph is composed of three main steps: (1) creating a k-nearest neighbor network (kNN) of single cells using a distance measurement calculated from their protein marker abundances, (2) adding weights to the network through calculating Jaccard index, and (3) partitioning cells into coherent cell populations using the Louvain algorithm 6 . We modified PhenoGraph as follows: first, we replaced the kNN step with a fast kNN approximation, Hierarchical Navigable Small World (HNSW), which uses logarithmic scaling due to the hierarchical structure of the search space 7 . We next parallelized the Jaccard index step for multithreaded execution, and lastly, we replaced the Louvain algorithm with a fast parallelized version, Grappolo 8 .To test for cell assignment accuracy, we benchmarked FastPG against PhenoGraph, PARC 4 , and FlowSOM 2 . We compared FastPG to PhenoGraph to ensure our method was getting similar accuracy, and FastPG to PARC and FlowSOM due to the published fast speed of these algorithms. Briefly, PARC's first step employs HNSW, the same kNN approximation as FastPG. PARC next uses a graph-pruning method followed by Leiden 9 for community detection.

Workflow for high-dimensional flow cytometry analysis of T cells from tumor metastases

“…To further dissect the data set, we applied the FlowSOM algorithm ( Van Gassen et al, 2015 ), which could not selectively assign iNKT cells to any cluster or meta-cluster even though they were tightly grouped on the t-SNE map ( Fig S16A and B ). To overcome these problems, we implemented our recently described cytoChain application ( Manfredi et al, 2021 ) with FastPhenoGraph (FastPG) algorithm ( Fig S16C ) ( Bodenheimer et al, 2020 Preprint ), which computed 28 clusters that were superimposed on the t-SNE map, allowing both the visualization of their distribution within the data set ( Fig 8B ) and of their phenotype by a heat-map generated based on the fluorescence intensity associated to each expressed marker ( Fig 8C ). For instance, a specific phenotype signature was found to be enriched among CD8 T cells for clusters 21, 17, 24, 13, 27, and 23 that expressed both tissue residence (CD69-CD103) and activation/exhaustion markers to a variable extent (i.e., CD39, CD95, TIGIT, 2B4, PD1, HLA-DR, ICOS, and GITR).…”

“…If multiple .FCS files are uploaded they must be merged (concatenated) before the dimensionality reduction process. Clustering on the reported CRC-LM data set was run both with by FlowSOM ( Van Gassen et al, 2015 ) and FastPG ( Bodenheimer et al, 2020 Preprint ) algorithms. In our case, we estimated that FastPG outperformed FlowSOM.…”

DISCO: a database of Deeply Integrated human Single-Cell Omics data

“…To visualize the integrated data, we performed dimension reduction by PCA and used the top 50 PCs for Uniform Manifold Approximation and Projection (UMAP) ( 21 ). FastPG, a fast phenograph-like clustering method, was used to cluster the cells ( 22 ). Finally, we performed cell type identification of clusters using CELLiD.…”

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

“…Consequently, efficient parallel algorithms can play a significant role in enabling single-cell analysis. The ExaGraph team integrated their scalable community detection method with a newly developed tool named FastPG (Bodenheimer et al, 2020). FastPG builds on the state-of-the-art-method graph-based algorithm PhenoGraph by parallelizing key steps in the workflow, where the final step is clustering using Grappolo .…”

“…Using a set of standard datasets with known ground truth, the team demostrates that FastPG has the same cell assignment accuracy but is on average 27× faster than other tools. FastPG also has higher cell assignment accuracy than two other fast clustering methods, FlowSOM and PARC (Bodenheimer et al, 2020).…”