scSPARKL: Apache Spark based parallel analytical framework for the downstream analysis of scRNA-seq data

Abstract

As the field of single-cell genomics continues to develop, the generation of large-scale scRNA-seq datasets has become more prevalent. While these datasets offer tremendous potential for shedding light on the complex biology of individual cells, the sheer volume of data presents significant challenges for management and analysis. To address these challenges, a new discipline, known as "big single-cell data science," has emerged. Within this field, a variety of computational tools have been developed to facilitate the processing and interpretation of scRNA-seq data. In this paper, we present a novel parallel analytical framework, scSPARKL, that leverages the power of Apache Spark to enable the efficient analysis of single-cell transcriptomic data. Our methodology incorporates six key operations for dealing with single-cell Big Data, including data reshaping, data preprocessing, cell/gene filtering, data normalization, dimensionality reduction, and clustering. By utilizing Spark's unlimited scalability, fault tolerance, and parallelism, scSPARKL enables researchers to rapidly and accurately analyze scRNA-seq datasets of any size. We demonstrate the utility of our framework through a series of experiments on simulated and real-world scRNA-seq data. Overall, our results suggest that scSPARKL represents a powerful and flexible tool for the analysis of single-cell transcriptomic data, with broad applications across the fields of biology and medicine.