A Benchmark of Pre-processing Effect on Single Cell RNA Sequencing Integration Methods

Abstract

Single-cell data analysis can transform the practice of personalised medicine by facilitating characterisation of disease-associated molecular changes across every single cell. Advanced single-cell multimodal assays can now simultaneously measure different types of molecules (e.g., DNA, RNA, Protein) across hundreds of thousands of individual cells to provide a complete molecular readout of the state of the cell. A key analytical challenge is to integrate single-cell measurements across different modalities. In recent years, different methods have been developed to integrate different molecular modalities of single cell measurements, yet there is no systematic evaluation on the performance of different techniques with respect to different pre-processig strategies. Here, we consider a general pipeline of single-cell data analyses comprising normalisation and data integration followed by the most popular dimensionality reduction methods where several algorithms can be adopted to each module in the pipeline. The performance of different algorithm combinations often depends on the sizes and characteristics of datasets. This paper explores the optimal framework of different algorithms using six datasets across diverse modalities, tissues and organisms. We use three evaluation metrics (Silhouette Coefficient Score, Adjusted Rand Index and Calinski-Harabasz Index) to explore the model's overall performance from the two aspects, including clustering performance and time efficiency. We conduct experiments based on combinations of seven normalization methods, four dimensional reduction methods, and five integration methods. Our results demonstrate that for the data integration module, the clustering performance of Seurat and Harmony are more prominent, but the time efficiency of Harmony was better. At the same time, the performance of Seurat for small data sets is superior. For the dimensionality reduction module, the UMAP method shows promising results in compatibility with the integration methods. The data normalization method could be differentiated according to different integration methods.

Omics data integration in computational biology viewed through the prism of machine learning paradigms

“…An optimality criterion is tested at each iteration to assess whether batch mixing is sufficient, using a local purity metric called Local Inverse Simpson's Index (LISI). Due to its simplicity and availability with both Python and R packages, Harmony is widely used today and still achieves respectable results in benchmarks (Anaissi et al, 2022) despite being limited when facing strong batch effects (Luecken et al, 2022).…”