Systematic Design, Generation, and Application of Synthetic Datasets for Flow Cytometry

Abstract

Application of synthetic datasets in training and validation of analysis tools have led to improvements in many decision-making tasks in a range of domains from computer vision to digital pathology. Synthetic datasets overcome the constraints of real-world datasets, namely difficulties in collection and labelling, expense, time and privacy concerns.In flow cytometry, real cell-based datasets are limited by properties such as size, number of parameters, distance between cell populations and distributions, and are often focused on a narrow range of disease or cell types. Researchers in some cases have designed these desired properties into synthetic datasets, however operators have implemented them in inconsistent approaches and there is a scarcity of publicly available, high-quality synthetic datasets.In this research, we propose a method to systematically design and generate flow cytometry synthetic datasets with highly controlled characteristics. We demonstrate the generation of two-cluster synthetic datasets with specific degrees of separation between cell populations, and of non-normal distributions with increasing levels of skewness and orientations of skew pairs. We apply our synthetic datasets to test the performance of a popular automated cell populations identification software, SPADE3, and define the region where the software performance decreases as the clusters get closer together.Application of the synthetic skewed dataset suggests the software is capable of processing non-normal data. We calculate the classification accuracy of SPADE3 with robustness not achievable with real-world datasets. Our approach aims to advance research towards generation of high-quality synthetic flow cytometry datasets, and to increase their awareness among the community.The synthetic datasets can be utilised in benchmarking studies that critically evaluate cell population identification tools and help illustrate potential digital platform inconsistencies.

Assessment of Automated Flow Cytometry Data Analysis Tools within Cell and Gene Therapy Manufacturing

“…Our previous work on the inter-comparison between synthetic and a real dataset showed clear correlation among cell distribution characteristics examples [ 21 ]. Consequently, for this particular cross-platform comparison we are confident that the synthetic data mirrors, to an appropriate level, the key characteristics of low dimensionality cluster data, demonstrates design flexibility and application, and allows for traceable benchmarking (absolute accuracy and repeatability), without the further need to run the platforms through further real data.…”

“…A recognised limitation of this work is that the number of markers simulated is lower than those in real data (usually -colour panels) because a priority in this study has been to understand and benchmark how algorithms behave with two or three clusters before introducing further complexities into the datasets. Noting the successful referencing and correlation study we have already completed between synthetic and real data [ 21 ], overall, real data have been excluded from this initial research because they are significantly more complex, containing sources of variation from upstream processes and noise components that cannot be controlled to transparently understand the ’black box’ nature of the algorithms investigated. Additionally, it is very difficult to achieve absolute cell counts for real data, so defining measurement accuracy (a critical component of this study) would not be possible.…”

“…In order to perform a fair comparison between different automated data analysis software, synthetic flow cytometry reference datasets were designed and generated (as described in [ 21 ]). Out of the commonly recognised data characteristics or potential statistical attributes identified, we targeted the separation and the skew characteristics to control and modify in our datasets, because these properties had not been addressed in previous work and/or the designs had not been approached in a systematic manner.…”

“…In order to address these issues, synthetic flow cytometry datasets have been previously developed to aid validation of automated cell population identification tools and were shown to successfully mimic real cell data [ 21 ]. These synthetic datasets demonstrated clear similarities in cell distribution characteristics when compared against real-world flow cytometry data ( Figure 1 ), and therefore can be used as credible substitutes to represent actual experimental data.…”