What is a “unimodal” cell population? Using statistical tests as criteria for unimodality in automated gating and quality control

Johnsson, Kerstin; Linderoth, Magnus; Fontes, Magnus

doi:10.1002/cyto.a.23173

Cited by 10 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our report thus far, we have used unimodal simulations to show how random sampling affects statistical results. However, there has been an increased interest in understanding data multimodality in various biological processes 52,53 for which new statistical approaches have been proposed. Methods to simulate multimodal distributions are however not trivial, in part due to the unknown nature of multimodality in biological processes.…”

Section: Resultsmentioning

confidence: 99%

“…The dip test 61 quantifies departures from unimodality and does not require a priori knowledge of potential multimodality and thus information can be easily interpreted from the test statistics and the P-value 62,63 . Although reports and comparative analysis of statistical performance have been described for various multimodality tests ( e.g ., Dip test, Bimodality test, Silverman’s test and likelihood ratio test 64 , and kernel methods), including simpler alternatives that use benchmarks to determine the influence of data outliers 52,53,62,65 , it is important to emphasize that every method depends on its intended application and data set (and data shape), 66 and therefore must be accompanied by the inspection of the data distributions (‘shoulders’, ‘bumps’, and respective ‘valleys’).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Patterns of ‘Analytical Irreproducibility’ in Multimodal Diseases

Basson

Cominelli

Rodriguez-Palacios

2020

Preprint

View full text Add to dashboard Cite

Highlights 17• Multimodal diseases are those in which affected individuals can be divided into subtypes (or 'data 18 modes'); for instance, 'mild' vs. 'severe', based on (unknown) modifiers of disease severity. 20• The role of the microbiome in multimodal diseases has been studied in animals; however, findings 21 are often deemed irreproducible, or unreasonably biased, with pathogenic roles in 95% of reports. 23• As a solution to repeatably, investigators have been told to seek funds to increase the number of 24 human-microbiome donors (N) to increase the reproducibility of animal studies. 26• Herein, we illustrate that although increasing N could help identify statistical effects (patterns of 27 analytical irreproducibility), clinically-relevant information will not always be identified. 29• Depending on which diseases need to be compared, 'random sampling' alone leads to reproducible 30 'patterns of analytical irreproducibility' in multimodal disease simulations. 32• Instead of solely increasing N, we illustrate how disease multimodality could be understood, 33 visualized and used to guide the study of diseases by selecting and focusing on 'disease modes'. 35 36Abstract 37 Multimodal diseases are those in which affected individuals can be divided into subtypes (or 'data 38 modes'); for instance, 'mild' vs. 'severe', based on (unknown) modifiers of disease severity. Studies have 39 shown that despite the inclusion of a large number of subjects, the causal role of the microbiome in 40 human diseases remains uncertain. The role of the microbiome in multimodal diseases has been studied 41 in animals; however, findings are often deemed irreproducible, or unreasonably biased, with pathogenic 42 roles in 95% of reports. As a solution to repeatability, investigators have been told to seek funds to 43 increase the number of human-microbiome donors (N) to increase the reproducibility of animal studies 44 (doi:10.1016/j.cell.2019.12.025). Herein, through simulations, we illustrate that increasing N will not 45 uniformly/universally enable the identification of consistent statistical differences (patterns of analytical 46 irreproducibility), due to random sampling from a population with ample variability in disease and the 47 presence of 'disease data subtypes' (or modes). We also found that studies do not use cluster statistics 48 when needed (97.4%, 37/38, 95%CI=86.5,99.5), and that scientists who increased N, concurrently 49 reduced the number of mice/donor (y=-0.21x, R 2 =0.24; and vice versa), indicating that statistically, 50 scientists replace the disease variance in mice by the variance of human disease. Instead of assuming 51 that increasing N will solve reproducibility and identify clinically-predictive findings on causality, we 52 propose the visualization of data distribution using kernel-density-violin plots (rarely used in rodent 53 studies; 0%, 0/38, 95%CI=6.9e-18,9.1) to identify 'disease data subtypes' to self-correct, guide and 54 promote the personalized investigation of disease subtype mechanisms. 56Key...

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Patterns of ‘Analytical Irreproducibility’ in Multimodal Diseases

Basson

Cominelli

Rodriguez-Palacios

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, personalized research has the potential to identify different functional microbiome subtypes (on a given outcome, e.g., assay or hGM-FMT mice) for one individual. and kernel methods), including simpler alternatives that use benchmarks to determine the in uence of data outliers [52,53,62,65], it is important to emphasize that every method depends on its intended application and data set (and data shape), [66] and therefore must be accompanied by the inspection of the data distributions ('shoulders', 'bumps', and respective 'valleys').…”

Section: Discussionmentioning

confidence: 99%

Patterns of 'Analytical Irreproducibility' in Multimodal Diseases

Basson

Cominelli

Rodriguez-Palacios

2020

Preprint

View full text Add to dashboard Cite

Background: Multimodal diseases are those in which affected individuals can be divided into subtypes (or ‘data modes’); for instance, ‘mild’ vs. ‘severe’, based on (unknown) modifiers of disease severity exemplified in the majority of microbiome-mediated human diseases. Studies have shown that despite the inclusion of a large number of subjects, the causal role of the microbiome in human diseases remains uncertain. The role of the microbiome in multimodal diseases has been studied in animals; however, findings are often deemed irreproducible, or unreasonably biased, with pathogenic roles in 95% of reports. As a solution to repeatability, investigators have been recently recommended to seek funds to increase the number of human-microbiome donors (N) to increase the reproducibility of animal studies. Herein, we outline the constraints of such recommendation. Results: Using published (observed) mean±SD microbiome data from human gut microbiota (hGM)-associated rodent studies, we illustrate through a series of simulations, that increasing N will not uniformly/universally enable the identification of consistent statistical differences (patterns of analytical irreproducibility), due to random sampling from a population with ample variability in disease and the presence of ‘disease data subtypes’ (or modes). To visualize data distribution, we used kernel-density-violin plots (rarely used in rodent studies; 0%, 0/38, 95%CI=6.9e-18,9.1) as a method to identify ‘disease data subtypes’. We also found that hGM preclinical rodent studies do not use cluster statistics when needed (97.4%, 37/38, 95%CI=86.5,99.5), and that scientists who increased N, concurrently reduced the number of mice/donor ( y =-0.21 x, R 2 =0.24 ; and vice versa), indicating that statistically, scientists replace the disease variance in mice by the variance of human disease in their studies. Conclusion: Instead of assuming that increasing N will solve reproducibility and identify clinically-predictive findings on causality in preclinical microbiome studies, we propose the visualization of data distribution using kernel-density’-violin plots to identify ‘disease data subtypes’ to self-correct, guide and promote the personalized investigation of disease subtype mechanisms.

show abstract

“…Determining whether empirical data is multimodal is a difficult task (e.g., (26)). Recently, this issue has been revisited in detail (27). In the context of modeling FCM data (18), it was addressed by using Hartigan's dip test for unimodality (28).…”

Section: Theorymentioning

confidence: 99%

Modeling of cytometry data in logarithmic space: When is a bimodal distribution not bimodal?

et al. 2018

View full text Add to dashboard Cite

Recent efforts in systems immunology lead researchers to build quantitative models of cell activation and differentiation. One goal is to account for the distributions of proteins from single-cell measurements by flow cytometry or mass cytometry as readout of biological regulation. In that context, large cell-to-cell variability is often observed in biological quantities. We show here that these readouts, viewed in logarithmic scale may result in two easily-distinguishable modes, while the underlying distribution (in linear scale) is unimodal. We introduce a simple mathematical test to highlight this mismatch. We then dissect the flow of influence of cell-to-cell variability proposing a graphical model which motivates higher-dimensional analysis of the data. Finally we show how acquiring additional biological information can be used to reduce uncertainty introduced by cell-to-cell variability, helping to clarify whether the data is uni- or bimodal. This communication has cautionary implications for manual and automatic gating strategies, as well as clustering and modeling of single-cell measurements. © 2018 International Society for Advancement of Cytometry.

show abstract

What is a “unimodal” cell population? Using statistical tests as criteria for unimodality in automated gating and quality control

Cited by 10 publications

References 26 publications

Patterns of ‘Analytical Irreproducibility’ in Multimodal Diseases

Patterns of ‘Analytical Irreproducibility’ in Multimodal Diseases

Patterns of 'Analytical Irreproducibility' in Multimodal Diseases

Modeling of cytometry data in logarithmic space: When is a bimodal distribution not bimodal?

Contact Info

Product

Resources

About