The Definition and Measurement of Heterogeneity

Abstract

Heterogeneity is an important concept in psychiatric research and science more broadly. It negatively impacts effect size estimates under case-control paradigms, and it exposes important flaws in our existing categorical nosology. Yet, our field has no precise definition of heterogeneity proper. We tend to quantify heterogeneity by measuring associated correlates such as entropy or variance: practices which are akin to accepting the radius of a sphere as a measure of its volume. Under a definition of heterogeneity as the degree to which a system deviates from perfect conformity, this paper argues that its proper measure roughly corresponds to the size of a system’s event/sample space, and has units known as numbers equivalent. We arrive at this conclusion through focused review of more than 100 years of (re)discoveries of indices by ecologists, economists, statistical physicists, and others. In parallel, we review psychiatric approaches for quantifying heterogeneity, including but not limited to studies of symptom heterogeneity, microbiome biodiversity, cluster-counting, and time-series analyses. We argue that using numbers equivalent heterogeneity measures could improve the interpretability and synthesis of psychiatric research on heterogeneity. However, significant limitations must be overcome for these measures—largely developed for economic and ecological research—to be useful in modern translational psychiatric science.

Multiplicative Decomposition of Heterogeneity in Mixtures of Continuous Distributions

“…Parametric Gaussian mixtures are an important class of models commonly used in mixed-effects meta-analyses [ 21 ], where one models the effect size of each of studies as Gaussians whose means are themselves Gaussian distributed with “true” effect-size and variance . The variance of the true effect, , is often taken as an index of between-study heterogeneity, but unfortunately variance does not satisfy the replication principle [ 4 ]. A parametric Gaussian mixture can also be used to measure the effective number of natural images embedded in the real valued latent space of a variational autoencoder (a probabilistic deep learning model used to learn compressed representations of high-dimensional data) [ 5 ].…”

“…Ecologists are interested in the heterogeneity of ecosystems’ biological composition (biodiversity) [ 1 ], economists are interested in the heterogeneity of resource ownership (wealth equality) [ 2 ], and medical researchers and physicians are interested in the heterogeneity of diseases and their presentations [ 3 ]. Using Rényi heterogeneity [ 3 , 4 , 5 ], which for categorical random variables corresponds to ecologists’ Hill numbers [ 6 ] and economists’ Hannah–Kay indices [ 7 ], one can measure a system’s heterogeneity as its effective number of distinct configurations.…”

“…However, many data are better modeled by continuous embeddings, including word semantics [ 10 , 11 , 12 ], genetic population structure [ 13 ], and natural images [ 14 ]. Unfortunately, there is considerably less understood about how to decompose Rényi heterogeneity in such cases where data are represented on non-categorical spaces [ 4 ]. Although there are decomposable functional diversity indices expressed in numbers equivalent, they require categorical partitioning of the data (in order to supply species (dis)similarity matrices) [ 15 , 16 , 17 , 18 ] and setting sensitivity or threshold parameters for (dis)similarities [ 16 , 18 ].…”

“…Although there are decomposable functional diversity indices expressed in numbers equivalent, they require categorical partitioning of the data (in order to supply species (dis)similarity matrices) [ 15 , 16 , 17 , 18 ] and setting sensitivity or threshold parameters for (dis)similarities [ 16 , 18 ]. For many research applications, such as those in psychiatry [ 3 , 4 , 19 ] or involving unsupervised learning [ 13 , 14 ], we may not have categorical partitions of the observable space that are valid, reliable, and of semantic relevance. If we are to apply Rényi heterogeneity to such continuous-space systems, then we must demonstrate that its multiplicative decomposition of -heterogeneity into and components is retained.…”

Measuring Heterogeneity in Normative Models as the Effective Number of Deviation Patterns

“…Heterogeneity measurement has been studied for more than a century [11], but most indices, such as variance and entropies [12][13][14], have inconsistent units and can scale counterintuitively [15,16]. Conversely, ecologists and others have adopted the Rényi heterogeneity family of indices as a "true diversity" index Draft Version June 6, 2020 [17][18][19][20].…”