Explanation of Variability and Removal of Confounding Factors from Data through Optimal Transport

Tabak, Esteban G.; Trigila, Giulio

doi:10.1002/cpa.21706

Cited by 24 publications

(38 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Wasserstein barycenter and its computation have been studied in many contexts, such as optimal transport theory (Cuturi and Doucet, 2014;Anderes et al, 2016). In Tabak and Trigila (2018), the Wasserstein barycenter has been suggested as a method to remove nuisance variation in high-throughput biological experiments. Two key ingredients of the Wasserstein barycenter are that (i) the nuisance variation is removed in the sense that a number of distinct distributions are transformed into a common distribution, and hence become indistinguishable; and (ii) the distributions are minimally perturbed by the transformations.…”

Section: General Approachmentioning

confidence: 99%

Peer Review #2 of "Correcting nuisance variation using Wasserstein distance (v0.1)"

2020

View full text Add to dashboard Cite

Profiling cellular phenotypes from microscopic imaging can provide meaningful biological information resulting from various factors affecting the cells. One motivating application is drug development: morphological cell features can be captured from images, from which similarities between different drug compounds applied at different doses can be quantified. The general approach is to find a function mapping the images to an embedding space of manageable dimensionality whose geometry captures relevant features of the input images. An important known issue for such methods is separating relevant biological signal from nuisance variation. For example, the embedding vectors tend to be more correlated for cells that were cultured and imaged during the same week than for those from different weeks, despite having identical drug compounds applied in both cases. In this case, the particular batch in which a set of experiments were conducted constitutes the domain of the data; an ideal set of image embeddings should contain only the relevant biological information (e.g. drug effects). We develop a general framework for adjusting the image embeddings in order to 'forget' domain-specific information while preserving relevant biological information. To achieve this, we minimize a loss function based on distances between marginal distributions (such as the Wasserstein distance) of embeddings across domains for each replicated treatment. For the dataset we present results with, the only replicated treatment happens to be the negative control treatment, for which we do not expect any treatment-induced cell morphology changes. We find that for our transformed embeddings (i) the underlying geometric structure is not only preserved but the embeddings also carry improved biological signal; and (ii) less domainspecific information is present.

show abstract

Section: General Approachmentioning

confidence: 99%

Peer Review #2 of "Correcting nuisance variation using Wasserstein distance (v0.1)"

2020

View full text Add to dashboard Cite

show abstract

“…In order to provide a more flexible framework for data science applications, sample-based techniques to solve the OT problem were developed in [19,7,20]. A central question to address when posing sample-based OT problems is the meaning of the push-forward condition T # µ = ν when µ and ν are only known through samples {x i }, {y j }.…”

Section: Introductionmentioning

confidence: 99%

Adaptive optimal transport

Essid

Laefer²,

Tabak

2019

Information and Inference: A Journal of the IMA

View full text Add to dashboard Cite

An adaptive, adversarial methodology is developed for the optimal transport problem between two distributions µ and ν, known only through a finite set of independent samples (xi)i=1..N and (yj)j=1..M . The methodology automatically creates features that adapt to the data, thus avoiding reliance on a priori knowledge of data distribution. Specifically, instead of a discrete point-bypoint assignment, the new procedure seeks an optimal map T (x) defined for all x, minimizing the Kullback-Leibler divergence between (T (xi)) and the target (yj). The relative entropy is given a sample-based, variational characterization, thereby creating an adversarial setting: as one player seeks to push forward one distribution to the other, the second player develops features that focus on those areas where the two distributions fail to match. The procedure solves local problems matching consecutive, intermediate distributions between µ and ν. As a result, maps of arbitrary complexity can be built by composing the simple maps used for each local problem. Displaced interpolation is used to guarantee global from local optimality. The procedure is illustrated through synthetic examples in one and two dimensions. arXiv:1807.00393v2 [math.OC]From the samples provided, we seek a map T that would perform the transport well when applied to other independent realizations of the unknown distributions µ, ν. We can assume that the source and target distribution are close:Remark. Solving the problem for nearby distributions is the building block of a general procedure for arbitrary distributions and for finding the Wasserstein barycenter of distributions [7]. This more general procedure is presented in Section 2.4.

show abstract

“…The authors have recently developed Tabak and Trigila [2017] a unified framework for the explanation of variability, based on extensions of the mathematical theory of optimal transport. In this framework, one estimates the conditional probability distributions ρ(x|z) by mapping them to their Wasserstein barycenter Agueh and Carlier [2011].…”

Section: Introductionmentioning

confidence: 99%

“…In this framework, one estimates the conditional probability distributions ρ(x|z) by mapping them to their Wasserstein barycenter Agueh and Carlier [2011]. It was shown in Tabak and Trigila [2017] that principal components emerge naturally from the methodology's simplest setting, with maps restricted to rigid translations, and hence capturing not the full conditional probability distribution ρ(x|z) but only its conditional expectationx(z). From here stems one of this article's legs: if one thinks of principal components in terms of explanation of variability, it is natural to consider more general scenarios, where the covariates, though still with values a priori unknown, are associated with particular attributes such as space, time or activity networks, and hence required to be smooth in the topologies associated with these attributes.…”

Section: Introductionmentioning

confidence: 99%

“…For clarity of exposition, rather than posing the proposed procedure from the start, this article follows its conceptual evolution through a series of simple steps. Thus, after this Introduction, section 2 first briefly summarizes the relation between explanation of variability and optimal transport developed in Tabak and Trigila [2017] (subsection 2.1), then restricts the maps defining the Wasserstein barycenter to rigid translations (subsection 2.2) and further restricts to linear the functional dependence of these translations on the covariates z (subsection 2.3). Here one recovers least-square-error linear regression, the very humble starting point of our conceptual development.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Conditional expectation estimation through attributable components

Tabak

Trigila

2018

Information and Inference: A Journal of the IMA

Self Cite

View full text Add to dashboard Cite

A general methodology is proposed for the explanation of variability in a quantity of interest x in terms of covariates z = (z1, …, zL). It provides the conditional mean $\bar{x}(z)$ as a sum of components, where each component is represented as a product of non-parametric one-dimensional functions of each covariate zl that are computed through an alternating projection procedure. Both x and the zl can be real or categorical variables; in addition, some or all values of each zl can be unknown, providing a general framework for multi-clustering, classification and covariate imputation in the presence of confounding factors. The procedure can be considered as a preconditioning step for the more general determination of the full conditional distribution $\boldsymbol{\rho}(x|z) $ through a data-driven optimal-transport barycenter problem. In particular, just iterating the procedure once yields the second order structure (i.e. the covariance) of $\boldsymbol{\rho}(x|z) $. The methodology is illustrated through examples that include the explanation of variability of ground temperature across the continental United States and the prediction of book preference among potential readers.

show abstract

Explanation of Variability and Removal of Confounding Factors from Data through Optimal Transport

Cited by 24 publications

References 21 publications

Peer Review #2 of "Correcting nuisance variation using Wasserstein distance (v0.1)"

Peer Review #2 of "Correcting nuisance variation using Wasserstein distance (v0.1)"

Adaptive optimal transport

Conditional expectation estimation through attributable components

Contact Info

Product

Resources

About