Solving Schrödinger Bridges via Maximum Likelihood

Vargas, Francisco; Thodoroff, Pierre; Lamacraft, Austen; Lawrence, Neil D.

doi:10.3390/e23091134

Cited by 23 publications

(36 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6). In contrast, Vargas et al [28] and Bunne et al [3] estimated the sample trajectories of biological systems using the SDE solution to the SB problem. Vargas et al [28] proposed an iterative proportional Table 2: Evaluation results for population-level dynamics at time of observation for scRNA-seq data.…”

Section: Discussionmentioning

confidence: 99%

“…In contrast, Vargas et al [28] and Bunne et al [3] estimated the sample trajectories of biological systems using the SDE solution to the SB problem. Vargas et al [28] proposed an iterative proportional Table 2: Evaluation results for population-level dynamics at time of observation for scRNA-seq data. The MDD value at t k is computed between the ground-truth and the samples predicted from the previous ground-truth samples at t k−1 for each k = 1, 2, 3 and 4.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Neural Lagrangian Schrödinger Bridge

Koshizuka¹,

Sato²

2022

Preprint

View full text Add to dashboard Cite

Population dynamics is the study of temporal and spatial variation in the size of populations of organisms and is a major part of population ecology. One of the main difficulties in analyzing population dynamics is that we can only obtain observation data with coarse time intervals from fixed-point observations due to experimental costs or other constraints. Recently, modeling population dynamics by using continuous normalizing flows (CNFs) and dynamic optimal transport has been proposed to infer the expected trajectory of samples from a fixed-point observed population. While the sample behavior in CNF is deterministic, the actual sample in biological systems moves in an essentially random yet directional manner. Moreover, when a sample moves from point A to point B in dynamical systems, its trajectory is such that the corresponding action has the smallest possible value, known as the principle of least action. To satisfy these requirements of the sample trajectories, we formulate the Lagrangian Schrödinger bridge (LSB) problem and propose to solve it approximately using neural SDE with regularization. We also develop a model architecture that enables faster computation. Our experiments show that our solution to the LSB problem can approximate the dynamics at the population level and that using the prior knowledge introduced by the Lagrangian enables us to estimate the trajectories of individual samples with stochastic behavior.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Neural Lagrangian Schrödinger Bridge

Koshizuka¹,

Sato²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For instance, VESDE [1] satisfies f ≡ 0 with G = dσ 2 (t)/ dtI and VPSDE [1] satisfies f = − 1 2 β(t)x t ∝ x t with G = β(t)I. Few concurrent works have expanded linear diffusions to nonlinear diffusions by 1) transforming the data diffusion into a latent diffusion using VAE in LSGM [13], 2) approximating the nonlinear diffusion to a discretized diffusion with a nonlinear drift using a flow model in DiffFlow [14], and 3) reformulating the diffusion model into a Schrodinger Bridge Problem (SBP) [15][16][17]. We further analyze these approaches in Section 5.…”

Section: Preliminarymentioning

confidence: 99%

“…SBP [15][16][17] is a problem of min ρ θ ∈P(pr,π) D KL (ρ θ µ), where P(p r , π) is the collection of path measure with p r and π as its marginal distributions at t = 0 and t = T , respectively. It is a biconstrained optimization problem as any path measure on a search space should satisfy boundary conditions at both t = 0 and t = T with p 0 = p r and p T = π, respectively.…”

Section: Related Work: Comparison To Data Diffusion Learning Modelsmentioning

confidence: 99%

“…This regularization restricts the score network from not deviating S div too far by keeping the rotation term, u θ t , being consistently small. Consequently, a fastly converging rotation term is advantageous in reducing the variational gap (see Inequality (15)), and this regularization helps the MLE training of DDPM++. Proposition 7 proves that S div is identical to a class of score functions that have symmetric derivatives.…”

Section: B3 Restricting Search Space Of S θ Into S DIVmentioning

confidence: 99%

See 1 more Smart Citation

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

Kim¹,

Na²,

Kwon³

et al. 2022

Preprint

View full text Add to dashboard Cite

Whereas diverse variations of diffusion models exist, expanding the linear diffusion into a nonlinear diffusion process is investigated only by a few works. The nonlinearity effect has been hardly understood, but intuitively, there would be more promising diffusion patterns to optimally train the generative distribution towards the data distribution. This paper introduces such a data-adaptive and nonlinear diffusion process for score-based diffusion models. The proposed Implicit Nonlinear Diffusion Model (INDM) learns the nonlinear diffusion process by combining a normalizing flow and a diffusion process. Specifically, INDM implicitly constructs a nonlinear diffusion on the data space by leveraging a linear diffusion on the latent space through a flow network. This flow network is the key to forming a nonlinear diffusion as the nonlinearity fully depends on the flow network. This flexible nonlinearity is what improves the learning curve of INDM to nearly MLE training, compared against the non-MLE training of DDPM++, which turns out to be a special case of INDM with the identity flow. Also, training the nonlinear diffusion empirically yields a sampling-friendly latent diffusion that the sample trajectory of INDM is closer to an optimal transport than the trajectories of previous research. In experiments, INDM achieves the state-of-the-art FID on CelebA. * Equal contribution Preprint. Under review.

show abstract

Bayesian learning via neural Schrödinger–Föllmer flows

et al. 2022

Self Cite

View full text Add to dashboard Cite

In this work we explore a new framework for approximate Bayesian inference in large datasets based on stochastic control. We advocate stochastic control as a finite time and low variance alternative to popular steady-state methods such as stochastic gradient Langevin dynamics. Furthermore, we discuss and adapt the existing theoretical guarantees of this framework and establish connections to already existing VI routines in SDE-based models.

show abstract

Solving Schrödinger Bridges via Maximum Likelihood

Cited by 23 publications

References 32 publications

Neural Lagrangian Schrödinger Bridge

Neural Lagrangian Schrödinger Bridge

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

Bayesian learning via neural Schrödinger–Föllmer flows

Contact Info

Product

Resources

About