Schrödinger Bridge Samplers

Abstract

Consider a reference Markov process with initial distribution π 0 and transition kernels {M t } t∈[1:T ] , for some T ∈ N. Assume that you are given distribution π T , which is not equal to the marginal distribution of the reference process at time T . In this scenario, Schrödinger addressed the problem of identifying the Markov process with initial distribution π 0 and terminal distribution equal to π T which is the closest to the reference process in terms of Kullback-Leibler divergence. This special case of the so-called Schrödinger bridge problem can be solved using iterative proportional fitting, also known as the Sinkhorn algorithm. We leverage these ideas to develop novel Monte Carlo schemes, termed Schrödinger bridge samplers, to approximate a target distribution π on R d and to estimate its normalizing constant. This is achieved by iteratively modifying the transition kernels of the reference Markov chain to obtain a process whose marginal distribution at time T becomes closer to π T = π, via regression-based approximations of the corresponding iterative proportional fitting recursion. We report preliminary experiments and make connections with other problems arising in the optimal transport, optimal control and physics literatures.

Quantitative Uniform Stability of the Iterative Proportional Fitting Procedure

“…More recently however Schödinger bridges and entropy-regularised OT are being studied for their own sake, finding applications in control, computational statistics and machine learning, see e.g. Bernton et al (2019); Chen et al (2021); Corenflos et al (2021); De Bortoli et al (2021); Huang et al (2021); Vargas et al (2021). In these applications, the entropy regularisation may be a desirable feature rather than an approximation, and the main source of error is the fact that the marginal distributions are typically intractable and often approximated by empirical versions.…”

An optimal control approach to particle filtering

“…When the cost is of the form (8), it turns out that the above nonlinear optimal control problem can be solved in a linear manner [17,16,18,4,5,19,20,21,22,2,3,23]. One way to see it is through the logarithmic transformation [24] of the HJB equation (10).…”

“…There are many methods that can generate suboptimal controller for (28), including differential dynamic programming (DDP) [27] and iterative linear quadratic regulator (iLQR) [28]. One can also start from the original smoothing problem for (21) and adopt suboptimal smoothing methods such as extended Rauch-Rung-Striebel (ERTS) [29]. These suboptimal smoothing methods induce suboptimal π 0 and u for (28).…”

Deterministic particle flows for constraining stochastic nonlinear systems

“…for estimating the free energy differences between two equilibrium states [18,19], or for identifying optimal protocols that drive a system from one equilibrium to another in finite time [20]. Similar problems appear also often in chemistry, biology, finance, and engineering, required for computation of rare event probabilities [21,22], state estimation of partially observed systems [23][24][25], or for precise manipulation of stochastic systems to target states [26,27] with applications in artificial selection [28,29], motor control [30], epidemiology, and more [31][32][33][34][35][36]. Albeit the prior developments, the problem of controlling nonlinear systems in the presence of random fluctuations remains still considerably challenging.…”