Control Theory and Experimental Design in Diffusion Processes

Abstract

This paper considers the problem of designing time-dependent, real-time control policies for controllable nonlinear diffusion processes, with the goal of obtaining maximally-informative observations about parameters of interest. More precisely, we maximize the expected Fisher information for the parameter obtained over the duration of the experiment, conditional on observations made up to that time. We propose to accomplish this with a twostep strategy: when the full state vector of the diffusion process is observable continuously, we formulate this as an optimal control problem and apply numerical techniques from stochastic optimal control to solve it. When observations are incomplete, infrequent, or noisy, we propose using standard filtering techniques to first estimate the state of the system, then apply the optimal control policy using the posterior expectation of the state. We assess the effectiveness of these methods in 3 situations: a paradigmatic bistable model from statistical physics, a model of action potential generation in neurons, and a model of a simple ecological system.

Optimal Design for Estimation in Diffusion Processes from First Hitting Times

“…Gradient ascent procedure. Maximization of the MI, (10), is approached as a gradient-based iterative method, where the infinite-dimensional gradient, δI in (14), is calculated by some finite-dimensional approximation. The process involves three basic stages:…”

“…An alternative approach is to maximize the expected Fisher information of the experiment as in [10]. This is especially effective if the Fisher information is available as an analytical formula.…”

“…This is especially effective if the Fisher information is available as an analytical formula. That is the case when one observes trajectories from the diffusion process, as in [10], but not in the much more limited context of observations of hitting times.…”

Timing observations of diffusions

“…We assume that sample paths of this system start from a known initial condition x 0 . By extending [1] and [6], we address the experimental design question:…”

“…We will see that our optimal policy depends on the very parameter, θ, that it is designed to estimate. Section 4 addresses this limitation by adapting the Bayesian machinery proposed by [1] and [6]. Section 5 concludes.…”