Investigating the effect of changes in model parameters on optimal control policies, time to absorption, and mixing times

Abstract

Many processes in biology and medicine have been modeled using Markov decision processes which provides a rich algorithmic theory for model analysis and optimal control. An optimal control problem for stochastic discrete systems consists of deriving a control policy that dictates how the system will move from one state to another such that the probability of reaching a desired state is maximized. In this paper, we focus on the class of Markov decision processes that is obtained by considering stochastic Boolean networks equipped with control actions. Here, we study the effect of changes in model parameters on optimal control policies. Specifically, we conducted a sensitivity analysis on optimal control policies for a Boolean model of the T-cell large granular lymphocyte (\textit{T-LGL}). For this model, we quantified how the choice of propensity parameters impacts the effectiveness of the optimal policy and then we provide thresholds at which the effectiveness is guaranteed. We also examined the effect on the optimal control policies of the level of noise that is usually added for simulations. Finally, we studied the effect on changes in the propensity parameters on the time to absorption and the mixing time for a Boolean model of the Repressilator.

Phenotype control techniques for Boolean gene regulatory networks

“…These types of interventions are instances where the protocol is not adjusted based on the state of the system, inducing the control only at the front end. This is contrasted with optimal control, where the goal is to find a control policy that specifies the ideal control action for each state [38][39][40][41][42]. Thus, phenotype control theory is primarily concerned with identifying key markers of the system that aid in understanding the various functions of cells and their molecular mechanisms.…”