Investigating the Effect of Changes in Model Parameters on Optimal Control Policies, Time to Absorption, and Mixing Times

Abstract

Many processes in biology and medicine have been modeled using Markov decision processes which provides a rich algorithmic theory for model analysis and optimal control. An optimal control problem for stochastic discrete systems consists of deriving a control policy that dictates how the system will move from one state to another such that the probability of reaching a desired state is maximized. In this paper, we focus on the class of Markov decision processes that is obtained by considering stochastic Boolean networks equipped with control actions. Here, we study the effect of changes in model parameters on optimal control policies. Specifically, we conducted a sensitivity analysis on optimal control policies for a Boolean model of the T-cell large granular lymphocyte (\textit{T-LGL}). For this model, we quantified how the choice of propensity parameters impacts the effectiveness of the optimal policy and then we provide thresholds at which the effectiveness is guaranteed. We also examined the effect on the optimal control policies of the level of noise that is usually added for simulations. Finally, we studied the effect on changes in the propensity parameters on the time to absorption and the mixing time for a Boolean model of the Repressilator.