Determining controllability of sepsis using genetic algorithms on a proxy agent-based model of systemic inflammation

Abstract

Sepsis, a manifestation of the body's inflammatory response to injury and infection, has a mortality rate of between 28%-50% and affects approximately 1 million patients annually in the United States. Currently, there are no therapies targeting the cellular/molecular processes driving sepsis that have demonstrated the ability to control this disease process in the clinical setting. We propose that this is in great part due to the considerable heterogeneity of the clinical trajectories that constitute clinical "sepsis," and that determining how this system can be controlled back into a state of health requires the application of concepts drawn from the field of dynamical systems. In this work, we consider the human immune system to be a random dynamical system, and investigate its potential controllability using an agent-based model of the innate immune response (the Innate Immune Response ABM or IIRABM) as a surrogate, proxy system. Simulation experiments with the IIRABM provide an explanation as to why single/limited cytokine perturbations at a single, or small number of, time points is unlikely to significantly improve the mortality rate of sepsis. We then use genetic algorithms (GA) to explore and characterize multi-targeted control strategies for the random dynamical immune system that guide it from a persistent, non-recovering inflammatory state (functionally equivalent to the clinical states of systemic inflammatory response syndrome (SIRS) or sepsis) to a state of health. We train the GA on a single parameter set with multiple stochastic replicates, and show that while the calculated results show good generalizability, more advanced strategies are needed to achieve the goal of adaptive personalized medicine. This work evaluating the extent of interventions needed to control a simplified surrogate model of sepsis provides insight into the scope of the clinical challenge, and can serve as a guide on the path towards true "precision control" of sepsis. Author summarySepsis, characterized by the body's inflammatory response to injury and infection, has a mortality rate of between 28%-50% and affects approximately 1 million patients annually in the United States. Currently, there are no therapies targeting the cellular/molecular processes driving sepsis that have demonstrated the ability to control this disease process. In this work, we utilize a computational model of the human immune response to infectious injury to offer an explanation as to why previously attempted treatment strategies are inadequate and why the current approach to drug/therapy-development is inadequate. We then use evolutionary computation algorithms to explore drug-intervention space using this same computational model. This allows us to characterize the scale and scope of interventions needed to successfully control sepsis, as well as the types of data needed to derive these interventions. We demonstrate that multi-point and time-dependent varying controls are necessary and able to control the cytokine network dynamics of the im...

Precision medicine as a control problem: Using simulation and deep reinforcement learning to discover adaptive, personalized multi-cytokine therapy for sepsis

“…For this investigation, we use a previously developed agent-based simulation model of systemic inflammation: the Innate Immune Response agent-based model (IIRABM) [10] as a surrogate system for clinical sepsis. Simulation experiments with the IIRABM provide an explanation as to why single/limited cytokine perturbations at a single, or small number of, time points is unlikely to significantly improve the mortality rate of sepsis [11]. This result is consistent with the failures of previous static and non-personalized attempts at treating sepsis.…”

“…Consequently, in contrast to our approach, only previously attempted therapeutics can be considered. Also similar in goal but different in methodology is a recent approach using genetic algorithms to search for a non-adaptive control strategy using the same IIRABM as this work [11].…”

“…Once intervention stops, to ensure that it provides a non-transient, lasting effect, a patient must proceed without intervention for at least 12 hr before the health condition is checked; this represents the fact that any putative intervention must cease. The death condition occurs when total damage exceeds 80% (regardless of infection level) [11,20]. Clinically, this threshold represents the ability of current medical technologies to keep patients alive (i.e.…”

“…The IIRABM was implemented in C++ as in [11,20] to maximize performance. We exposed it to Python using the Boost C++ libraries [40] and recast it as an OpenAI Gym environment [22].…”

“…This result is consistent with the failures of previous static and non-personalized attempts at treating sepsis. Genetic algorithms have been used to explore and characterize multi-cytokine control strategies for the simulation that guide it from a persistent, non-recovering inflammatory state (functionally equivalent to the clinical states of systemic inflammatory response syndrome (SIRS) or sepsis) to a state of health [11]. While the calculated results show some generalizability, the authors recognized that more advanced strategies are needed to achieve the goal of adaptive personalized medicine in order to achieve lower simulated mortalities across a broader range of simulated patients.…”