Strain design optimization using reinforcement learning

Abstract

Engineered microbial cells present a sustainable alternative to fossil-based synthesis of chemicals and fuels. Cellular synthesis routes are readily assembled and introduced into microbial strains using state-of-the-art synthetic biology tools. However, the optimization of the strains required to reach industrially feasible production levels is far less efficient. It typically relies on trial-and-error leading into high uncertainty in total duration and cost. New techniques that can cope with the complexity and limited mechanistic knowledge of the cellular regulation are called for guiding the strain optimization. In this paper, we put forward a multi-agent reinforcement learning (MARL) approach that learns from experiments to tune the metabolic enzyme levels so that the production is improved. Our method is model-free and does not assume prior knowledge of the microbe’s metabolic network or its regulation. The multi-agent approach is well-suited to make use of parallel experiments such as multi-well plates commonly used for screening microbial strains. We demonstrate the method’s capabilities using the genome-scale kinetic model of Escherichia coli, k-ecoli457, as a surrogate for an in vivo cell behaviour in cultivation experiments. We investigate the method’s performance relevant for practical applicability in strain engineering i.e. the speed of convergence towards the optimum response, noise tolerance, and the statistical stability of the solutions found. We further evaluate the proposed MARL approach in improving L-tryptophan production by yeast Saccharomyces cerevisiae, using publicly available experimental data on the performance of a combinatorial strain library. Overall, our results show that multi-agent reinforcement learning is a promising approach for guiding the strain optimization beyond mechanistic knowledge, with the goal of faster and more reliably obtaining industrially attractive production levels.

Simulated Design–Build–Test–Learn Cycles for Consistent Comparison of Machine Learning Methods in Metabolic Engineering

“…The challenge of effectively suggesting new strain designs for the next DBTL cycle using machine learning remains challenging. Several recommendation algorithms have been introduced in the literature, − among which the automated recommendation tool (ART) stands out as a notable example . While a rigorous benchmark of the different recommendation algorithms is outside the scope of this study, an example of how simulated DBTL cycles can be used for this purpose is reported in the Supporting Information (see Figure S8).…”

“…For metabolic flux optimization, this ranges from identifying the targets for engineering through unsupervised learning to predicting metabolite concentrations from proteomics data using supervised learning . Another potential application of machine learning is for recommending new strain designs for the next DBTL cycle by learning from a small set of experimentally probed input designs, which would allow (semi)-automated iterative metabolic engineering. − One example is the automated recommendation tool, which uses an ensemble of machine learning models to create a predictive distribution, from which it samples new designs for the next DBTL cycle given a user-specified exploration/exploitation parameter . While the automated recommendation tool was successfully applied to optimize the production of dodecanol and tryptophan, instances where the method did not perform well were also reported .…”

Machine learning in bioprocess development: from promise to practice

“…Guiding strain optimization, Sabzevari et al [62] applied a multi-agent reinforcement learning algorithm to both experimental data and data from a genome-scale kinetic model to tune metabolic enzyme levels. The algorithm outperforms another ML approach, namely Bayesian optimization on Gaussian processes (GPs) [Box 3], as well as a random search approach.…”

From genotype to phenotype: computational approaches for inferring microbial traits relevant to the food industry

“…However, this approach can be generally used for interpretation of metabolic modeling results, which is normally a complex and time-consuming task when performed manually. Indeed, this has been applied on a variety of levels, including applying random forest models on temporal microbial community model predictions for designing target communities (DiMucci et al 2018 ), using neural networks on E. coli GEM simulations of industrial conditions to predict cell factory performance (Oyetunde et al 2019 ), using regression analysis to understand antibiotic modes of action (Yang et al 2019 ), and using multiagent reinforcement learning to perform strain design (Sabzevari et al 2022 ). Overall, the approach shows promise in several areas relevant to the food industry, as a quick way of interpreting predicted fluxes from metabolic models.…”