AI Pontryagin or how artificial neural networks learn to control dynamical systems

Böttcher, Lucas; Antulov-Fantulin, Nino; Asikis, Thomas

doi:10.1038/s41467-021-27590-0

Cited by 41 publications

(27 citation statements)

References 40 publications

(70 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study opens up several avenues for future research. One worthwhile direction for future work is to combine our methods with control theory (Chehrazi et al 2019;Xia et al 2021;Asikis et al 2022;Böttcher et al 2022a) to study how many new antibiotics are needed on average in a certain time interval (e.g., 10-20 years) to create a stable supply of effective treatment options and to keep the emergence of antibiotic resistance at a minimum. Another important direction is to estimate the minimum size of the proposed funding scheme for different regions to make antibiotic R&D viable under current and/or modified market conditions.…”

Section: Discussionmentioning

confidence: 99%

A Refunding Scheme to Incentivize Narrow-Spectrum Antibiotic Development

Böttcher

Gersbach

2022

Bull Math Biol

Self Cite

View full text Add to dashboard Cite

The rapid rise of antibiotic resistance is a serious threat to global public health. The situation is exacerbated by the “antibiotics dilemma”: Developing narrow-spectrum antibiotics against resistant bacteria is most beneficial for society, but least attractive for companies, since their usage and sales volumes are more limited than for broad-spectrum drugs. After developing a general mathematical framework for the study of antibiotic resistance dynamics with an arbitrary number of antibiotics, we identify efficient treatment protocols. Then, we introduce a market-based refunding scheme that incentivizes pharmaceutical companies to develop new antibiotics against resistant bacteria and, in particular, narrow-spectrum antibiotics that target specific bacterial strains. We illustrate how such a refunding scheme can solve the antibiotics dilemma and cope with various sources of uncertainty that impede antibiotic R &D. Finally, connecting our refunding approach to the recently established Antimicrobial Resistance (AMR) Action Fund, we discuss how our proposed incentivization scheme could be financed.

show abstract

Section: Discussionmentioning

confidence: 99%

A Refunding Scheme to Incentivize Narrow-Spectrum Antibiotic Development

Böttcher

Gersbach

2022

Bull Math Biol

Self Cite

View full text Add to dashboard Cite

show abstract

“…To summarize, RL-based optimization is preferable in scenarios where the interactions between agent and environment cannot be fully characterized mathematically. If one wishes to optimize actions for controlling a known dynamical system, it is more efficient to employ direct neuralnetwork-based optimizations (Asikis et al 2020, Böttcher et al 2022.…”

Section: Neural-network-based Optimization and Controlmentioning

confidence: 99%

“…For deterministic, continuous-time dynamical systems with real-valued control signals, control frameworks that are based on neural ordinary differential equations (ODEs) were introduced by Asikis et al (2020). In a related work by Böttcher et al (2022), it has been shown that such neural ODE control frameworks are able to automatically learn control trajectories that resemble those of optimal control methods. Another work by Asikis (2021) extends the aforementioned framework with real-valued control signals to discrete action spaces.…”

Section: Neural-network-based Optimization and Controlmentioning

confidence: 99%

“…The optimization methods that we develop in this article build on previous works that use automatic differentiation (Linnainmaa 1976, Paszke et al 2017, Baydin et al 2018) and dynamicsinformed neural networks (Raissi et al 2019) to solve complex control and optimization problems (Holl et al 2020, Jin et al 2020, Asikis et al 2020, Böttcher et al 2022.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Solving Inventory Management Problems with Inventory-dynamics-informed Neural Networks

Böttcher¹,

Asikis²,

Fragkos³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A key challenge in inventory management is to identify policies that optimally replenish inventory from multiple suppliers. To solve such optimization problems, inventory managers need to decide what quantities to order from each supplier, given the on-hand inventory and outstanding orders, so that the expected backlogging, holding, and sourcing costs are jointly minimized. Inventory management problems have been studied extensively for over 60 years, and yet even basic dual sourcing problems, in which orders from an expensive supplier arrive faster than orders from a regular supplier, remain intractable in their general form.In this work, we approach dual sourcing from a neural-network-based optimization lens. By incorporating inventory dynamics into the design of neural networks, we are able to learn near-optimal policies of commonly used instances within a few minutes of CPU time on a regular personal computer. To demonstrate the versatility of inventory-dynamics-informed neural networks, we show that they are able to control inventory dynamics with empirical demand distributions that are challenging to tackle effectively using alternative, state-of-the-art approaches.

show abstract

“…In this work, we represent time-dependent control signals by artificial neural networks (ANNs) [29]. To parameterize and learn control functions, we use neural ordinary differential equations (neural ODEs) [30][31][32][33][34]. A schematic of the application of neural ODE control (NODEC) to a networked dynamical system is shown in figure 1.…”

Section: Introductionmentioning

confidence: 99%

Near-optimal control of dynamical systems with neural ordinary differential equations

Böttcher

Asikis

2022

Mach. Learn.: Sci. Technol.

Self Cite

View full text Add to dashboard Cite

Optimal control problems naturally arise in many scientific applications where one wishes to steer a dynamical system from an initial state $\mathbf{x}_0$ to a desired target state $\mathbf{x}^*$ in finite time $T$. Recent advances in deep learning and neural network\textendash based optimization have contributed to the development of numerical methods that can help solve control problems involving high-dimensional dynamical systems. In particular, the framework of neural ordinary differential equations (neural ODEs) provides an efficient means to iteratively approximate continuous-time control functions associated with analytically intractable and computationally demanding control tasks. Although neural ODE controllers have shown great potential in solving complex control problems, the understanding of the effects of hyperparameters such as network structure and optimizers on learning performance is still very limited. Our work aims at addressing some of these knowledge gaps to conduct efficient hyperparameter optimization. To this end, we first analyze how truncated and non-truncated backpropagation through time affect both runtime performance and the ability of neural networks to learn optimal control functions. Using analytical and numerical methods, we then study the role of parameter initializations, optimizers, and neural-network architecture. Finally, we connect our results to the ability of neural ODE controllers to implicitly regularize control energy.

show abstract

AI Pontryagin or how artificial neural networks learn to control dynamical systems

Cited by 41 publications

References 40 publications

A Refunding Scheme to Incentivize Narrow-Spectrum Antibiotic Development

A Refunding Scheme to Incentivize Narrow-Spectrum Antibiotic Development

Solving Inventory Management Problems with Inventory-dynamics-informed Neural Networks

Near-optimal control of dynamical systems with neural ordinary differential equations

Contact Info

Product

Resources

About