Deep Reinforcement Learning Control of Cylinder Flow Using Rotary Oscillations at Low Reynolds Number

Abstract

We apply deep reinforcement learning to active closed-loop control of a two-dimensional flow over a cylinder oscillating around its axis with a time-dependent angular velocity representing the only control parameter. Experimenting with the angular velocity, the neural network is able to devise a control strategy based on low frequency harmonic oscillations with some additional modulations to stabilize the Kármán vortex street at a low Reynolds number Re=100. We examine the convergence issue for two reward functions showing that later epoch number does not always guarantee a better result. The performance of the controller provide the drag reduction of 14% or 16% depending on the employed reward function. The additional efforts are very low as the maximum amplitude of the angular velocity is equal to 8% of the incoming flow in the first case while the latter reward function returns an impressive 0.8% rotation amplitude which is comparable with the state-of-the-art adjoint optimization results. A detailed comparison with a flow controlled by harmonic oscillations with fixed amplitude and frequency is presented, highlighting the benefits of a feedback loop.

Machine-Learning Methods for Complex Flows

“…The fifth article, written by Tokarev et al [35], is entitled "Deep reinforcement learning control of cylinder flow using rotary oscillations at low Reynolds number". The authors focus on reducing the drag in a two-dimensional cylinder, which oscillates around its axis with time-dependent angular velocity.…”

Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution

“…The main framework of the RL consists of an agent (e.g., a neural network in deep RL) that interacts with an environment to learn a policy that will maximize the cumulative reward over a long time horizon [315]. In recent years, the RL has been explored for fluid dynamics problems including animal locomotion [116,279,339], control of chaotic dynamics [41,59,337], drag reduction of bluff bodies [271,282,330], flow separation control [307], and turbulence closure modeling [242]. Along with a computer simulation environment, RL has been effectively applied for active flow control around bluff bodies in an experimental setup [98].…”

Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution

“…The main framework of the RL consists of an agent (for example, a neural network in deep RL) that interacts with an environment to learn a policy that will maximize the cumulative reward over a long time horizon [333] . In recent years, the RL has been explored for fluid dynamics problems including animal locomotion [334][335][336] , control of chaotic dynamics [337][338][339] , drag reduction of bluff bodies [340][341][342] , flow separation control [343] , and turbulence closure modeling [344] . Along with a computer simulation environment, RL has been effectively applied for active flow control around bluff bodies in an experimental setup [345] .…”