Flow Control in Wings and Discovery of Novel Approaches via Deep Reinforcement Learning

Vinuesa, Ricardo; Lehmkuhl, O.; Lozano-Durán, Adrián; Rabault, Jean

doi:10.3390/fluids7020062

“…Furthermore, DRL has been applied to a number of other control tasks, ranging from simple onedimensional falling-fluid instabilities [40], convection problems [41], chaotic turbulent combustion systems [42] to a variety of engineering cases [43][44][45][46].…”

Section: Reinforcement Learning In Fluid Mechanicsmentioning

confidence: 99%

Deep reinforcement learning for turbulent drag reduction in channel flows

Guastoni

¹

,

Rabault

²

,

Schlatter

³

et al. 2023

Self Cite

View full text Add to dashboard Cite

We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a complex, turbulent physical system that has been a major topic of research for over two centuries, and remains, even today, the subject of many unanswered questions. The control is applied in the form of blowing and suction at the wall, while the observable state is configurable, allowing to choose different variables such as velocity and pressure, in different locations of the domain. Given the complex nonlinear nature of turbulent flows, the control strategies proposed so far in the literature are physically grounded, but too simple. DRL, by contrast, enables leveraging the high-dimensional data that can be sampled from flow simulations to design advanced control strategies. In an effort to establish a benchmark for testing data-driven control strategies, we compare opposition control, a state-of-the-art turbulence-control strategy from the literature, and a commonly used DRL algorithm, deep deterministic policy gradient. Our results show that DRL leads to 43% and 30% drag reduction in a minimal and a larger channel (at a friction Reynolds number of 180), respectively, outperforming the classical opposition control by around 20 and 10 percentage points, respectively.

show abstract

“…Furthermore, DRL has been applied to a number of other control tasks, ranging from simple one-dimensional falling-fluid instabilities [34], convection problems [35], chaotic turbulent combustion systems [36] to a variety of engineering cases [37][38][39][40].…”

Section: Reinforcement Learning In Fluid Mechanicsmentioning

confidence: 99%

Deep reinforcement learning for turbulent drag reduction in channel flows

Guastoni¹,

Rabault²,

Schlatter³

et al. 2023

Preprint

View full text Add to dashboard Cite

We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally-efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a complex, turbulent physical system that has been a major topic of research for over two centuries, and remains, even today, the subject of many unanswered questions. The control is applied in the form of blowing and suction at the wall, while the observable state is configurable, allowing to choose different variables such as velocity and pressure, in different locations of the domain. Given the complex nonlinear nature of turbulent flows, the control strategies proposed so far

show abstract

“…Bucci et al (2019) showcased the use of RL to control chaotic systems such as the one-dimensional (1-D) Kuramoto-Sivashinsky equation; Beintema et al (2020) used it to control heat transport in a two-dimensional (2-D) Rayleigh-Bénard system while Belus et al (2019) used RL to control the interface of unsteady liquid films. Ongoing efforts in the use of DRL for flow control are focused with increasing the complexity of the analysed test cases, either by increasing the Reynolds number in academic test cases (see Ren, Rabault & Tang 2021) or by considering realistic configurations (Vinuesa et al 2022).…”

Section: Introductionmentioning

confidence: 99%

Comparative analysis of machine learning methods for active flow control

Pino

¹

,

Schena

²

,

Rabault

³

et al. 2023

Self Cite

View full text Add to dashboard Cite

Machine learning frameworks such as genetic programming and reinforcement learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, benchmarking some of their most representative algorithms against global optimization techniques such as Bayesian optimization and Lipschitz global optimization. First, we review the general framework of the model-free control problem, bringing together all methods as black-box optimization problems. Then, we test the control algorithms on three test cases. These are (1) the stabilization of a nonlinear dynamical system featuring frequency cross-talk, (2) the wave cancellation from a Burgers’ flow and (3) the drag reduction in a cylinder wake flow. We present a comprehensive comparison to illustrate their differences in exploration versus exploitation and their balance between ‘model capacity’ in the control law definition versus ‘required complexity’. Indeed, we discovered that previous RL control attempts of controlling the cylinder wake were performing linear control and that the wide observation space was limiting their performances. We believe that such a comparison paves the way towards the hybridization of the various methods, and we offer some perspective on their future development in the literature of flow control problems.

show abstract

Flow Control in Wings and Discovery of Novel Approaches via Deep Reinforcement Learning

Cited by 50 publications

References 114 publications

Deep reinforcement learning for turbulent drag reduction in channel flows

Deep reinforcement learning for turbulent drag reduction in channel flows

Deep reinforcement learning for turbulent drag reduction in channel flows

Comparative analysis of machine learning methods for active flow control

Contact Info

Product

Resources

About