Robust flow control and optimal sensor placement using deep reinforcement learning

Paris, Romain; Beneddine, Samir; Dandois, Julien

doi:10.1017/jfm.2020.1170

Cited by 95 publications

(53 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, effective estimation of the flow field does not guarantee effective control performance and vice versa. Similar conclusions have been drawn while investigating the sensor placement problem for a one-dimensional flow (Oehler & Illingworth 2018) and for the cylinder flow using deep reinforcement learning (Paris, Beneddine & Dandois 2021). Note that the OE problem considered in this study is merely a part of the whole optimal feedback control problem.…”

Section: Effect Of Domain Sizesupporting

confidence: 72%

Optimal sensor and actuator placement for feedback control of vortex shedding

2021

View full text Add to dashboard Cite

We consider linear feedback control of the two-dimensional flow past a cylinder at low Reynolds numbers, with a particular focus on the optimal placement of a single sensor and a single actuator. To accommodate the high dimensionality of the flow, we compute its leading resolvent forcing and response modes to enable the design of $\mathcal {H}_2$ -optimal estimators and controllers. We then investigate three control problems: (i) optimal estimation (OE) in which we measure the flow at a single location and estimate the entire flow; (ii) full-state information control (FIC) in which we measure the entire flow but actuate at only one location; and (iii) the overall feedback control problem in which a single sensor is available for measurement and a single actuator is available for control. We characterize the performance of these control arrangements over a range of sensor and actuator placements and discuss implications for effective feedback control when using a single sensor and a single actuator. The optimal sensor and actuator placements found for the OE and FIC problems are also compared with those found for the overall feedback control problem over a range of Reynolds numbers. This comparison reveals the key factors and conflicting trade-offs that limit feedback control performance.

show abstract

Section: Effect Of Domain Sizesupporting

confidence: 72%

Optimal sensor and actuator placement for feedback control of vortex shedding

2021

View full text Add to dashboard Cite

show abstract

“…Xu et al (2020) used RL-based control to stabilise the wake of the main cylinder by rotating two small cylinders located at two symmetrical positions downstream of the main cylinder. Paris, Beneddine & Dandois (2021) used a stochastic gated input layer in the RL agent to select an optimal subset from some initially placed probes. Ren, Rabault & Tang (2021) performed a follow-up study of and presented a successful application of the RL control in weakly turbulent conditions (Re = 1000) with a drag reduction of 30 %.…”

Section: Reinforcement Learning As a Flow Control Strategymentioning

confidence: 99%

“…We extend this investigation and further determine that the probes are better placed in the regions that are important in the sensitivity analysis. This heuristic approach may also be helpful to be combined with the optimal searching method proposed by Paris et al (2021). Furthermore, the definition of the reward function is important in RL-based control.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…In real-world applications, the control system always faces uncertain noise. Tang et al (2020) and Paris et al (2021) have shown that the RL-based control can be robust to the Reynolds number variation and systematic noise. There is another uncertainty which is the incoming disturbances penetrating into the inlet domain.…”

Section: Necessity Of a Persistent Oscillating Controlmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement-learning-based control of confined cylinder wakes with stability analyses

Zhang

2021

J. Fluid Mech.

View full text Add to dashboard Cite

This work studies the application of a reinforcement learning (RL)-based flow control strategy to the flow past a cylinder confined between two walls to suppress vortex shedding. The control action is blowing and suction of two synthetic jets on the cylinder. The theme of this study is to investigate how to use and embed physical information of the flow in the RL-based control. First, global linear stability and sensitivity analyses based on the time-mean flow and the steady flow (which is a solution to the Navier–Stokes equations) are conducted in a range of blockage ratios and Reynolds numbers. It is found that the most sensitive region in the wake extends itself when either parameter increases in the parameter range we investigated here. Then, we use these physical results to help design RL-based control policies. We find that the controlled wake converges to the unstable steady base flow, where the vortex shedding can be successfully suppressed. A persistent oscillating control seems necessary to maintain this unstable state. The RL algorithm is able to outperform a gradient-based optimisation method (optimised in a certain period of time) in the long run. Furthermore, when the flow stability information is embedded in the reward function to penalise the instability, the controlled flow may become more stable. Finally, according to the sensitivity analyses, the control is most efficient when the probes are placed in the most sensitive region. The control can be successful even when few probes are properly placed in this manner.

show abstract

“…RL represents a self-learning strategy introducing an agent who interacts with the environment through particular actions in order to get a maximum reward. Recent examples mainly consider the manipulation of the flow over a cylinder using multiple synthetic jets and numerical simulations at low Reynolds numbers delivering robust DRL-based control strategies [23][24][25][26][27][28] as well as other applications [29][30][31][32].…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning Control of Cylinder Flow Using Rotary Oscillations at Low Reynolds Number

2020

View full text Add to dashboard Cite

We apply deep reinforcement learning to active closed-loop control of a two-dimensional flow over a cylinder oscillating around its axis with a time-dependent angular velocity representing the only control parameter. Experimenting with the angular velocity, the neural network is able to devise a control strategy based on low frequency harmonic oscillations with some additional modulations to stabilize the Kármán vortex street at a low Reynolds number Re=100. We examine the convergence issue for two reward functions showing that later epoch number does not always guarantee a better result. The performance of the controller provide the drag reduction of 14% or 16% depending on the employed reward function. The additional efforts are very low as the maximum amplitude of the angular velocity is equal to 8% of the incoming flow in the first case while the latter reward function returns an impressive 0.8% rotation amplitude which is comparable with the state-of-the-art adjoint optimization results. A detailed comparison with a flow controlled by harmonic oscillations with fixed amplitude and frequency is presented, highlighting the benefits of a feedback loop.

show abstract

Robust flow control and optimal sensor placement using deep reinforcement learning

Abstract: Abstract

Cited by 95 publications

References 63 publications

Optimal sensor and actuator placement for feedback control of vortex shedding

Optimal sensor and actuator placement for feedback control of vortex shedding

Reinforcement-learning-based control of confined cylinder wakes with stability analyses

Deep Reinforcement Learning Control of Cylinder Flow Using Rotary Oscillations at Low Reynolds Number

Contact Info

Product

Resources

About