We present the first application of an Artificial Neural Network trained through a Deep Reinforcement Learning agent to perform active flow control. It is shown that, in a 2D simulation of the Kármán vortex street at moderate Reynolds number (Re = 100), our Artificial Neural Network is able to learn an active control strategy from experimenting with the mass flow rates of two jets on the sides of a cylinder. By interacting with the unsteady wake, the Artificial Neural Network successfully stabilizes the vortex alley and reduces drag by about 8 %. This is performed while using small mass flow rates for the actuation, on the order of 0.5 % of the mass flow rate intersecting the cylinder cross section once a new pseudo-periodic shedding regime is found. This opens the way to a new class of methods for performing active flow control. arXiv:1808.07664v5 [physics.flu-dyn]

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300, and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and actively reduce the drag by ∼5.7%, 21.6%, 32.7%, and 38.7%, at Re = 100, 200, 300, and 400, respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone toward the development of practical applications of DRL to active flow control.

Deep Reinforcement Learning (DRL) has recently been proposed as a methodology to discover complex Active Flow Control (AFC) strategies [Rabault, J., Kuchta, M., Jensen, A., Réglade, U., & Cerardi, N. (2019): "Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control", Journal of Fluid Mechanics, 865, 281-302]. However, while promising results were obtained on a simple 2D benchmark flow at a moderate Reynolds number, considerable speedups will be required to investigate more challenging flow configurations. In the case of DRL trained with Computational Fluid Dynamics (CFD) data, it was found that the CFD part, rather than training the Artificial Neural Network, was the limiting factor for speed of execution. Therefore, speedups should be obtained through a combination of two approaches. The first one, which is well documented in the literature, is to parallelize the numerical simulation itself. The second one is to adapt the DRL algorithm for parallelization. Here, a simple strategy is to use several independent simulations running in parallel to collect experiences faster. In the present work, we discuss this solution for parallelization. We illustrate that perfect speedups can be obtained up to the batch size of the DRL agent, and slightly suboptimal scaling still takes place for an even larger number of simulations. This is, therefore, an important step towards enabling the study of more sophisticated Fluid Mechanics problems through DRL.

Sea ice is highly complex due to the inhomogeneity of the physical properties (e.g. temperature and salinity) as well as the permeability and mixture of water and a matrix of sea ice and/or sea ice crystals. Such complexity has proven itself to be difficult to parameterize in operational wave models. Instead, we assume that there exists a self-similarity scaling law which captures the first order properties. Using dimensional analysis, an equation for the kinematic viscosity is derived, which is proportional to the wave frequency and the ice thickness squared. In addition, the model allows for a two-layer structure where the oscillating pressure gradient due to wave propagation only exists in a fraction of the total ice thickness. These two assumptions lead to a spatial dissipation rate that is a function of ice thickness and wavenumber. The derived dissipation rate compares favourably with available field and laboratory observations.

Traditional programs based on feature engineering are under performing on a steadily increasing number of tasks compared with Artificial Neural Networks (ANNs), in particular for image analysis. Image analysis is widely used in Fluid Mechanics when performing Particle Image Velocimetry (PIV) and Particle Tracking Velocimetry (PTV), and therefore it is natural to test the ability of ANNs to perform such tasks. We report for the first time the use of Convolutional Neural Networks (CNNs) and Fully Connected Neural Networks (FCNNs) for performing end-to-end PIV. Realistic synthetic images are used for training the networks and several synthetic test cases are used to assess the quality of each network predictions and compare them with state-of-the-art PIV software. In addition, we present tests on real-world data that prove that ANNs can be used not only with synthetic images but also with more noisy, imperfect images obtained in a real experimental setup. While the ANNs we present have slightly higher Root Mean Square (RMS) error than state-ofthe-art cross-correlation methods, they perform better near edges and allow for higher spatial resolution than such methods. In addition, it is likely that one could with further work develop ANNs which perform better that the proof-of-concept we offer.