Learning the Aerodynamic Design of Supercritical Airfoils Through Deep Reinforcement Learning

Abstract

The aerodynamic design of modern civil aircraft requires a true sense of intelligence since it requires a good understanding of transonic aerodynamics and sufficient experience.Reinforcement learning is an artificial general intelligence that can learn sophisticated skills by trial-and-error, rather than simply extracting features or making predictions from data.The present paper utilizes a deep reinforcement learning algorithm to learn the policy for reducing the aerodynamic drag of supercritical airfoils. The policy is designed to take actions based on features of the wall Mach number distribution so that the learned policy can be more general. The initial policy for reinforcement learning is pretrained through imitation learning, and the result is compared with randomly generated initial policies. The policy is then trained in environments based on surrogate models, of which the mean drag reduction of 200 airfoils can be effectively improved by reinforcement learning. The policy is also tested by multiple airfoils in different flow conditions using computational fluid dynamics calculations. The results show that the policy is effective in both the training condition and other similar conditions, and the policy can be applied repeatedly to achieve greater drag reduction.

Machine Learning in Aerodynamic Shape Optimization

“…DRL can mimic human intuition to solve ASO problems. Li et al [244] applied DRL to learn an optimal policy to reduce the aerodynamic drag of supercritical airfoils.…”

“…To generalize the optimizer trained by RL, Li et al [244] used physical features such as the shock wave location and the suction peak Mach number as the state variables to learn the drag minimization policy for supercritical airfoil design. (Wall Mach number is defined as the equivalent Mach number calculated based on an isentropic relation using the local pressure coefficient and the free-stream Mach number.)…”

“…One drawback of RL-based optimization is that it does not scale well with the number of design variables, and the overall computational cost may turn out to be similar to performing evolutionary optimization [481]. Computational cost reductions have been achieved by using low-fidelity models [477,479] and surrogate models [244,480]. These approaches trained an initial policy using lowcost models and then used it as a starting point in training the RL-based optimizer with high-fidelity models.…”

“…Training an RL policy requires a clear definition of the objective and constraint functions to evaluate the reward. Current applications either explicitly treated the constraint as a penalty in the reward function [477,478] or implicitly imposed it into the aerodynamic analysis (for example, the lift constraint [244]). Thus, the corresponding RL policy cannot be directly applied to ASO design tasks with different constraints.…”

Multi-fidelity reinforcement learning framework for shape optimization

“…al. [40] carried a similar study for the design of super-critical airfoils using DRL. A lowfidelity surrogate model was constructed using sparse high-fidelity evaluations, and was used to learn the initial design policy, i.e.…”