Deep reinforcement learning for the control of conjugate heat transfer

Abstract

This research gauges the ability of deep reinforcement learning (DRL) techniques to assist the control of conjugate heat transfer systems. It uses a novel, "degenerate" version of the proximal policy optimization (PPO) algorithm to train a neural network in optimizing said system only once per learning episode, and an in-house stabilized finite elements environment combining variational multiscale (VMS) modeling of the governing equations, immerse volume method, and multi-component anisotropic mesh adaptation to compute the numerical reward fed to the neural network. Several test cases of natural and forced convection are used as testbed for developing the methodology, that proves successful to alleviate the heat transfer enhancement related to the onset of convection in a two-dimensional, differentially heated square cavity, and to improve the homogeneity of temperature across the surface of two and three-dimensional hot workpieces under impingement cooling. Beyond adding value to the shallow literature on this subject, these findings establish the potential of single-step PPO for reliable black-box optimization of computational fluid dynamics (CFD) systems, and pave the way for future progress in the optimal control of conjugate heat transfer using this new class of methods.

Reinforcement learning of control strategies for reducing skin friction drag in a fully developed turbulent channel flow

“…Previous studies cover a variety of purposes, such as drag reduction (Koizumi, Tsutsumi & Shima 2018;Rabault & Kuhnle 2019;Fan et al 2020;Tang et al 2020;Tokarev, Palkin & Mullyadzhanov 2020;Xu et al 2020;Ghraieb et al 2021;Paris, Beneddine & Dandois 2021;Ren, Rabault & Tang 2021), control of heat transfer (Beintema et al 2020;Hachem et al 2021), optimization of microfluidics (Dressler et al 2018;Lee et al 2021), optimization of artificial swimmers (Novati et al 2018;Yan et al 2020;Zhu et al 2021) and shape optimization (Yan et al 2019;Li, Zhang & Chen 2021;Viquerat et al 2021;Qin et al 2021). In terms of drag reduction considered in the present study, considered control of a two-dimensional flow around a cylinder at a low Reynolds number.…”

PINN-CHK: Physics-informed neural network for high-fidelity prediction of early-age cement hydration kinetics

“…It has been utilized to tackle temperature-related concerns since the 1990s, starting with the use of artificial neural network (ANN) to learn about the convective heat transfer coefficient [16]. Subsequently, various neural network architectures, including convolutional neural network (CNN), have been employed to address heat transfer challenges [17][18][19][20][21][22]. However, these models often rely on supervised learning and require substantial training data, limiting their practical applications.…”

Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections

“…Recent years also have witnessed a blossoming of RL in fluid mechanics 26 . Some typical applications include the behavior adaption of swimmers 27,28 , active flow control for drag reduction [29][30][31][32] and conjugate heat transfer 33,34 , and aerodynamic shape optimization 35,36 .…”