Framework for design optimization using deep reinforcement learning

Yonekura, Kazuo; Hattori, Hitoshi

doi:10.1007/s00158-019-02276-w

Cited by 49 publications

(23 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Advances in machine learning have promised a renaissance in understanding intrinsic features of many complex systems and gaining unprecedented attention not only in computer science but also in many other disciplines, such as fluid mechanics, [54][55][56] partial differential equations, 57,58 or design optimization. 59,60 Reinforcement learning is one of the main branches of machine learning and recently attracted a lot of interest following Google DeepMind, defeating top human professionals at the game of Go. 61 Unlike other machine learning methods such as supervised learning, which consists in learning to map an input to its corresponding output based on labeled examples provided by a knowledgeable external supervisor, or unsupervised learning, which is typically interested in finding transformations and clustering properties hidden in data, reinforcement learning is concerned with how to interact with an environment so as to maximize a numerical reward signal.…”

Section: Drl Control Algorithmmentioning

confidence: 99%

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Tang

Rabault

Kuhnle³

et al. 2020

Physics of Fluids

164

112

View full text Add to dashboard Cite

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300, and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and actively reduce the drag by ∼5.7%, 21.6%, 32.7%, and 38.7%, at Re = 100, 200, 300, and 400, respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone toward the development of practical applications of DRL to active flow control.

show abstract

Section: Drl Control Algorithmmentioning

confidence: 99%

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Tang

Rabault

Kuhnle³

et al. 2020

Physics of Fluids

164

112

View full text Add to dashboard Cite

show abstract

“…Among those, in Viquerat et al (2021), proximal policy optimization (PPO) was used in shape optimization, where indirect supervision from a generic reward signal is used as a non-linear optimizer. Deep Q-Network (DQN) was shown to be successful in optimizing the design of the angle of attack of airfoils (Yonekura and Hattori, 2019) and similarly, double-DQN with hindsight experience replay (HER) demonstrated good performance in design optimization of microfluidic devices for flow sculpting (Lee et al, 2019). However, utilizing machine learning for soft robot design has not been fully explored and our work complements the existing research in this field.…”

Section: Introductionmentioning

confidence: 93%

Design Optimization of a Pneumatic Soft Robotic Actuator Using Model-Based Optimization and Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

We present two frameworks for design optimization of a multi-chamber pneumatic-driven soft actuator to optimize its mechanical performance. The design goal is to achieve maximal horizontal motion of the top surface of the actuator with a minimum effect on its vertical motion. The parametric shape and layout of air chambers are optimized individually with the firefly algorithm and a deep reinforcement learning approach using both a model-based formulation and finite element analysis. The presented modeling approach extends the analytical formulations for tapered and thickened cantilever beams connected in a structure with virtual spring elements. The deep reinforcement learning-based approach is combined with both the model- and finite element-based environments to fully explore the design space and for comparison and cross-validation purposes. The two-chamber soft actuator was specifically designed to be integrated as a modular element into a soft robotic pad system used for pressure injury prevention, where local control of planar displacements can be advantageous to mitigate the risk of pressure injuries and blisters by minimizing shear forces at the skin-pad contact. A comparison of the results shows that designs achieved using the deep reinforcement based approach best decouples the horizontal and vertical motions, while producing the necessary displacement for the intended application. The results from optimizations were compared computationally and experimentally to the empirically obtained design in the existing literature to validate the optimized design and methodology.

show abstract

“…The majority of DRL modeling in engineering has thus far centered on more classical control problems which fit within a MDP, and stimulate agent learning [32]. A general framework for design optimization using DRL was proposed in [33] and was applied to the air foil angle of attack for a hypothetical stator with varying rotor designs. The work in [34] also shows the strong performance of DRL in the design task of fatigue resistant solutions for ship design compared to evolutionary methods.…”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning

Caputo

Cardin

2021

Journal of Mechanical Design

View full text Add to dashboard Cite

Engineering systems provide essential services to society e.g., power generation, transportation. Their performance, however, is directly affected by their ability to cope with uncertainty, especially given the realities of climate change and pandemics. Standard design methods often fail to recognize uncertainty in early conceptual activities, leading to rigid systems that are vulnerable to change. Real Options and Flexibility in Design are important paradigms to improve a system's ability to adapt and respond to unforeseen conditions. Existing approaches to analyze flexibility, however, do not leverage sufficiently recent developments in machine learning enabling deeper exploration of the computational design space. There is untapped potential for new solutions that are not readily accessible using existing methods. Here, a novel approach to analyze flexibility is proposed based on Deep Reinforcement Learning (DRL). It explores available datasets systematically and considers a wider range of adaptability strategies. The methodology is evaluated on an example waste-to-energy system. Low and high flexibility DRL models are compared against stochastically optimal inflexible and flexible solutions using decision rules. The results show highly dynamic solutions, with action space parametrized via artificial neural network. They show improved expected economic value up to 69% compared to previous solutions. Combining information from action space probability distributions along expert insights and risk tolerance helps make better decisions in real-world design and system operations. Out of sample testing shows that the policies are generalizable, but subject to tradeoffs between flexibility and inherent limitations of the learning process.

show abstract

Framework for design optimization using deep reinforcement learning

Cited by 49 publications

References 6 publications

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Design Optimization of a Pneumatic Soft Robotic Actuator Using Model-Based Optimization and Deep Reinforcement Learning

Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning

Contact Info

Product

Resources

About