Reliability evaluation of reinforcement learning methods for mechanical systems with increasing complexity

Abstract

Reinforcement learning (RL) is one of the emerging fields of artificial intelligence (AI) intended for designing agents that take actions in the physical environment. RL has many vital applications, including robotics and autonomous vehicles. The key characteristic of RL is its ability to learn from experience without requiring direct programming or supervision. To learn, an agent interacts with an environment by acting and observing the resulting states and rewards. In most practical applications, an environment is implemented as a virtual system due to cost, time, and safety concerns. Simultaneously, multibody system dynamics (MSD) is a framework for efficiently and systematically developing virtual systems of arbitrary complexity. MSD is commonly used to create virtual models of robots, vehicles, machinery, and humans. The features of RL and MSD make them perfect companions in building sophisticated, automated, and autonomous mechatronic systems. The research demonstrates the use of RL in controlling multibody systems. While AI methods are used to solve some of the most challenging tasks in engineering, their proper understanding and implementation are demanding. Therefore, we introduce and detail three commonly used RL algorithms to control the inverted N-pendulum on the cart. Single-, double-, and triple-pendulum configurations are investigated, showing the capability of RL methods to handle increasingly complex dynamical systems. We show 2D state space zones where the agent succeeds or fails the stabilization. Despite passing randomized tests during training, blind spots may occur where the agent’s policy fails. Results confirm that RL is a versatile, although complex, control engineering approach.

Stochastic calculus-guided reinforcement learning: A probabilistic framework for optimal decision-making

“…RL has also been applied in controlling multibody systems, including robots and vehicles, showing its abilities toward dynamical systems increasingly coupled [ 3 ]. RL has successfully used the swing and balance of an inverted pendulum on a hydromechanical cart to bring out data-driven control approaches [ 4 ].…”

Balancing of Robustness and Performance for Triple Inverted Pendulum Using μ-Synthesis and Gazelle Optimization

“…Type-1 fuzzy logic, IT2FLC, and adaptive neural fuzzy inference system (ANFIS) PID control strategies were applied to the triple inverted pendulum without testing system's robustness [14][15][16]. More recently published studies did not investigate the robustness of the proposed control system neither [17][18][19][20][21].…”