Deep Reinforcement Learning Based Dynamic Proportional-Integral (PI) Gain Auto-Tuning Method for a Robot Driver System

Abstract

To meet the growing trend of stringent fuel economy regulations, automakers around the world are designing modules such as engines, motors, transmissions and batteries to be as efficient as possible. In order to verify the effect of these designs on the overall fuel efficiency of the vehicle, the vehicle equipped with each module is placed on the chassis dynamometer, driven to follow the target vehicle speed, and actual fuel efficiency is measured. These tests are traditionally performed by human operators, but are now being replaced by robots (physical or software) to ensure the accuracy and reliability of test results. Although the conventionally proposed proportional integral (PI)-based controller has a simple structure and is easy to implement, it requires the process of finding the optimal gain whenever the test conditions such as vehicle or drive cycle change, which is difficult and time consuming. In this study, we propose a proportional integral controller gain adjustment algorithm using deep reinforcement learning. The reinforcement learning agent learns to dynamically modify the PI gain value of the acceleration/deceleration pedal to better follow the target vehicle in a simulation environment. The perturbation is used in each training episode to reduce the difference between the simulation and real testing environment. Upon completion of the training process, the trained agent performs an adjustment process that generates a reference gain table. We then use this reference gain table to perform a real test. The performance of the proposed system was evaluated using Hyundai Tucson HEV (NX4) on an AVL chassis dynamometer. We also compared the performance of our proposed algorithm to traditional fuzzy logic-based PI controllers. The obtained experimental results show that the proposed control system achieved a performance improvement of aounrd 46.8% compared to the conventional PI control system in terms of root mean square error.

A Comparative Study of Cutting-Edge Bi-Directional Power Converters and Intelligent Control Methodologies for Advanced Electric Mobility

“…This validated the practical efficacy of the proposed method. Cascade control and MRAC techniques offer a promising solution for enhancing and boosting converter system performance and stability [74][75][76][77]. In a DC-DC converter with a modified PI controller for electric vehicle (EV) battery charging.…”

PID Tuning Using Differential Evolution With Success-Based Particle Adaptations

“…Also, hybrids with Neural Networks and Fuzzy Logic for PID tuning have been proposed. For instance, [47] used a deep reinforcement learning (D3QN) for a robot driver system, [48] used artificial hydrocarbon network trained with backpropagation for a two-tank system, [93], [94] proposed the hybrid with neural networks for inverted pendulum, and [85] used a three-layer neural network optimized by DE. [95] used Fuzzy-PID in formation control and Takagi-Sugeno Fuzzy inference, [44] studied the hybrid between Fuzzy-PID, wolf colony algorithm and cuckoo search for smart grid, [45] tackled the Fuzzy-PID control with online optimization by DE for the semi-active suspension system, [46] used the hybrid between Fuzzy Logic, PID control and PSO-based parameter optimization for pH control in water and fertilizer, [63] used a hybrid between a swarm learning process (SLP) and Q-learning for weight updating SLP through a deterministic rule, [64] used a single variable for online robustness for a PID-control of a canonical tank system.…”

Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning

“…The controller utilizes the RBF network to adaptively adjust the PID controller's parameters based on the system's current state errors, which achieves stable control without system modeling in advance. Park adjusted proportion-integration (PI) parameters online through reinforcement learning so that the vehicle could follow the target more accurately [32]. According to the above research, the combination of reinforcement learning and the traditional PID controller improves the performance of control systems, which is useful for practical engineering applications.…”