Deep Reinforcement Learning using Cyclical Learning Rates

Gulde, Ralf; Tuscher, Marc; Csiszar, Akos; Riedel, Oliver; Verl, Alexander

doi:10.1109/ai4i49448.2020.00014

Cited by 6 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In deep reinforcement learning, the single-step average reward value of each episode is an important indicator to measure the training effect [ 30 , 31 , 32 , 33 ]. This paper counts the average single-step rewards of [ 22 ] and DCPER-DDPG algorithm in 6000 episodes.…”

Section: Results Analysismentioning

confidence: 99%

Lane Following Method Based on Improved DDPG Algorithm

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

In an autonomous vehicle, the lane following algorithm is an important component, which is a basic function of autonomous driving. However, the existing lane following system has a few shortcomings: first, the control method it adopts requires an accurate system model, and different vehicles have different parameters, which needs a lot of parameter calibration work. The second is that it may fail on road sections where the lateral acceleration requirements of vehicles are large, such as large curves. Third, its decision-making system is defined based on rules, which has disadvantages: it is difficult to formulate; human subjective factors cannot guarantee objectivity; coverage is difficult to guarantee. In recent years, the deep deterministic policy gradient (DDPG) algorithm has been widely used in the field of autonomous driving due to its strong nonlinear fitting ability and generalization performance. However, the DDPG algorithm has overestimated state action values and large cumulative errors, low training efficiency and other issues. Therefore, this paper improves the DDPG algorithm based on the double critic networks and priority experience replay mechanism. Then this paper proposes a lane following method based on this algorithm. Experiment shows that the algorithm can achieve excellent following results under various road conditions.

show abstract

Section: Results Analysismentioning

confidence: 99%

Lane Following Method Based on Improved DDPG Algorithm

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Thus, having an adaptive Discount Factor could lead to better learning performance and adapt itself when results (in our case, QoE value) are wrong. In the same way as the discount factor, with an adaptive learning rate such as a cyclical learning rate, which consists of varying the learning rate cyclically between two boundary values [20], there would be no need to find the best values and schedule for the global learning rates, so accuracy could be improved in fewer iterations.…”

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Learning techniques for dynamic task offloading in the 5G edge-cloud continuum

Nieto,

de la Iglesia,

Lopez-Novoa

et al. 2024

J Cloud Comp

View full text Add to dashboard Cite

The integration of new Internet of Things (IoT) applications and services heavily relies on task offloading to external devices due to the constrained computing and battery resources of IoT devices. Up to now, Cloud Computing (CC) paradigm has been a good approach for tasks where latency is not critical, but it is not useful when latency matters, so Multi-access Edge Computing (MEC) can be of use. In this work, we propose a distributed Deep Reinforcement Learning (DRL) tool to optimize the binary task offloading decision, this is, the independent decision of where to execute each computing task, depending on many factors. The optimization goal in this work is to maximize the Quality-of-Experience (QoE) when performing tasks, which is defined as a metric related to the battery level of the UE, but subject to satisfying tasks’ latency requirements. This distributed DRL approach, specifically an Actor-Critic (AC) algorithm running on each User Equipment (UE), is evaluated through the simulation of two distinct scenarios and outperforms other analyzed baselines in terms of QoE values and/or energy consumption in dynamic environments, also demonstrating that decisions need to be adapted to the environment’s evolution.

show abstract

“…In DRL, it is necessary to provide the agent with a set of optimal hyperparameters to improve the performance and effect of learning [24].…”

Section: The Selection Of Hyperparametermentioning

confidence: 99%

A Frequency and Voltage Coordinated Control Strategy of Island Microgrid including Electric Vehicles

et al. 2021

View full text Add to dashboard Cite

Frequency and voltage deviation are important standards for measuring energy indicators. It is important for microgrids to maintain the stability of voltage and frequency (VF). Aiming at the VF regulation of microgrid caused by wind disturbance and load fluctuation, a comprehensive VF control strategy for an islanded microgrid with electric vehicles (EVs) based on Deep Deterministic Policy Gradient (DDPG) is proposed in this paper. First of all, the SOC constraints of EVs are added to construct a cluster-EV charging model, by considering the randomness of users’ travel demand and charging behavior. In addition, a four-quadrant two-way charger capacity model is introduced to build a microgrid VF control model including load, micro gas turbine (MT), EVs, and their random power increment constraints. Secondly, according to the two control goals of microgrid frequency and voltage, the structure of DDPG controller is designed. Then, the definition of space, the design of global and local reward functions, and the selection of optimal hyperparameters are completed. Finally, different scenarios are set up in an islanded microgrid with EVs, and the simulation results are compared with traditional PI control and R(λ) control. The simulation results show that the proposed DDPG controller can quickly and efficiently suppress the VF fluctuations caused by wind disturbance and load fluctuations at the same time.

show abstract

Deep Reinforcement Learning using Cyclical Learning Rates

Cited by 6 publications

References 11 publications

Lane Following Method Based on Improved DDPG Algorithm

Lane Following Method Based on Improved DDPG Algorithm

Deep Reinforcement Learning techniques for dynamic task offloading in the 5G edge-cloud continuum

A Frequency and Voltage Coordinated Control Strategy of Island Microgrid including Electric Vehicles

Contact Info

Product

Resources

About