Autonomous Power Management With Double-<i>Q</i>Reinforcement Learning Method

Huang, Hui; Lin, Man; Yang, Laurence T.; Zhang, Qingchen

doi:10.1109/tii.2019.2953932

Cited by 39 publications

(22 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the training process, sub-tasks are trained by a double Qlearning separately. A double Q-learning is used to tackle the problem of traditional over-estimation in reinforcement learning systems involving the maximization operation and it is often suggested that this algorithm outperforms the traditional RL methods [33]. For each sub-task using a double Q-learning, an alternative double estimator is used to calculate the estimate for the maximum value function and a limit double Q-learning can converge to the optimal policy.…”

Section: The Training Methods For Sub-tasks Using the Double Q-lementioning

confidence: 99%

An Experience Aggregative Reinforcement Learning With Multi-Attribute Decision-Making for Obstacle Avoidance of Wheeled Mobile Robot

Ning

Meng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

A variety of reinforcement learning (RL) methods are developed to achieve the motion control for the robotic systems, which has been a hot issue. However, the performance of the conventional RL methods often encounters a bottleneck, because the robots have difficulty in choosing an appropriate action in the control task due to the exploration-exploitation dilemma. To address this problem and improve the learning performance, this work introduces an experience aggregative reinforcement learning method with a Multi-Attribute Decision-Making (MADM) to achieve the real-time obstacle avoidance of wheeled mobile robot (WMR). The proposed method employs an experience aggregation method to cluster experiential samples and it can achieve more effective experience storage. Moreover, to achieve the effective action selection using the prior experience, an action selection policy based on a Multi-Attribute Decision-Making is proposed. Inspired by the hierarchical decision-making, this work decomposes the original obstacle avoidance task into two sub-tasks using a divide-and-conquer approach. Each sub-task is trained individually by a double Q-learning using a simple reward function. Each sub-task learns an action policy, which enables the sub-task to selects an appropriate action to achieve a single goal. The standardized rewards of sub-tasks are calculated when fusing these sub-tasks to eliminate differences in rewards for sub-tasks. Then, the proposed method integrates the prior experience of three trained sub-tasks via an action policy based on a MADM to complete the source task. Simulation results show that the proposed method outperforms competitors.

show abstract

Section: The Training Methods For Sub-tasks Using the Double Q-lementioning

confidence: 99%

An Experience Aggregative Reinforcement Learning With Multi-Attribute Decision-Making for Obstacle Avoidance of Wheeled Mobile Robot

Ning

Meng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…With the continuous scaling and growing power density, power management has been a concerning topic for microprocessors to achieve good energy efficiency [1][2][3]. This is especially critical for battery-powered mobile devices, which are featured with frequent power state transition and management for varying workloads [4,5].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, deep learning is found as another effective alternative to efficiently model very complex physical details. Thus, several researchers have proposed the deployment of different deep learning neural networks to enable DVFS [1,3]. However, how to efficiently incorporate such models into an actual system, while keeping flexibility of switching between different targets, remains unclear.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Cao

Shen

Zhang

et al. 2020

Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD

View full text Add to dashboard Cite

At the era of Artificial Intelligence and Internet of Things (AIoT), battery-powered mobile devices are required to perform more sophisticated tasks featured with fast varying workloads and constrained power supply, demanding more efficient run-time power management. In this paper, we propose a deep reinforcement learning framework for dynamic power and thermal co-management. We build several machine learning models that incorporate the physical details for an ARM Cortex-A72, with on average 3% and 1% error for power and temperature predictions, respectively. We then build an efficient deep reinforcement learning control incorporating the machine learning models and facilitating the run-time dynamic voltage and frequency scaling (DVFS) strategy selection based on the predicted power, workloads and temperature. We evaluate our proposed framework, and compare the performance with existing management methods. The results suggest that our proposed framework can achieve 6.8% performance improvement compared with other alternatives. CCS CONCEPTS • Computing methodologies → Modeling and simulation; • Hardware → Chip-level power issues; • Computer systems organization → Embedded systems.

show abstract

“…Energy efficiency is the foremost challenge while sustaining the performance of the concurrently running applications in mobile edge computing environment. The first article entitled "Autonomous power management with double-Q reinforcement learning method" by Huang et al [1] proposes a double-Q power management approach to overcome the static operation frequency scaling policies of traditional dynamic voltage and frequency scaling using learning for extended energy sustainability. The experimental results depict that the random selection and updation of one of the two Q-tables for storage and measurement, respectively, reduces overestimation, thereby saving energy in comparison to the on-demand, conservative, and Q learning-based methods.…”

mentioning

confidence: 99%

Guest Editorial: Special Section on Intelligent Informatics for Edge of Things in Smart Industrial Ecosystem

Zomaya

Kumar

Rodrigues

et al. 2020

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Autonomous Power Management With Double-QReinforcement Learning Method

Cited by 39 publications

References 22 publications

An Experience Aggregative Reinforcement Learning With Multi-Attribute Decision-Making for Obstacle Avoidance of Wheeled Mobile Robot

An Experience Aggregative Reinforcement Learning With Multi-Attribute Decision-Making for Obstacle Avoidance of Wheeled Mobile Robot

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Guest Editorial: Special Section on Intelligent Informatics for Edge of Things in Smart Industrial Ecosystem

Contact Info

Product

Resources

About