Deep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge Computing

Zhou, Huan; Jiang, Kai; Liu, Xuxun; Li, Xiuhua; Leung, Victor C. M.

doi:10.1109/jiot.2021.3091142

Cited by 195 publications

(67 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with outdoor air quality, indoor air quality (IAQ) is more significant because people spend nearly 80% of their time staying in indoor environments (homes, schools, offices, and so on) [3,4]. Therefore, there is an urgent need to conduct effective IAQ monitoring and control for avoiding respiratory illness and protecting people's health [5,6].…”

Section: Introductionmentioning

confidence: 99%

Efficient Monitoring and Adaptive Control of Indoor Air Quality Based on IoT Technology and Fuzzy Inference

Zhao

Zhou

Chen

et al. 2022

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

In recent years, more and more occupants have suffered from respiratory illness due to poor indoor air quality (IAQ). In order to address this issue, this paper presents a method to achieve efficient monitoring and adaptive control of IAQ. Firstly, an indoor air quality monitoring and control system (IAQMCS) is developed using IoT technology. Then, based on fuzzy inference, a novel fuzzy air quality index (FAQI) model is proposed to effectively assess IAQ. Furthermore, a simple adaptive control mechanism, called SACM, is designed to automatically control the IAQMCS according to a real-time FAQI value. Finally, extensive experiments are performed by comparing with regular control (time-based control), which show that our proposed method effectively measures various air parameters (CO2, VOC, HCHO, PM2.5, PM10, etc.) and has good performance in terms of evaluation accuracy, average FAQI value, and overall IAQ.

show abstract

Section: Introductionmentioning

confidence: 99%

Efficient Monitoring and Adaptive Control of Indoor Air Quality Based on IoT Technology and Fuzzy Inference

Zhao

Zhou

Chen

et al. 2022

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, hybrid channel access with nonorthogonal multiple access (NOMA) and orthogonal multiple access (OMA) users was considered in [128] to achieve the optimal policy of partial offloading decisions and channel resource allocation using actor-critic DQN. In the time-varying MEC system with multiple users accessing an individual MEC server, to minimize the total energy consumption, the offloading strategy was proposed in [129] by considering the heterogeneous resource requirements and delay constraints in communication and computation. In this regard, Q-learning and its low-complexity version, i.e., double DQN, are adopted to achieve the optimal strategy.…”

Section: ) Dynamic Spectrum Accessmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei

Guo

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep reinforcement learning (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. INDEX TERMS Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL).

show abstract

“…The Q-learning, DQN, and Dueling DQN are all valuebased deep reinforcement models that can formulate the dynamic offloading problem as a Markov decision process, using DQN to approximate the objective value function to obtain an optimized offloading policy, even for multi-user mobile edge computing networks, DQN-based algorithms have advantages over traditional algorithms [30][31][32]. DQN may suffer from overestimation due to choosing the maximum action value each time in order to avoid this situation, Double DQN are generated, in [33], delay constraints and uncertain resource requirements of heterogeneous computing tasks, and in order to avoid dimensional disasters and overestimation using DDQN-based algorithms, which eventually proved the effectiveness of the algorithm. It is also possible to change the network structure of DQN to produce Dueling DQN, [34] firstly preprocesses the data and then uses Dueling DQN for optimization of the objective function, its convergence speed is faster than DQN.…”

Section: Related Workmentioning

confidence: 99%

A Vehicle-Assisted Computation Offloading Algorithm Based on Proximal Policy Optimization in Vehicle Edge Networks

Chen

Zeng

et al. 2022

Mobile Netw Appl

View full text Add to dashboard Cite

With the continuous development of the Internet of Vehicles(IoV), Vehicle Edge Computing(VEC) has become a key technology for computational resource scheduling, but more and more smart devices are connected to the internet, which makes it difficult for traditional Vehicle Edge Networks(VEN) to deal with tasks in time. In this paper, in order to cope with the challenges of the large number of devices accessing the internet, we propose a vehicle-assisted computation offloading algorithm based on proximal policy optimization(VCOPPO) for User Equipment(UE) tasks, and it combines dynamic parked vehicles incentives mechanism and computational resource allocation strategy by using road vehicles and parked vehicles as edge servers. Firstly, a non-convex optimization problem combining VEN utility and task processing delay is formulated, subject to the constraints of the residual energy and the transmission rate of the task. Secondly, the proposed VCOPPO is used to solve the formulated non-convex optimization problem, and we use stochastic policy to obtain the optimal computation offloading decisions and resource allocation schemes. Finally, the experimental results have shown that the proposed VCOPPO has an excellent performance in network reward and task processing delay respectively, and it can effectively schedule and allocate computational resources. Compared with using Dueling Deep Q Network(Dueling DQN), Deep Q Network(DQN) and Q-learning methods, the proposed VCOPPO improves the network reward by 31%, 18% and 91%, reduces the delay in task processing by 78%, 63% and 74%, respectively.

show abstract

Deep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge Computing

Cited by 195 publications

References 42 publications

Efficient Monitoring and Adaptive Control of Indoor Air Quality Based on IoT Technology and Fuzzy Inference

Efficient Monitoring and Adaptive Control of Indoor Air Quality Based on IoT Technology and Fuzzy Inference

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

A Vehicle-Assisted Computation Offloading Algorithm Based on Proximal Policy Optimization in Vehicle Edge Networks

Contact Info

Product

Resources

About