Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications

Qiu, Chengrun; Hu, Yang; Chen, Yan; Zeng, Bing

doi:10.1109/jiot.2019.2921159

Cited by 284 publications

(113 citation statements)

References 27 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…This method is a policy-based DRL algorithm, that is, in DDPG, a Action network can directly generate corresponding output actions based on the input state vector. A Critic network in DDPG will evaluate the actions generated by the Action network and continuously optimize the action selection strategy of the Action network [36]- [38].…”

Section: Future Workmentioning

confidence: 99%

Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

Zhao

Kong

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Vehicle fog computing (VFC) is proposed as a solution that can significantly reduce the task processing overload of base station during the peak time, where the vehicle as a fog node contributes idle computing resource for task processing. However, there are still many challenges in the deployment of VFC, such as the lack of specific incentives of resource contribution, high system complexity, and offloading collisions between vehicles when the vehicles are offloading tasks simultaneously. In this paper, we first propose a novel contract-based incentive mechanism that combines resource contribution and resource utilization. Based on that, we propose to use distributed deep reinforcement learning to allocate resources and reduce system complexity. Task offloading method based on the queuing model is also proposed to avoid decision collisions in multi-vehicles task offloading. Numerical experiment results demonstrate that our proposed scheme has achieved a significant improvement in task offloading and resource allocation performance. INDEX TERMS Vehicular fog computing, contract theory, deep reinforcement learning, resource allocation, task offloading.

show abstract

Section: Future Workmentioning

confidence: 99%

Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

Zhao

Kong

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The work in [14] considered joint optimization of traffic scheduling and power allocation and minimized the total on-grid power consumption of macro and small cells, while guaranteeing users' traffic requirement. Recently, some works have focused on solving the EH problems with reinforcement learning, where novel energy management was proposed in [15] based on reinforcement learning to maximize packet transmission rates while avoiding energy outage for wireless sensor networks, and energy management was investigated in [16] to maximize the net bit rate in EH wireless communications by using deep deterministic policy gradient.…”

Section: Related Workmentioning

confidence: 99%

“…The computational complexity of calculating the proposed MDP scheme mainly comes from (16). Note that the total number of system states is N 2 h N c N 2 b and the number of actions at each state is no more than 2 N b + 1.…”

Section: F Implementation Complexity and Overheadmentioning

confidence: 99%

“…By using (16), the summation over the variables q 1 and q 2 in (C.1) can be classified into five parts in terms of the ranges of q 1 and q 2 , given by where the values for the cases of q 1 = N b − k or q 2 = N b − l − 1 are equal to zero, and the inequality in the second part is due to Lemma 2. Hence, l (i , j , c , q 1 , q 2 ) is always nonpositive.…”

Section: Appendix C Proof Of Lemmamentioning

confidence: 99%

“…We adopt the induction method to prove this lemma. Without loss of generality, we fix the battery state k. For m = 1, the declaration is true because (1) (s) = 1−R a on (s) according to (12) and (16), where R a on (s) only depends on the channel state c for any (k, l) ∈ B + , and thereby (1) (s) keeps the same value along the direction of the battery state l, for any given i, j ∈ H, and c ∈ C. By assuming that the difference function (r) (s) is non-decreasing in the battery direction of l and using (17) and (18), the difference function for m = r +1, (r+1) (s), can be written as (D.1), shown at the bottom of the this page. Due to the non-decreasing property of (r) (s), the following two functions are also non-increasing in the battery direction of l: = R a off (s)−R a on (s)+λE s min{V (r)…”

Section: Appendix D Proof Of Theoremmentioning

confidence: 99%

See 2 more Smart Citations

On Stochastic Link and Energy Scheduling for Energy Harvesting Bidirectional Communications

Chang

Lin

2020

IEEE Access

View full text Add to dashboard Cite

In this paper, a solar-powered bidirectional communication system is studied for a pair of energy harvesting (EH) nodes that intend to communicate with each other over wireless fading channels. The conventional time-division duplex (TDD) transmission is revisited by proposing a stochastic resource scheduling scheme to minimize an average rate outage probability based on a Markov decision process (MDP) design framework. Different from the conventional TDD transmission, the proposed scheme can adjust the link direction and energy expenditure for data transmissions between the two EH nodes, in response to the dynamics of the solar EH, channel fading and battery storage conditions. A downstairs threshold structure is theoretically proved under a special optimal on-off policy, in which two-dimensional thresholds pinpoint the interplay between the transmission actions and the available energy in the batteries of the two nodes. Also the optimal on-off policy at asymptotically high signal-to-noise power ratios (SNRs) is revealed. The outage performance of the proposed stochastic resource scheduling scheme is validated by extensive computer simulations, and it shows that the proposed optimal MDP policy can achieve significant performance gains over the combinations of other compared schemes, including round-robin and battery state-oriented link scheduling schemes, and greedy and conservative energy scheduling schemes. INDEX TERMS Energy harvesting, solar-powered communications, bidirectional communications, time-division duplex, resource scheduling.

show abstract

Deep reinforcement learning‐based resource allocation in multi‐access edge computing

Khani,

Sadr,

Jamali

2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryNetwork architects and engineers face challenges in meeting the increasing complexity and low‐latency requirements of various services. To tackle these challenges, multi‐access edge computing (MEC) has emerged as a solution, bringing computation and storage resources closer to the network's edge. This proximity enables low‐latency data access, reduces network congestion, and improves quality of service. Effective resource allocation is crucial for leveraging MEC capabilities and overcoming limitations. However, traditional approaches lack intelligence and adaptability. This study explores the use of deep reinforcement learning (DRL) as a technique to enhance resource allocation in MEC. DRL has gained significant attention due to its ability to adapt to changing network conditions and handle complex and dynamic environments more effectively than traditional methods. The study presents the results of applying DRL for efficient and dynamic resource allocation in MEC Computing, optimizing allocation decisions based on real‐time environment and user demands. By providing an overview of the current research on resource allocation in MEC using DRL, including components, algorithms, and the performance metrics of various DRL‐based schemes, this review article demonstrates the superiority of DRL‐based resource allocation schemes over traditional methods in diverse MEC conditions. The findings highlight the potential of DRL‐based approaches in addressing challenges associated with resource allocation in MEC.

show abstract

Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications

Cited by 284 publications

References 27 publications

Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

On Stochastic Link and Energy Scheduling for Energy Harvesting Bidirectional Communications

Deep reinforcement learning‐based resource allocation in multi‐access edge computing

Contact Info

Product

Resources

About