Delay-Optimal Random Access for Large-Scale Energy Harvesting Networks

Wang, Dezhi; Wang, Wei; Zhang, Zhaoyang; Huang, Aiping

doi:10.1109/icc.2018.8422272

Cited by 6 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the policies learned using the proposed mean-field MARL approach achieve throughput close to centralized policies. In contrast to earlier work [22], [23], our algorithm is provably convergent and does not require any knowledge about the statistics of the EH process and of the wireless channels. In order to learn the optimal power control policy, each node only needs to know the state of its own channel and battery.…”

Section: Introductionmentioning

confidence: 96%

“…However, the proposed method is not guaranteed to converge, since each individual node experiences an inherently non-stationary environment [26]. In [23], a distributed solution is developed to minimize the communication delay in EHbased large networks, assuming the information about the statistics of the EH process and of the wireless channel are known. Interestingly, the interactions among the devices are modeled as a mean-field game (MFG), a framework specifically conceived to analyze the evolution of systems composed of a very large number of distributed decision-makers [27]- [29].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach

Sharma

Zappone

Assaad

et al. 2019

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power control problem as a discrete-time mean-field game (MFG), and analytically show that the MFG has a unique stationary solution. Next, we leverage the fictitious play property of the mean-field games, and the deep reinforcement learning technique to learn the stationary solution of the game, in a completely distributed fashion. We analytically show that the proposed procedure converges to the unique stationary solution of the MFG. This, in turn, ensures that the optimal policies can be learned in a completely distributed fashion. In order to benchmark the performance of the distributed policies, we also develop a deep neural network (DNN) based centralized as well as distributed online power control schemes. Our simulation results show the efficacy of the proposed power control policies. In particular, the DNN based centralized power control policies provide a very good performance for large EH networks for which the design of optimal policies is intractable using the conventional methods such as Markov decision processes. Further, performance of both the distributed policies is close to the throughput achieved by the centralized policies.

show abstract

Section: Introductionmentioning

confidence: 96%