Deep Echo State Q-Network (DEQN) and Its Application in Dynamic Spectrum Sharing for 5G and Beyond

Chang, Hao-Hsuan; Liu, Lingjia; Yi, Yang

doi:10.1109/tnnls.2020.3029711

Cited by 38 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al and some people say that digital technology innovation represented by big data and artificial intelligence has brought subversive changes and innovations to all walks of life. As a key stage in the empowerment of the digital economy, smart cities have once again become the focus of attention [4]. Cloud computing is shown in Figure 2.…”

Section: Literature Reviewmentioning

confidence: 99%

Analysis on the Application of Mobile Information System in the 5G Era in the Transformation of Ethnic PE Teaching in Colleges and Universities

Song

Peng

Zhan

et al. 2022

Scientific Programming

View full text Add to dashboard Cite

In order to solve the problem of the application of mobile information system in the transformation of ethnic physical education in colleges and universities in the 5G era, an analysis method based on the application of mobile information system in the transformation of ethnic physical education in colleges and universities in the 5G era is proposed. This method discusses the path of the transformation of ethnic physical education in colleges and universities through the analysis of the technological advantages in the 5G era. The experimental results show that: at present, under the influence of 5G technology, an important development path of ethnic physical education in colleges and universities is the modernization of education informatization. Through the construction and application of mobile information system, it can meet the increasing diversified sports needs of college students and promote the reform of college physical education. The mean value of competition video of film and television works related to classroom teaching content played before class is 4.576 standard deviation 0.673. Conclusion. On the basis of full use of 5G technology, teaching elements should be optimized, teaching resources should be enriched, and teaching mode should be reconstructed to promote the transformation of PHYSICAL education to digital information teaching.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Analysis on the Application of Mobile Information System in the 5G Era in the Transformation of Ethnic PE Teaching in Colleges and Universities

Song

Peng

Zhan

et al. 2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…This study uses deep echo state Q-network [23] in which the agent stores the observation o and the reservoir output x including the memory of the context of the observation. Fig.…”

Section: Deep Echo State Q-networkmentioning

confidence: 99%

“…Chang proposed deep echo state Q-network in which the system stores the output of the ESN in replay memory to avoid the issues of large computational cost and vanishing/exploding gradients problem [23]. The basic idea of this approach is to input the observation from the environment to the reservoir and store the observations and the reservoir output containing the spatio-temporally expanded memory of the observations into the replay memory.…”

Section: Introductionmentioning

confidence: 99%

Deep Q-network using reservoir computing with multi-layered readout

Matsuki¹

2022

Preprint

View full text Add to dashboard Cite

Recurrent neural network (RNN) based reinforcement learning (RL) is used for learning context-dependent tasks and has also attracted attention as a method with remarkable learning performance in recent research. However, RNN-based RL has some issues that the learning procedures tend to be more computationally expensive, and training with backpropagation through time (BPTT) is unstable because of vanishing/exploding gradients problem. An approach with replay memory introducing reservoir computing has been proposed, which trains an agent without BPTT and avoids these issues. The basic idea of this approach is that observations from the environment are input to the reservoir network, and both the observation and the reservoir output are stored in the memory. This paper shows that the performance of this method improves by using a multi-layered neural network for the readout layer, which regularly consists of a single linear layer. The experimental results show that using multi-layered readout improves the learning performance of four classical control tasks that require time-series processing.

show abstract

“…The outline of FL in MARL can be found in Figure 1. We select Signal-to-Interference-plus-Noise Ratio (SINR) as our quality measurement [9] which characterises all the factors to SUs network, such as background noise, the transmission power, the interference between simultaneously transmission pairs. In addition, we choose channel capacity as the reward function.…”

Section: Configuration Of Marl Enabled Flmentioning

confidence: 99%

“…FedProx modifies the FedAvg approach somewhat by allowing for partial work to be distributed among devices based on the underlying system restrictions and then using the proximal term to securely include the partial work. Furthermore, when applying the FL framework in RL, the widely used Q-learning or DQN (discrete actions) in solving DSS problems [9], [15] requires a solid theoretic proof when making convergence analysis based on FedAVG or FedProx. Therefore, policy gradient as an alternative approach is introduced in this article to update the shared model by averaging aggregated local policies.…”

Section: Asynchronous Fl Optimizationmentioning

confidence: 99%

Federated Dynamic Spectrum Access

Song,

Chang,

Zhou

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the growing volume of data traffic produced by the surge of Internet of Things (IoT) devices, the demand for radio spectrum resources is approaching their limitation defined by Federal Communications Commission (FCC). To this end, Dynamic Spectrum Access (DSA) is considered as a promising technology to handle this spectrum scarcity. However, standard DSA techniques often rely on analytical modeling wireless networks, making its application intractable in under-measured network environments. Therefore, utilizing neural networks to approximate the network dynamics is an alternative approach. In this article, we introduce a Federated Learning (FL) based framework for the task of DSA, where FL is a distributive machine learning framework that can reserve the privacy of network terminals under heterogeneous data distributions. We discuss the opportunities, challenges, and opening problems of this framework. To evaluate its feasibility, we implement a Multi-Agent Reinforcement Learning (MARL)-based FL as a realization associated with its initial evaluation results.

show abstract

Deep Echo State Q-Network (DEQN) and Its Application in Dynamic Spectrum Sharing for 5G and Beyond

Cited by 38 publications

References 16 publications

Analysis on the Application of Mobile Information System in the 5G Era in the Transformation of Ethnic PE Teaching in Colleges and Universities

Analysis on the Application of Mobile Information System in the 5G Era in the Transformation of Ethnic PE Teaching in Colleges and Universities

Deep Q-network using reservoir computing with multi-layered readout

Federated Dynamic Spectrum Access

Contact Info

Product

Resources

About