Throughput and latency in the distributed Q-learning random access mMTC networks

Silva, Giovanni Maciel Ferreira; Abrão, Taufik

doi:10.1016/j.comnet.2022.108787

Cited by 9 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of Figures 4 and 5 were generated considering

\alpha

= 0.1, a typical value found in the literature, 19‐21 as it considers that each reward sent by the central node weights only 10% in each

Q

‐table update. However, when proposing the MPL‐QL algorithm in the NOMA scenario, it is necessary to assess whether the change in the value of

\alpha

impacts the choice of the most suitable

𝒫

.…”

Section: Numerical Resultsmentioning

confidence: 99%

“…The performance of the proposed MPL‐QL algorithm was compared with other methods available in the literature, specifically: (a) SA, where there is no feedback from the central node to the devices. The devices only send all their UL packets and the number of successes is obtained when there is no collision; (b) Independent QL; 19 (c) Collaborative QL; 19 and (d) Packet‐Based QL 21 …”

Section: Numerical Resultsmentioning

confidence: 99%

“…Packet‐based QL RA strategy favors devices that still have a lot of packets to transmit, sending them a greater reward w.r.t. devices that are already close to convergence 21 …”

Section: Numerical Resultsmentioning

confidence: 99%

“…In Reference 20, a nonorthogonal multiple access (NOMA)‐based QL algorithm is proposed in which the device can transmit at up to three different power levels to generate power diversity at the receiver while increase throughput. Recently, Reference 21 proposed a packet‐based QL scheme that can benefit devices that still have many packets to transmit, sending them a bigger reward.…”

Section: Introductionmentioning

confidence: 99%

“…The contribution of this work is 2‐fold: first, we propose a multipower‐level QL algorithm (MPL‐QL), evaluating the impact of increasing power levels on the throughput and latency, differing from what was done in Reference 20 where only three power levels are proposed, which does not exploit the full benefit of the power domain of NOMA. Second, we compare performance metrics, such as throughput, of the proposed MPL‐QL protocol with four well‐established RA protocols, the SA, the independent QL, 19 the collaborative QL, 19 and the packet‐based QL 21 .…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Multipower‐level Q‐learning algorithm for random access in nonorthogonal multiple access massive machine‐type communications systems

Silva

Abrão

2022

Trans Emerging Tel Tech

Self Cite

View full text Add to dashboard Cite

The massive machine-type communications (mMTC) service will be part of new services planned to integrate the beyond fifth generation of wireless communication. In mMTC, thousands of devices sporadically access available resource blocks on the network. In this scenario, the massive random access problem arises when two or more devices collide when selecting the same resource block.There are several techniques to deal with this problem. One of them deploys Q-learning (QL), in which devices store in their Q-table the rewards sent by the central node that indicate the quality of the transmission performed. The device learns which are the best resource blocks to select and transmit in order to avoid collisions. We propose a multipower-level QL (MPL-QL) algorithm that uses nonorthogonal multiple access (NOMA) transmit scheme to generate transmission power diversity and allow accommodate more than one device in the same time-slot as long as the signal-to-interference-plus-noise ratio (SINR) exceeds a threshold value. The numerical results reveal that the best performance-complexity trade-off is obtained by using a higher number of power levels, typically eight levels. The proposed MPL-QL can deliver better throughput and lower latency when compared to other recent QL-based algorithms found in the literature. INTRODUCTIONMachine-type wireless communication will be more widely used in applications such as internet of things (IoT), smart house, virtual reality, etc. 1,2 The goal of the fifth generation (5G) of wireless communications involves achieve ubiquitous communication in networks with ultra-dense devices allocation. [3][4][5] A data consumption of nearly 5 zettabytes per month is estimated across 17 billion devices. 6 In addition, due to the outbreak of the COVID-19 pandemic, there has been an remarkable increase in remote activities in work, health and education areas, which will be much more frequent in the post-pandemic environment. 7 Devices connected to the wireless network use different types of service. In the 5G of wireless communications systems, a clear division into three main use modes was defined: 8 enhanced mobile broadband (eMBB) for devices that require high data rates as an augmented reality user; ultra-reliable low-latency communications (URLLC) for applications that require 99.999% communication reliability such as remote surgery, while holding end-to-end latency below 1

show abstract

“…The results of Figures 4 and 5 were generated considering

\alpha

= 0.1, a typical value found in the literature, 19‐21 as it considers that each reward sent by the central node weights only 10% in each

Q

‐table update. However, when proposing the MPL‐QL algorithm in the NOMA scenario, it is necessary to assess whether the change in the value of

\alpha

impacts the choice of the most suitable

𝒫

.…”

Section: Numerical Resultsmentioning

confidence: 99%

Section: Numerical Resultsmentioning

confidence: 99%

“…Packet‐based QL RA strategy favors devices that still have a lot of packets to transmit, sending them a greater reward w.r.t. devices that are already close to convergence 21 …”

Section: Numerical Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%