Multiple Channel Access using Deep Reinforcement Learning for Congested Vehicular Networks

Choe, Chungjae; Choi, Junsung; Ahn, Jangyong; Park, Dongryul; Ahn, Seungyoung

doi:10.1109/vtc2020-spring48590.2020.9128853

Cited by 14 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [67], Choe et al proposed a self-adaptive MAC layer algorithm employing DQN with a novel contention information-based state representation to improve the performance of the V2V safety packet broadcast for infrastructure-less congested VANET. They evaluated the algorithm with two criterions: Packet Delivery Ratio (PDR) and end-to-end delay.…”

Section: Collision Managementmentioning

confidence: 99%

“…In [71], Q-Learning is used to control the contention window through a hybrid back-off that combines EIED and Linear Increase Linear Decrease (LILD [73]) back-off as vehicles in the network become agents. However, comparing to [67], these works assume single-channel operation of the control channel and did not consider the multi-channel operation of DSRC standard.…”

Section: Collision Managementmentioning

confidence: 99%

“…References Role Networking Dynamic spectrum access [56,57,58,59,60] Channel assignment and power allocation for V2V links [61,62] Channel assignment and power allocation for V2V links taking into account vehicle mobility [63,64,65,66] Transmission mode selection, channel assignment and power allocation for V2V links Collision management [68,69,70] Contention Window control, EIED, single-channel operation [71] Contention window control, LIED and EIED, single-channel operation [67] Contention window control, LIED and EIED, multi-channel operation Joint user association and beamforming [74] User association for load balancing with one access point [75,76] User association for load balancing with multiple access points [77] Beam tracking [79] Hand-off management Computing and caching Computation and data offloading [81,82,83,84,87] Find the optimal offloading strategy [80,85,86] Find the optimal offloading strategy and MEC server assignment Caching [88,89,90] Optimize content caching [91,92,93,94,95] Optimize caching, networking, and computing Energy Roadside units scheduling [96,100] Adaptively change Time Division Duplex (TDD) configuration…”

Section: Issuesmentioning

confidence: 99%

See 2 more Smart Citations

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Mekrache

Bradai

Moulay

et al. 2022

Vehicular Communications

View full text Add to dashboard Cite

Employing machine learning into 6G vehicular networks to support vehicular application services is being widely studied and a hot topic for the latest research works in the literature. This article provides a comprehensive review of research works that integrated reinforcement and deep reinforcement learning algorithms for vehicular networks management with an emphasis on vehicular telecommunications issues. Vehicular networks have become an important research area due to their specific features and applications such as standardization, efficient traffic management, road safety, and infotainment. In such networks, network entities need to make decisions to maximize network performance under uncertainty. To achieve this goal, Reinforcement Learning (RL) can effectively solve decision-making problems. However, the state and action spaces are massive and complex in large-scale wireless networks. Hence, RL may not be able to find the best strategy in a reasonable time. Therefore, Deep Reinforcement Learning (DRL) has been developed to combine RL with Deep Learning (DL) to overcome this issue. In this survey, we first present vehicular networks and give a brief overview of RL and DRL concepts. Then we review RL and especially DRL approaches to address emerging issues in 6G vehicular networks. We finally discuss and highlight some unresolved challenges for further study.

show abstract

Section: Collision Managementmentioning

confidence: 99%

Section: Collision Managementmentioning

confidence: 99%

Section: Issuesmentioning

confidence: 99%

See 1 more Smart Citation

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Mekrache

Bradai

Moulay

et al. 2022

Vehicular Communications

View full text Add to dashboard Cite

show abstract

“…Um mecanismo para ajuste dinâmico de tempos de backoff no protocolo MAC CSMA/CA em redes IEEE802.11 com aprendizado por reforçoé apresentado em [Amuru et al 2015]. Em [Choe et al 2020] propõe-se um algoritmo adaptativo para um protocolo MAC, em que se usa Deep-Q Network (DQN) para melhorar o desempenho de broadcast em redes veiculares. Em [Edalat and Obraczka 2019] usa-se aprendizado de máquina para ajustar a janela de disputa (CW -Contention Window) do protocolo MAC CSMA/CA em redes IEEE 802.11, obtendo maiores vazões e menores latências.…”

Section: Figura 1 Dinâmica Entre Agente E Ambienteunclassified

Um controlador para um protocolo MAC híbrido em redes sem-fio baseado em aprendizado por reforço

Sousa¹,

Sobral²

2020

Anais Da XVIII Escola Regional De Redes De Computadores (ERRC 2020)

View full text Add to dashboard Cite

Neste trabalho, propõe-se um mecanismo baseado em aprendizado por reforço para escolher o modo de operação de um protocolo MAC híbrido, dependendo da carga de tráfego existente na rede sem-fio. Protocolos como esse combinam MAC do tipo CSMA/CA e TDMA, buscando proporcionar maiores vazões (throughput) e menores latências, e têm sido usados em redes de acesso sem-fio com tecnologia IEEE 802.11. Este trabalho apresenta um modelo para um controlador para esse tipo de MAC, o qual usa experiência para aprender a escolher o melhor modo de operação dependendo da percepção sobre o estado da rede.

show abstract

“…In addition, the RL and DRL algorithms can now be used to study CW optimization because of the high computing capabilities in modern network devices. Some recent works [16], [21][22][23][24][25] discuss the CW optimization through the effectiveness of Q-learning and deep Q-learning network (DQN) algorithms, thus describing the problem of optimizing the CW value in mobile ad-hoc networks (MANETs), VANETs, and for both LTE-LAA and Wi-Fi networks.…”

Section: Introductionmentioning

confidence: 99%

Applying Deep Reinforcement Learning to Improve Throughput and Reduce Collision Rate in IEEE 802.11 Networks

2022

KSII TIIS

View full text Add to dashboard Cite

The effectiveness of Wi-Fi networks is greatly influenced by the optimization of contention window (CW) parameters. Unfortunately, the conventional approach employed by IEEE 802.11 wireless networks is not scalable enough to sustain consistent performance for the increasing number of stations. Yet, it is still the default when accessing channels for singleusers of 802.11 transmissions. Recently, there has been a spike in attempts to enhance network performance using a machine learning (ML) technique known as reinforcement learning (RL). Its advantage is interacting with the surrounding environment and making decisions based on its own experience. Deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments (such as continuous state spaces or actions spaces) and to get optimum rewards. As a result, we present a new approach of CW control mechanism, which is termed as contention window threshold (𝐶𝐶𝐶𝐶 𝑇𝑇ℎ𝑟𝑟𝑟𝑟𝑟𝑟ℎ𝑜𝑜𝑜𝑜𝑜𝑜 ). It uses the DRL principle to define the threshold value and learn optimal settings under various network scenarios. We demonstrate our proposed method, known as a smart exponential-threshold-linear backoff algorithm with a deep Q-learning network (SETL-DQN). The simulation results show that our proposed SETL-DQN algorithm can effectively improve the throughput and reduce the collision rates.

show abstract

Multiple Channel Access using Deep Reinforcement Learning for Congested Vehicular Networks

Cited by 14 publications

References 17 publications

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Um controlador para um protocolo MAC híbrido em redes sem-fio baseado em aprendizado por reforço

Applying Deep Reinforcement Learning to Improve Throughput and Reduce Collision Rate in IEEE 802.11 Networks

Contact Info

Product

Resources

About