A comparison between UCB and UCB-Tuned as selection policies in GGP

Francisco-Valencia, Iván; Marcial‐Romero, J. Raymundo; Valdovinos, R. M.

doi:10.3233/jifs-179052

Cited by 14 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LinUCB [ 20 ] extends the Auer’s UCB algorithm in [ 10 , 22 ] to the contextual concept. Its main clue is to figure out each arm’s probable reward by finding a linear relationship between the previous rewards of the arm and its current context vector as given in (10).…”

Section: Proposed Ea-cmab Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Hashima

Hatano

Kasban

et al. 2021

Sensors

View full text Add to dashboard Cite

The unique features of millimeter waves (mmWaves) motivate its leveraging to future, beyond-fifth-generation/sixth-generation (B5G/6G)-based device-to-device (D2D) communications. However, the neighborhood discovery and selection (NDS) problem still needs intelligent solutions due to the trade-off of investigating adjacent devices for the optimum device choice against the crucial beamform training (BT) overhead. In this paper, by making use of multiband (μW/mmWave) standard devices, the mmWave NDS problem is addressed using machine-learning-based contextual multi-armed bandit (CMAB) algorithms. This is done by leveraging the context information of Wi-Fi signal characteristics, i.e., received signal strength (RSS), mean, and variance, to further improve the NDS method. In this setup, the transmitting device acts as the player, the arms are the candidate mmWave D2D links between that device and its neighbors, while the reward is the average throughput. We examine the NDS’s primary trade-off and the impacts of the contextual information on the total performance. Furthermore, modified energy-aware linear upper confidence bound (EA-LinUCB) and contextual Thomson sampling (EA-CTS) algorithms are proposed to handle the problem through reflecting the nearby devices’ withstanding battery levels, which simulate real scenarios. Simulation results ensure the superior efficiency of the proposed algorithms over the single band (mmWave) energy-aware noncontextual MAB algorithms (EA-UCB and EA-TS) and traditional schemes regarding energy efficiency and average throughput with a reasonable convergence rate.

show abstract

Section: Proposed Ea-cmab Algorithmsmentioning

confidence: 99%

“…We examine the effect of Wi-Fi contextual information on the overall system performance by leveraging LinUCB [ 20 ] and CTS [ 21 ] algorithms and compare them with their noncontextual versions, i.e., UCB [ 22 ] and TS [ 23 ].…”

Section: Introductionmentioning

confidence: 99%

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Hashima

Hatano

Kasban

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The motivation behind using online learning comes from its ability to deal with both complex and dynamic environments effectively [26], without any prior information, where an agent learns to enhance its future actions based only on its past actions/observations. Towards this end, the gateway UAV selection problem is formulated as a budget-constrained multi-player multi-armed bandit (MAB) problem [27][28][29]. MAB is a particular type of online learning, where an agent wants to maximize its long-term rewards (minimize regrets) via utilizing its previous best arm selection or investigating new choices, known as the exploitation-exploration tradeoff [27][28][29].…”

Section: Introductionmentioning

confidence: 99%

“…Towards this end, the gateway UAV selection problem is formulated as a budget-constrained multi-player multi-armed bandit (MAB) problem [27][28][29]. MAB is a particular type of online learning, where an agent wants to maximize its long-term rewards (minimize regrets) via utilizing its previous best arm selection or investigating new choices, known as the exploitation-exploration tradeoff [27][28][29]. Since MAB techniques work online without any prior knowledge about the environment other than the player's observations while playing, they are considered as the most appropriate solutions for this deemed problem.…”

Section: Introductionmentioning

confidence: 99%

“…Only based on their previous successive observed rewards, access UAVs try to compromise the exploitation-exploration tradeoff, i.e., either exploiting their best-selected gateway UAVs so-far or exploring new ones. In this paper, three MAB algorithms, namely upper confidence bound (UCB) [29], Thompson sampling (TS) [30], and the exponential weight algorithm for exploration and exploitation (EXP3) [31], are modified to address such gateway UAV selection problem. Despite the adversarial setting of the problem and the selfish behavior of the access UAVs, the modified MAB algorithms learn to play actions that enhance the overall system performance, as demonstrated in [24,30] and further discussed in this paper.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Gateway Selection in Millimeter Wave UAV Wireless Networks Using Multi-Player Multi-Armed Bandit

Mohamed

Hashima

Aldosary

et al. 2020

Sensors

View full text Add to dashboard Cite

Recently, unmanned aerial vehicle (UAV)-based communications gained a lot of attention due to their numerous applications, especially in rescue services in post-disaster areas where the terrestrial network is wholly malfunctioned. Multiple access/gateway UAVs are distributed to fully cover the post-disaster area as flying base stations to provide communication coverage, collect valuable information, disseminate essential instructions, etc. The access UAVs after gathering/broadcasting the necessary information should select and fly towards one of the surrounding gateways for relaying their information. In this paper, the gateway UAV selection problem is addressed. The main aim is to maximize the long-term average data rates of the UAVs relays while minimizing the flights’ battery cost, where millimeter wave links, i.e., using 30~300 GHz band, employing antenna beamforming, are used for backhauling. A tool of machine learning (ML) is exploited to address the problem as a budget-constrained multi-player multi-armed bandit (MAB) problem. In this setup, access UAVs act as the players, and the arms are the gateway UAVs, while the rewards are the average data rates of the constructed relays constrained by the battery cost of the access UAV flights. In this decentralized setting, where information is neither prior available nor exchanged among UAVs, a selfish and concurrent multi-player MAB strategy is suggested. Towards this end, three battery-aware MAB (BA-MAB) algorithms, namely upper confidence bound (UCB), Thompson sampling (TS), and the exponential weight algorithm for exploration and exploitation (EXP3), are proposed to realize gateways selection efficiently. The proposed BA-MAB-based gateway UAV selection algorithms show superior performance over approaches based on near and random selections in terms of total system rate and energy efficiency.

show abstract

WiGig access point selection using non-contextual and contextual multi-armed bandit in indoor environment

Mohamed

2022

J Ambient Intell Human Comput

View full text Add to dashboard Cite

A comparison between UCB and UCB-Tuned as selection policies in GGP

Cited by 14 publications

References 12 publications

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Gateway Selection in Millimeter Wave UAV Wireless Networks Using Multi-Player Multi-Armed Bandit

WiGig access point selection using non-contextual and contextual multi-armed bandit in indoor environment

Contact Info

Product

Resources

About