Reconfigurable and Computationally Efficient Architecture for Multi-Armed Bandit Algorithms

Santosh, Sandeep; Darak, Sumit J.

doi:10.1109/iscas45731.2020.9180702

“…In [29], we discussed the mapping of the UCB algorithm and its extensions on Zynq SoC via a hardware-software codesign approach. In [17] we proposed the modified KL-UCB algorithm by replacing optimization function in Eq.…”

Section: Mab Algorithms and State-of-the-art Reviewmentioning

confidence: 99%

“…5). Since steps 1 and 3 are identical to UCB and KL-UCB algorithms, we request readers to refer to [17], [29] for in-depth understanding and implementation. Due to limited page constraints, the discussion is focused only on Step 3: QF calculation though our implementation and tutorials include all three steps.…”

Section: Synthesizable Thompson Sampling Algorithm Formentioning

confidence: 99%

Multi-armed Bandit Algorithms on System-on-Chip: Go Frequentist or Bayesian?

Santosh

¹

,

Darak

²

2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

Multi-armed Bandit (MAB) algorithms identify the best arm among multiple arms via exploration-exploitation trade-off without prior knowledge of arm statistics. Their usefulness in wireless radio, IoT, and robotics demand deployment on edge devices, and hence, a mapping on system-on-chip (SoC) is desired. Theoretically, the Bayesian approach-based Thompson Sampling (TS) algorithm offers better performance than the frequentist approach-based Upper Confidence Bound (UCB) algorithm. However, TS is not synthesizable due to \textit{Beta} function. We address this problem by approximating it via a pseudo-random number generator-based approach and efficiently realize the TS algorithm on Zynq SoC. In practice, the type of arms distribution (e.g., Bernoulli, Gaussian, etc.) is unknown and hence, a single algorithm may not be optimal. We propose a reconfigurable and intelligent MAB (RI-MAB) framework. Here, intelligence enables the identification of appropriate MAB algorithms for a given environment, and reconfigurability allows on-the-fly switching between algorithms on the SoC. This eliminates the need for parallel implementation of algorithms resulting in huge savings in resources and power consumption. We analyze the functional correctness, area, power, and execution time of the proposed and existing architectures for various arm distributions, word-length, and hardware-software co-design approaches. We demonstrate the superiority of the RI-MAB over TS and UCB only architectures.

show abstract

“…Later, the resource utilization, power consumption, and execution time of these algorithms are analyzed. MAB algorithms such as KL-UCB, UCB v, and UCB t are not considered since UCB offers regret which is close to these algorithms with significant savings in resources, power consumption and execution time [17], [29].…”

mentioning

confidence: 99%

Multi-armed Bandit Algorithms on System-on-Chip: Go Frequentist or Bayesian?

Santosh¹,

Darak²

2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

Multi-armed Bandit (MAB) algorithms identify the best arm among multiple arms via exploration-exploitation trade-off without prior knowledge of arm statistics. Their usefulness in wireless radio, IoT, and robotics demand deployment on edge devices, and hence, a mapping on system-on-chip (SoC) is desired. Theoretically, the Bayesian approach-based Thompson Sampling (TS) algorithm offers better performance than the frequentist approach-based Upper Confidence Bound (UCB) algorithm. However, TS is not synthesizable due to \textit{Beta} function. We address this problem by approximating it via a pseudo-random number generator-based approach and efficiently realize the TS algorithm on Zynq SoC. In practice, the type of arms distribution (e.g., Bernoulli, Gaussian, etc.) is unknown and hence, a single algorithm may not be optimal. We propose a reconfigurable and intelligent MAB (RI-MAB) framework. Here, intelligence enables the identification of appropriate MAB algorithms for a given environment, and reconfigurability allows on-the-fly switching between algorithms on the SoC. This eliminates the need for parallel implementation of algorithms resulting in huge savings in resources and power consumption. We analyze the functional correctness, area, power, and execution time of the proposed and existing architectures for various arm distributions, word-length, and hardware-software co-design approaches. We demonstrate the superiority of the RI-MAB over TS and UCB only architectures.

show abstract