Thompson Sampling on Asymmetric $α$-Stable Bandits

Abstract

In algorithm optimization in reinforcement learning, how to deal with the exploration-exploitation dilemma is particularly important. Multi-armed bandit problem can optimize the proposed solutions by changing the reward distribution to realize the dynamic balance between exploration and exploitation. Thompson Sampling is a common method for solving multi-armed bandit problem and has been used to explore data that conform to various laws. In this paper, we consider the Thompson Sampling approach for multi-armed bandit problem, in which rewards conform to unknown asymmetric α-stable distributions and explore their applications in modelling financial and wireless data.

Deep zero-inflated negative binomial model and its application in scRNA-seq data integration

“…In UCB-type algorithms, the confidence level δ is set to 4/T 2 , maintaining consistency. The prior parameters and tuning parameters for both TS-type algorithms are configured in accordance with the recommendations provided in [23,39] for the MOTS algorithm and GMS generation. The simulation results with different size of p = max k∈[K] p k are shown in Figure 1, Figure 2, and Figure 3.…”

“…where the function g(•, ϵ) is defined in Lemma 2. and U µ k = µ At + M 1/(1+ϵ) 32 log t/c k (t) ϵ/(1+ϵ) ; end 'Chambers-Mallows-Stuck (CMS) Generation,' to rescale the non-zero part to a sub-Gaussian tail. Further details on this can be found in [50,17,39]. Diverging from the standard TS algorithm for Gaussian rewards, we use a clipped Gaussian distribution cl N (µ, σ 2 ; ϑ) := max N (µ, σ 2 ), ϑ as the posterior for the non-zero sub-Gaussian part X.…”

“…For Gaussian and mixed-Gaussian rewards, we can directly apply both Algorithm 3 and the MOTS algorithm. But, when applying with Exponential rewards, we adopt Algorithm 1 from [39]. In doing so, we integrate their step 5 with our algorithm and the MOTS algorithm.…”