Cooperative Thresholded Lasso for Sparse Linear Bandit

Abstract

We present a novel approach to address the multi-agent sparse contextual linear bandit problem, in which the feature vectors have a high dimension d whereas the reward function depends on only a limited set of features - precisely s0 ≪ d. Furthermore, the learning follows under information-sharing constraints. The proposed method employs Lasso regression for dimension reduction, allowing each agent to independently estimate an approximate set of main dimensions and share that information with others depending on the network’s structure. The information is then aggregated through a specific process and shared with all agents. Each agent then resolves the problem with ridge regression focusing solely on the extracted dimensions. We represent algorithms for both a star-shaped network and a peer-to-peer network. The approaches effectively reduce communication costs while ensuring minimal cumulative regret per agent. Theoretically, we show that our proposed methods have a regret bound of order O(s0 log d + s0 √T) with high probability, where T is the time horizon. To our best knowledge, it is the first algorithm that tackles row-wise distributed data in sparse linear bandits, achieving comparable performance compared to the state-of-the-art single and multi-agent methods. Besides, it is widely applicable to high-dimensional multi-agent problems where efficient feature extraction is critical for minimizing regret. To validate the effectiveness of our approach, we present experimental results on both synthetic and real-world datasets.

New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problem

“…Thresholded lasso bandit (Ariu, Abe, and Proutière 2022) The thresholded lasso bandit estimates the support of β * each round and a greedy arm selection policy is performed by the inner product of the arm features and the estimated parameter for β * on this support. The relaxed symmetry and the balanced covariance are introduced to ensure proper support estimation under the greedy policy.…”

“…These two generalizations provide theoretical guarantees of the greedy policy for a very wide range of the arm feature distributions. Moreover, we demonstrate the usefulness of our analysis by applying it to the other cases: thresholded lasso bandit (Ariu, Abe, and Proutière 2022), combinatorial setting, and non-sparse setting (Bastani and Bayati 2020).…”

“…Oh, Iyengar, and Zeevi (2021) proposed a sparsity-agnostic algorithm and, under the assumption for the arm feature distribution, called relaxed symmetry, gave a regret upper bound of O(s 0 log(dT )T ). The thresholded lasso bandit algorithm proposed by Ariu, Abe, and Proutière (2022) is another sparsity-agnostic algorithm. They showed an O( √ s 0 T ) upper bound under an additional assumption about the sparse positive definiteness of the arm feature distribution.…”