Membership Inference Attacks Against Temporally Correlated Data in Deep Reinforcement Learning

Abstract

While significant research advances have been made in the field of deep reinforcement learning, there have been no concreteadversarial attack strategies in literature tailored for studying the vulnerability of deep reinforcement learning algorithms tomembership inference attacks. In such attacking systems, the adversary targets the set of collected input data on which thedeep reinforcement learning algorithm has been trained. To address this gap, we propose an adversarial attack frameworkdesigned for testing the vulnerability of a state-of-the-art deep reinforcement learning algorithm to a membership inferenceattack. In particular, we design a series of experiments to investigate the impact of temporal correlation, which naturally existsin reinforcement learning training data, on the probability of information leakage. Moreover, we compare the performance ofcollective and individual membership attacks against the deep reinforcement learning algorithm. Experimental results showthat the proposed adversarial attack framework is surprisingly effective at inferring data with an accuracy exceeding 84% inindividual and 97% in collective modes in three different continuous control Mujoco tasks, which raises serious privacy concernsin this regard. Finally, we show that the learning state of the reinforcement learning algorithm influences the level of privacybreaches significantly.

ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning

“…Membership Inference Attack against RL [46], [23], [22]. Several membership inference attacks exist against DRL, which seem to address the problem studied in this paper.…”

“…Competitors. Recalling Section III-B, existing methods [46], [23], [22] are designed for the online reinforcement learning scenes, assuming that the auditor can continuously interact with the environment to obtain new data as the non-member example. Based on the behavioral difference of the model between the member examples and the non-member examples, they build the member inference method to detect whether an example is used to train the suspect model.…”

“…In offline RL scenes, since the distributions of the collected datasets depend on both environment and operator [17], it is difficult to determine suitable public dataset to train the surrogate model, making the audit basis hard to establish. The third category adopts the notion of membership inference [46], [23], [22]. By collecting the RL models' behaviors on the trained examples (members) and the untrained examples (non-members), a classifier is constructed to determine whether a data sample is used in the model's learning process.…”

“…By collecting the RL models' behaviors on the trained examples (members) and the untrained examples (non-members), a classifier is constructed to determine whether a data sample is used in the model's learning process. However, unlike online scenarios in [46], [23], [22], the auditor cannot collect additional data from the environment as the non-member examples in offline cases, where the auditor does not have access to the environment.…”

On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models

“…By using a binary classifier, we train labeling agents only for the environment map with which the MIA is faced. In another work, the authors of [18] develop an MIA that infers the membership of a batch-constrained deep Q-learning agent's roll-out trajectories stored in its replay buffer. We do not follow the above work's methodology because we do not restrict the algorithm that is used to train the reinforcement learning agents.…”