Knockoff Nets: Stealing Functionality of Black-Box Models

Abstract

Machine Learning (ML) models are increasingly deployed in the wild to perform a wide range of tasks. In this work, we ask to what extent can an adversary steal functionality of such "victim" models based solely on blackbox interactions: image in, predictions out. In contrast to prior work, we present an adversary lacking knowledge of train/test data used by the model, its internals, and semantics over model outputs. We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" with queried image-prediction pairs. We make multiple remarkable observations: (a) querying random images from a different distribution than that of the blackbox training data results in a well-performing knockoff; (b) this is possible even when the knockoff is represented using a different architecture; and (c) our reinforcement learning approach additionally improves query sample efficiency in certain settings and provides performance gains. We validate model functionality stealing on a range of datasets and tasks, as well as on a popular image analysis API where we create a reasonable knockoff for as little as $30.

Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI

“…For example, imagine a model that some company has developed through many years of research in a specific field. The knowledge synthesized in the model built might be considered to be confidential, and it may be compromised even by providing only input and output access [356]. The latter shows that, under minimal assumptions, data model functionality stealing is possible.…”

“…Notwithstanding this explicit concern from regulatory bodies, loss of privacy has been compromised by DL methods in scenarios where no data fusion is performed. For instance, a few images are enough to threaten users' privacy even in the presence of image obfuscation [420], and the model parameters of a DNN can be exposed by simply performing input queries on the model [356,357]. An approach to explain loss of privacy is by using privacy loss and intent loss subjective scores.…”

“…The former provides a subjective measure of the severity of the privacy violation depending on the role of a face in the image, while the latter captures the intent of the bystanders to appear in the picture. These kind of explanations have motivated, for instance, secure matching cryptographic protocols for photographer and bystanders to preserve privacy [356,421,422]. We definite advocate for more efforts invested in this direction, namely, in ensuring that XAI methods do not pose a threat in regards to the privacy of the data used for training the ML model under target.…”

PRADA: Protecting Against DNN Model Stealing Attacks

“…In these adversary models, adversary is not assumed to have access to pre-trained models: both the target and the substitute model DNNs are trained from scratch. Since the time of this writing, stealing DNNs for more complicated datasets like CIFAR-10 6 , have been done by assuming both the target model and attacker models are finetuned from pre-trained ImageNet classifiers [37], [40]. These attacks benefit from correlations between different [40] or same [37] pre-trained models.…”

Survey of Attacks and Defenses on Edge-Deployed Neural Networks

“…A similar work [32] shows the simplicity of reverse-engineering black-box neural network weights, architecture, optimization method and the training/data split. In [33], authors reframe the goal from model theft, to arriving at a 'knockoff' model exhibiting the same functionality. In [34], authors ignore model parameters and instead attempt to steal the hyperparameters of a network.…”