Towards Reverse-Engineering Black-Box Neural Networks

Abstract

Many deployed learned models are black boxes: given input, returns output. Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable. This work shows that such attributes of neural networks can be exposed from a sequence of queries. This has multiple implications. On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -we show that the revealed internal information helps generate more effective adversarial examples against the black box model. On the other hand, this technique can be used for better protection of private content from automatic recognition models using adversarial examples. Our paper suggests that it is actually hard to draw a line between white box and black box models. The code is available at goo.gl/MbYfsv.

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

“…We further assume that the shadow model uses the same ML algorithm and has the same hyperparameters as the target model. To achieve this in practice, the adversary can either rely on the same MLaaS provider which builds the target model or perform model extraction to approximate the target model [43], [30], [45]. Later in this section, we show this assumption can be relaxed as well.…”

“…Despite being popular, ML models are vulnerable to various security and privacy attacks, such as model inversion [12], adversarial examples [15], and model extraction [43], [30], [45]. In this paper, we concentrate on one such attack, namely membership inference attack.…”

Survey of Attacks and Defenses on Edge-Deployed Neural Networks

“…However, for more complex models such as neural networks, the attacker cannot simply solve nonlinear equations to arrive at model weights, but must instead train a 'student' network on inputoutput pairs collected from the API [31]. A similar work [32] shows the simplicity of reverse-engineering black-box neural network weights, architecture, optimization method and the training/data split. In [33], authors reframe the goal from model theft, to arriving at a 'knockoff' model exhibiting the same functionality.…”

“…In order to violate the integrity of a machine learning model, attackers may attempt to find adversarial examples [35]. While most attacks rely on having access to the white-box model or the output gradient, several works have shown that even black-box networks [32] and networks with obfuscated gradients [36] are not resistant to determined attackers.…”

“…API defenses: The majority of API attacks we have mentioned attempt to steal the model or the model architecture, learn which inputs have been used to train the model, or find adversarial examples for the model running on the device. As finding adversarial examples typically involves first stealing the model [32], we focus only on defenses against model exfiltration and membership inference attacks.…”