Monitoring-Based Differential Privacy Mechanism Against Query Flooding-Based Model Extraction Attack

Yan, Haonan; Li, Xiaoguang; Li, Hui; Li, Jiamin; Sun, Wen‐Yih; Li, Fenghua

doi:10.1109/tdsc.2021.3069258

Cited by 30 publications

(13 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these defenses may significantly reduce the performance of legitimate users and could even be bypassed by adaptive attacks (Jia et al 2021;Maini, Yaghini, and Papernot 2021). Other works (Kesarwani et al 2018;Juuti et al 2019;Yan et al 2021) detected model stealing by identifying malicious queries. However, these methods relied on some assumptions of malicious query patterns, which may not be adopted by the adversaries in practice.…”

Section: Defenses Against Model Stealingmentioning

confidence: 99%

Defending against Model Stealing via Verifying Embedded External Features

Zhu

Jia

et al. 2022

AAAI

View full text Add to dashboard Cite

Obtaining a well-trained model involves expensive data collection and training procedures, therefore the model is a valuable intellectual property. Recent studies revealed that adversaries can `steal' deployed models even when they have no training samples and can not get access to the model parameters or structures. Currently, there were some defense methods to alleviate this threat, mostly by increasing the cost of model stealing. In this paper, we explore the defense from another angle by verifying whether a suspicious model contains the knowledge of defender-specified external features. Specifically, we embed the external features by tempering a few training samples with style transfer. We then train a meta-classifier to determine whether a model is stolen from the victim. This approach is inspired by the understanding that the stolen models should contain the knowledge of features learned by the victim model. We examine our method on both CIFAR-10 and ImageNet datasets. Experimental results demonstrate that our method is effective in detecting different types of model stealing simultaneously, even if the stolen model is obtained via a multi-stage stealing process. The codes for reproducing main results are available at Github (https://github.com/zlh-thu/StealingVerification).

show abstract

Section: Defenses Against Model Stealingmentioning

confidence: 99%

Defending against Model Stealing via Verifying Embedded External Features

Zhu

Jia

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Yan et al used an ESA to steal a MLP under a differential privacy defence [13], which adds noise to the outputs laying close to the decision boundary [75] (cf. Section 9.2.3).…”

Section: Equation-solving Attacksmentioning

confidence: 99%

“…In [11] the authors claim that this watermarking technique can be effective against knockoff SMA [3]. This approach is called monitor [95] or monitoring-based [13]. Kesarwani et al [95] implemented a monitor which estimates coverage of the data space by the queries issued, and can from this infer a kind-of "extraction completeness status".…”

Section: Defences Against Model Stealingmentioning

confidence: 99%

“…Later, Yan et al [13] broke BDPL with their query-flooding parameter duplication attack (QPD) (cf. Section 7.2), and subsequently proposed a new defence called MDP, which combines differential privacy as in [75] with monitoring to mitigate the QPD attack.…”

Section: Data Perturbationmentioning

confidence: 99%

“…The second approach is considered to prevent the attack, or at least make it less effective. Unfortunately, a lot of the defences can either be fooled [10][11][12] or are effective only under specific conditions [13]. This raises the importance of studying existing approaches and investigating new ones.…”

mentioning

confidence: 99%

See 2 more Smart Citations

I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences

Oliynyk¹,

Mayer²,

Rauber³

2022

Preprint

View full text Add to dashboard Cite

Machine Learning-as-a-Service (MLaaS) has become a widespread paradigm, making even the most complex machine learning models available for clients via e.g. a pay-per-query principle. This allows users to avoid time-consuming processes of data collection, hyperparameter tuning, and model training. However, by giving their customers access to the (predictions of their) models, MLaaS providers endanger their intellectual property, such as sensitive training data, optimised hyperparameters, or learned model parameters.Adversaries can create a copy of the model with (almost) identical behavior using the the prediction labels only. While many variants of this attack have been described, only scattered defence strategies have been proposed, addressing isolated threats. This raises the necessity for a thorough systematisation of the field of model stealing, to arrive at a comprehensive understanding why these attacks are successful, and how they could be holistically defended against. We address this by categorising and comparing model stealing attacks, assessing their performance, and exploring corresponding defence techniques in different settings. We propose a taxonomy for attack and defence approaches, and provide guidelines on how to select the right attack or defence strategy based on the goal and available resources. Finally, we analyse which defences are rendered less effective by current attack strategies.CCS Concepts: • Security and privacy → Vulnerability management; • Computing methodologies → Machine learning.

show abstract

Artificial Intelligence and Differential Privacy: Review of Protection Estimate Models

Kilpala,

Kärkkäinen

2024

Artificial Intelligence for Security

View full text Add to dashboard Cite

Monitoring-Based Differential Privacy Mechanism Against Query Flooding-Based Model Extraction Attack

Cited by 30 publications

References 26 publications

Defending against Model Stealing via Verifying Embedded External Features

Defending against Model Stealing via Verifying Embedded External Features

I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences

Artificial Intelligence and Differential Privacy: Review of Protection Estimate Models

Contact Info

Product

Resources

About