Proof-of-Learning: Definitions and Practice

Jia, Hengrui; Yaghini, Mohammad; Choquette-Choo, Christopher A.; Dullerud, Natalie; Thudi, Anvith; Chandrasekaran, Varun; Papernot, Nicolas

doi:10.48550/arxiv.2103.05633

Cited by 3 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Weber et al [265] and Levine et al [266] both propose newer certified defenses against data poisoning; the former uses techniques similar to randomized smoothing [240] and the latter uses collective knowledge from an ensemble of models to prove robustness. Finally, integrity of the computation and model parameter values can also be verified by a mechanism that provides a proof associated with the particular run of gradient descent which produced the model parameters, as suggested by Jia et al [151].…”

Section: B Assurance For Trainingmentioning

confidence: 98%

“…If a trusted third party can reproduce these states (within some acceptable tolerance threshold) with the help of the honest principal's secret information, then the honest principal's ownership of the model is validated. This is the premise of the proof-oflearning approach to enable ownership [151]. Call to action: First, notice that claiming model ownership and differentiating between two models are two sides of the same coin.…”

Section: B Model Ownershipmentioning

confidence: 99%

“…Consequently, the ML logged information may greatly differ from traditional event logs used in application, network, and system security. For example, the proof (or log, albeit it is not referred as such) created in the work of Jia et al [151] is created with the explicit goal of proving model (parameter) ownership and requires information about data used to train the model, randomness used in the training process (which is hard to precisely capture), and tracks the change in model parameters as training progresses.…”

Section: B Auditing To Identify New Model Failuresmentioning

confidence: 99%

See 2 more Smart Citations

SoK: Machine Learning Governance

Chandrasekaran,

Jia,

Thudi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The application of machine learning (ML) in computer systems introduces not only many benefits but also risks to society. In this paper, we develop the concept of ML governance to balance such benefits and risks, with the aim of achieving responsible applications of ML. Our approach first systematizes research towards ascertaining ownership of data and models, thus fostering a notion of identity specific to ML systems. Building on this foundation, we use identities to hold principals accountable for failures of ML systems through both attribution and auditing. To increase trust in ML systems, we then survey techniques for developing assurance, i.e., confidence that the system meets its security requirements and does not exhibit certain known failures. This leads us to highlight the need for techniques that allow a model owner to manage the life cycle of their system, e.g., to patch or retire their ML system. Put altogether, our systematization of knowledge standardizes the interactions between principals involved in the deployment of ML throughout its life cycle. We highlight opportunities for future work, e.g., to formalize the resulting game between ML principals.

show abstract

Section: B Assurance For Trainingmentioning

confidence: 98%

Section: B Model Ownershipmentioning

confidence: 99%

Section: B Auditing To Identify New Model Failuresmentioning

confidence: 99%

See 1 more Smart Citation

SoK: Machine Learning Governance

Chandrasekaran,

Jia,

Thudi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reactive defenses are post-hoc methods that are used to determine if a suspect model was stolen or not. The examples of such methods are watermarking (Jia et al, 2020), dataset inference (Maini et al, 2021), and Proof-of-Learning (PoL) (Jia et al, 2021)…”

Section: Defenses Against Model Extractionmentioning

confidence: 99%

On the Difficulty of Defending Self-Supervised Learning against Model Extraction

Dziedzic¹,

Dhawan²,

Kaleem³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Self-Supervised Learning (SSL) is an increasingly popular ML paradigm that trains models to transform complex inputs into representations without relying on explicit labels. These representations encode similarity structures that enable efficient learning of multiple downstream tasks. Recently, ML-as-a-Service providers have commenced offering trained SSL models over inference APIs, which transform user inputs into useful representations for a fee. However, the high cost involved to train these models and their exposure over APIs both make black-box extraction a realistic security threat. We thus explore model stealing attacks against SSL. Unlike traditional model extraction on classifiers that output labels, the victim models here output representations; these representations are of significantly higher dimensionality compared to the low-dimensional prediction scores output by classifiers. We construct several novel attacks and find that approaches that train directly on a victim's stolen representations are query efficient and enable high accuracy for downstream models. We then show that existing defenses against model extraction are inadequate and not easily retrofitted to the specificities of SSL.

show abstract

“…If one could obtain M , then they could avoid approximate unlearning altogether and use that as the unlearned model. Another issue is that one can retrain on the same sequence of data from the same initialization and obtain different terminal weights [20] (which is caused by randomness and numerical instabilities in floating point operations).…”

Section: Unlearning Metricsmentioning

confidence: 99%

Unrolling SGD: Understanding Factors Influencing Machine Unlearning

Thudi¹,

Deza²,

Chandrasekaran³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Machine unlearning is the process through which a deployed machine learning model forgets about one of its training data points. While naively retraining the model from scratch is an option, it is almost always associated with a large computational effort for deep learning models. Thus, several approaches to approximately unlearn have been proposed along with corresponding metrics that formalize what it means for a model to forget about a data point. In this work, we first taxonomize approaches and metrics of approximate unlearning. As a result, we identify verification error, i.e., the 2 difference between the weights of an approximately unlearned and a naively retrained model, as a metric approximate unlearning should optimize for as it implies a large class of other metrics. We theoretically analyze the canonical stochastic gradient descent (SGD) training algorithm to surface the variables which are relevant to reducing the verification error of approximate unlearning for SGD. From this analysis, we first derive an easy-to-compute proxy for verification error (termed unlearning error). The analysis also informs the design of a new training objective penalty that limits the overall change in weights during SGD and as a result facilitates approximate unlearning with lower verification error. We validate our theoretical work through an empirical evaluation on CIFAR-10, CIFAR-100, and IMDB sentiment analysis.

show abstract

Proof-of-Learning: Definitions and Practice

Cited by 3 publications

References 42 publications

SoK: Machine Learning Governance

SoK: Machine Learning Governance

On the Difficulty of Defending Self-Supervised Learning against Model Extraction

Unrolling SGD: Understanding Factors Influencing Machine Unlearning

Contact Info

Product

Resources

About