Survey: Leakage and Privacy at Inference Time

Jegorova, Marija; Kaul, Chaitanya; Mayor, Charlie; O’Neil, Alison Q.; Weir, Alexander; Murray-Smith, Roderick; Tsaftaris, Sotirios A.

doi:10.1109/tpami.2022.3229593

Cited by 27 publications

(11 citation statements)

References 171 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is an active area of ML research not confined to TREs, but at present these risk measures focus on theoretical risk; that is, they describe the risk and how it may come about [ 20 ], but there is little evidence of how meaningful it is in operational research environments. Development is needed to identify (a) practical risk and (b) the necessary conditions for the risk to occur.…”

Section: Discussionmentioning

confidence: 99%

Machine learning models in trusted research environments -- understanding operational risks

Ritchie,

Tilbrook,

Cole

et al. 2023

IJPDS

View full text Add to dashboard Cite

IntroductionTrusted research environments (TREs) provide secure access to very sensitive data for research. All TREs operate manual checks on outputs to ensure there is no residual disclosure risk. Machine learning (ML) models require very large amount of data; if this data is personal, the TRE is a well-established data management solution. However, ML models present novel disclosure risks, in both type and scale. ObjectivesAs part of a series on ML disclosure risk in TREs, this article is intended to introduce TRE managers to the conceptual problems and work being done to address them. MethodsWe demonstrate how ML models present a qualitatively different type of disclosure risk, compared to traditional statistical outputs. These arise from both the nature and the scale of ML modelling. ResultsWe show that there are a large number of unresolved issues, although there is progress in many areas. We show where areas of uncertainty remain, as well as remedial responses available to TREs. ConclusionsAt this stage, disclosure checking of ML models is very much a specialist activity. However, TRE managers need a basic awareness of the potential risk in ML models to enable them to make sensible decisions on using TREs for ML model development.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine learning models in trusted research environments -- understanding operational risks

Ritchie,

Tilbrook,

Cole

et al. 2023

IJPDS

View full text Add to dashboard Cite

show abstract

“…Since new attack methods with new attack goals have been actively proposed and studied, this paper does not go into detail and collectively refers to these types of attacks as information leakage attacks of training data. For details, see previous papers on surveys and taxonomies [136,145,146].…”

Section: Information Leakage Attacks Of Training Datamentioning

confidence: 99%

“…Survey literature. For technical details of attacks and defenses, see previous papers, e.g., [136,145,146,[162][163][164][165] Finally, this paper focuses on centralized (supervised) learning and does not deal with distributed learning. As for the information leakage attacks in federated learning, see, e.g., [166][167][168] 6.9 System-Level Vulnerabilities and Controls to Malicious Input to Systems…”

Section: A63: ML Componentmentioning

confidence: 99%

Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomy

Kawamoto¹,

Miyake²,

Konishi³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this article, we propose the Artificial Intelligence Security Taxonomy to systematize the knowledge of threats, vulnerabilities, and security controls of machine-learning-based (ML-based) systems. We first classify the damage caused by attacks against ML-based systems, define ML-specific security, and discuss its characteristics. Next, we enumerate all relevant assets and stakeholders and provide a general taxonomy for ML-specific threats. Then, we collect a wide range of security controls against ML-specific threats through an extensive review of recent literature. Finally, we classify the vulnerabilities and controls of an ML-based system in terms of each vulnerable asset in the system's entire lifecycle.

show abstract

“…Naveed et al. , 2015; Jegorova et al. , Forthcoming), which focuses on how certain knowledge about data could be inferred from various sources such as query answers over data, statistics or machine learning models built over data, etc.…”

Section: Implications On Future Privacy Researchmentioning

confidence: 99%

Privacy implications of blockchain systems: a data management perspective

Zhang

2023

OCJ

View full text Add to dashboard Cite

PurposePrivacy scholars appear to struggle in conceptualizing blockchain from a privacy perspective: is it a privacy-enhancing mechanism like differential privacy, a privacy-intruding tool like third-party cookies or a technology orthogonal to the issue of privacy? Blockchain does not seem to neatly fit into any of these buckets that we traditionally use to gauge the privacy implications of information technologies. In this article, the authors argue that blockchain transcends the extant conceptualization of privacy because it modifies the nature of data flow upon which the modern concept of privacy is based.Design/methodology/approachThe authors introduce a conceptualization of blockchain as a new mechanism for data management. Then, following this conceptualization, the authors present a functional review of blockchain, summarizing the features it provides for the data it manages. This review sets up the discussion of how blockchain redefines data flow by separating the power of collection, access and query of data to different entities. After illustrating how this change regrounds privacy concerns in a blockchain system, the authors conclude with a discussion of the recommendations for future privacy research on blockchain.FindingsThe authors demonstrate that blockchain, by design, separates three core data-centric operations that are assumed to be inextricably linked in the canonical conceptualization of privacy: the collection, access and query of data. Collection means to capture and then store the data; access means to modify or augment the data and query means the ability to test or verify certain properties of the data (e.g. whether a bank account has a zero balance). Traditionally, any entities that collect data can evidently read, modify or query the same data as they wish. With blockchain, however, an entity that stores the data may not be able to modify the data, yet an entity that cannot even read the data may be able to verify certain properties of the data.Originality/valuePrivacy scholars appear to struggle in conceptualizing blockchain from a privacy perspective: is it a privacy-enhancing mechanism like differential privacy, a privacy-intruding tool like third-party cookies or a technology orthogonal to the issue of privacy? In this article, the authors aim to respond to this important question.

show abstract

Survey: Leakage and Privacy at Inference Time

Cited by 27 publications

References 171 publications

Machine learning models in trusted research environments -- understanding operational risks

Machine learning models in trusted research environments -- understanding operational risks

Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomy

Privacy implications of blockchain systems: a data management perspective

Contact Info

Product

Resources

About