Sample-efficient proper PAC learning with approximate differential privacy

Ghazi, Badih; Golowich, Noah; Kumar, Ravi; Manurangsi, Pasin

doi:10.1145/3406325.3451028

Cited by 18 publications

(29 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beaulieu‐Jones et al (2019) also used DP with GAN to generate biomedical data for clinical diagnosis. Ghazi et al (2021) used randomized responses with prior and DP to improve the model performance with data privacy.…”

Section: Applications Of Ppdm and ML In Medical And Healthcarementioning

confidence: 99%

Privacy‐preserving data mining and machine learning in healthcare: Applications, challenges, and solutions

Naresh¹,

Thamarai²

2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Data mining (DM) and machine learning (ML) applications in medical diagnostic systems are budding. Data privacy is essential in these systems as healthcare data are highly sensitive. The proposed work first discusses various privacy and security challenges in these systems. To address these next, we discuss different privacy-preserving (PP) computation techniques in the context of DM and ML for secure data evaluation and processing. The state-of-the-art applications of these systems in healthcare are analyzed at various stages such as data collection, data publication, data distribution, and output phases regarding PPDM and input, model, training, and output phases in the context of PPML. Furthermore, PP federated learning is also discussed. Finally, we present open challenges in these systems and future research directions.

show abstract

Section: Applications Of Ppdm and ML In Medical And Healthcarementioning

confidence: 99%

Privacy‐preserving data mining and machine learning in healthcare: Applications, challenges, and solutions

Naresh¹,

Thamarai²

2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…A recent line of work revealed a qualitative characterization of DP-learnability in the PAC model: A total concept class H can be PAC learned by a DP-algorithm if and only if its Littlestone dimension LD(H) is finite (Alon, Livni, Malliaris, and Moran, 2019;Gonen, Hazan, and Moran, 2019;Bun, Livni, and Moran, 2020;Ghazi, Golowich, Kumar, and Manurangsi, 2020). (The Littlestone dimension is a combinatorial parameter which arises in the context of online learning, see Section A for a formal definition.)…”

Section: Littlestone Dimension Vs Private Learningmentioning

confidence: 99%

“…The known proofs of the direction "LD(H) < ∞ =⇒ H is DP-learnable" for total concept classes (Bun, Livni, and Moran, 2020;Ghazi, Golowich, Kumar, and Manurangsi, 2020) utilize (among other things) the ERM principle and uniform convergence which, as discussed earlier, is not satisfied by partial concept classes.…”

Section: Littlestone Dimension Vs Private Learningmentioning

confidence: 99%

A Theory of PAC Learnability of Partial Concept Classes

Alon¹,

Hanneke²,

Holzman³

et al. 2021

Preprint

View full text Add to dashboard Cite

We extend the classical theory of PAC learning in a way which allows to model a rich variety of practical learning tasks where the data satisfy special properties that ease the learning process. For example, tasks where the distance of the data from the decision boundary is bounded away from zero, or tasks where the data lie on a lower dimensional surface. The basic and simple idea is to consider partial concepts: these are functions that can be undefined on certain parts of the space. When learning a partial concept, we assume that the source distribution is supported only on points where the partial concept is defined.This way, one can naturally express assumptions on the data such as lying on a lower dimensional surface, or that it satisfies margin conditions. In contrast, it is not at all clear that such assumptions can be expressed by the traditional PAC theory using learnable total concept classes, and in fact we exhibit easy-to-learn partial concept classes which provably cannot be captured by the traditional PAC theory. This also resolves, in a strong negative sense, a question posed by Attias, Kontorovich, and Mansour (2019).We characterize PAC learnability of partial concept classes and reveal an algorithmic landscape which is fundamentally different than the classical one. For example, in the classical PAC model, learning boils down to Empirical Risk Minimization (ERM). This basic principle follows from

show abstract

“…A long line of work focuses on preserving differential privacy while extracting specific information from data, such as executing machine learning algorithms [Raskhodnikova et al, 2008, Chaudhuri et al, 2011, Feldman and Xiao, 2014, Bassily et al, 2014 or answering a number of pre-defined queries, see e.g. Dwork and Nissim [2004], Blum et al [2005], Hardt and Talwar [2010], Steinke and Ullman [2016], Ghazi et al [2021], Dagan and Kur [2022]. Another line of work aims to preserve differential privacy for a-priori unknown queries from a large function class by releasing a synthetic data set [Blum et al, 2008, Hardt and Rothblum, 2010, Hardt et al, 2012, Liu et al, 2021.…”

Section: Introductionmentioning

confidence: 99%

Sample-efficient private data release for Lipschitz functions under sparsity assumptions

Donhauser¹,

Lokna²,

Sanyal³

et al. 2023

Preprint

View full text Add to dashboard Cite

Differential privacy is the de facto standard for protecting privacy in a variety of applications. One of the key challenges is private data release, which is particularly relevant in scenarios where limited information about the desired statistics is available beforehand. Recent work has presented a differentially private data release algorithm that achieves optimal rates of order n −1/d , with n being the size of the dataset and d being the dimension, for the worst-case error over all Lipschitz continuous statistics. This type of guarantee is desirable in many practical applications, as for instance it ensures that clusters present in the data are preserved. However, due to the "slow" rates, it is often infeasible in practice unless the dimension of the data is small. We demonstrate that these rates can be significantly improved to n −1/s when only guarantees over s-sparse Lipschitz continuous functions are required, or to n −1/(s+1) when the data lies on an unknown s-dimensional subspace, disregarding logarithmic factors. We therefore obtain practically meaningful rates for moderate constants s which motivates future work on computationally efficient approximate algorithms for this problem.

show abstract

Sample-efficient proper PAC learning with approximate differential privacy

Cited by 18 publications

References 26 publications

Privacy‐preserving data mining and machine learning in healthcare: Applications, challenges, and solutions

Privacy‐preserving data mining and machine learning in healthcare: Applications, challenges, and solutions

A Theory of PAC Learnability of Partial Concept Classes

Sample-efficient private data release for Lipschitz functions under sparsity assumptions

Contact Info

Product

Resources

About