Ease.ML/Snoopy: Towards Automatic Feasibility Studies for ML via Quantitative Understanding of "Data Quality for ML"

Renggli, Cédric; Rimanic, Luka; Kolar, Luka; Wu, Wentao; Zhang, Ce

doi:10.48550/arxiv.2010.08410

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The notion of lowest possible error generalizes the framework proposed by Cai et al [7] in that the lowest error is naturally achieved if the features of all website are indistinguishable. Estimating the BER or its bounds using finite datasets is an extensively researched problem in the field of machine learning [5,12,15,19,20,50,55,61]. Inspired by Cover and Hart [12], Cherubin reduces the WF problem to a classification task and leverages the error of the Nearest Neighbor classifier as a proxy to estimate the lower bound for the error of any potential classifier used on predefined features.…”

Section: Security Estimation Of Wf Defensesmentioning

confidence: 99%

“…Guided by the reasoning in observation (2) and the fact that the sum of finite-sample bias and transformation bias are positive and typically strictly larger than the tightness of the lower bound estimate, it is natural that even without having knowledge of the additional induced biases the NN-based estimators can only benefit from deterministic feature transformations in the finite-sample case. A natural consequence of this is that one can simply achieve a better estimate of the BER by not only inspecting a single representation, but rather try many different transformations and report the minimal achieved BER [55]. Note that a theoretical counterpart for NN-based MI estimators (e.g., negative finite-sample bias) is missing.…”

Section: Theoretical Reasoningmentioning

confidence: 99%

“…We use the original attacks/defenses with their recommended parameters, i.e., we perform no parameter tuning (Appendix B). We also implemented the kNN-based BER estimator proposed by Renggli et al [55].…”

Section: Cross-validationmentioning

confidence: 99%

See 2 more Smart Citations

DeepSE-WF: Unified Security Estimation for Website Fingerprinting Defenses

Veicht¹,

Renggli²,

Barradas³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Website fingerprinting (WF) attacks, usually conducted with the help of a machine learning-based classifier, enable a network eavesdropper to pinpoint which web page a user is accessing through the inspection of traffic patterns. These attacks have been shown to succeed even when users browse the Internet through encrypted tunnels, e.g., through Tor or VPNs. To assess the security of new defenses against WF attacks, recent works have proposed feature-dependent theoretical frameworks that estimate the Bayes error of an adversary's features set or the mutual information leaked by manually-crafted features. Unfortunately, as stateof-the-art WF attacks increasingly rely on deep learning and latent feature spaces, security estimations based on simpler (and less informative) manually-crafted features can no longer be trusted to assess the potential success of a WF adversary in defeating such defenses. In this work, we propose DeepSE-WF, a novel WF security estimation framework that leverages specialized kNN-based estimators to produce Bayes error and mutual information estimates from learned latent feature spaces, thus bridging the gap between current WF attacks and security estimation methods. Our evaluation reveals that DeepSE-WF produces tighter security estimates than previous frameworks, reducing the required computational resources to output security estimations by one order of magnitude.

show abstract