Are Labels Always Necessary for Classifier Accuracy Evaluation?

Deng, Weijian; Zheng, Liang

doi:10.1109/cvpr46437.2021.01482

Cited by 58 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A separate line of work departs from complexity measures altogether and directly predicts OOD generalization from unlabelled test data. These methods either predict the correctness of the model directly on individual examples [14,32,15] or directly estimate the total error [19,24,9,10,68]. Although these methods work well in practice, they do not provide any insight into the underlying mechanism of generalization since they act only on the output layer of the network.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Out-of-Domain Generalization with Local Manifold Smoothness

Ng¹,

Cho²,

Hulkund³

et al. 2022

Preprint

View full text Add to dashboard Cite

Understanding how machine learning models generalize to new environments is a critical part of their safe deployment. Recent work has proposed a variety of complexity measures that directly predict or theoretically bound the generalization capacity of a model. However, these methods rely on a strong set of assumptions that in practice are not always satisfied. Motivated by the limited settings in which existing measures can be applied, we propose a novel complexity measure based on the local manifold smoothness of a classifier. We define local manifold smoothness as a classifier's output sensitivity to perturbations in the manifold neighborhood around a given test point. Intuitively, a classifier that is less sensitive to these perturbations should generalize better. To estimate smoothness we sample points using data augmentation and measure the fraction of these points classified into the majority class. Our method only requires selecting a data augmentation method and makes no other assumptions about the model or data distributions, meaning it can be applied even in out-of-domain (OOD) settings where existing methods cannot. In experiments on robustness benchmarks in image classification, sentiment analysis, and natural language inference, we demonstrate a strong and robust correlation between our manifold smoothness measure and actual OOD generalization on over 3,000 models evaluated on over 100 train/test domain pairs. Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Directly estimating the generalization of a trained model on test data is one approach to this problem [14,32,15,19]. However, these methods are typically calculated based on the output predictive distribution of a model, which can become poorly calibrated in out-of-domain settings.…”

Section: Introductionmentioning

confidence: 99%

Predicting Out-of-Domain Generalization with Local Manifold Smoothness

Ng¹,

Cho²,

Hulkund³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This work has only recently been applied to information security, specifically for Windows malware classification [27]. Directly targeting model evaluation instead, Deng and Zheng [28] proposed AutoEval to estimate the accuracy of a classifier on an unlabeled dataset by using feature statistics from the training set and synthetic datasets generated by applying transformations to the training set. Novák et al [29] use density estimation to similar ends.…”

Section: Related Workmentioning

confidence: 99%

Firenze: Model Evaluation Using Weak Signals

Soman¹,

Torkamani²,

Morais³

et al. 2022

Preprint

View full text Add to dashboard Cite

Data labels in the security field are frequently noisy, limited, or biased towards a subset of the population. As a result, commonplace evaluation methods such as accuracy, precision and recall metrics, or analysis of performance curves computed from labeled datasets do not provide sufficient confidence in the real-world performance of a machine learning (ML) model. This has slowed the adoption of machine learning in the field. In the industry today, we rely on domain expertise and lengthy manual evaluation to build this confidence before shipping a new model for security applications. In this paper, we introduce Firenze, a novel framework for comparative evaluation of ML models' performance using domain expertise, encoded into scalable functions called markers. We show that markers computed and combined over select subsets of samples called regions of interest can provide a robust estimate of their real-world performances. Critically, we use statistical hypothesis testing to ensure that observed differences-and therefore conclusions emerging from our framework-are more prominent than that observable from the noise alone. Using simulations and two real-world datasets for malware and domain-name-service reputation detection, we illustrate our approach's effectiveness, limitations, and insights. Taken together, we propose Firenze as a resource for fast, interpretable, and collaborative model development and evaluation by mixed teams of researchers, domain experts, and business owners.Preprint. Under review.

show abstract

“…Recently, several studies outside the field of TL have started to establish a connection between the distribution divergence and accuracy. Among them, the most related work is from Elsahar and Galle [11], [12], who used various methods such as H-divergence, Fréchet distance and confidence-based metrics to predict the accuracy drop of modern NLP and computer version (CV) models under domain shifts. Both studies above used predicted accuracy drops to evaluate the robustness of trained models.…”

Section: Related Workmentioning

confidence: 99%

Divergence-Based Transferability Analysis for Self-Adaptive Smart Grid Intrusion Detection With Transfer Learning

et al. 2022

View full text Add to dashboard Cite

Machine learning is a popular approach to security monitoring and intrusion detection in cyber-physical systems (CPS) like the smart grid. However, these highly dynamic CPS operating in open environments can result in significant data distribution divergence, which may require the adaptation of a learned model. While transfer learning has been an effective approach to retain the performance against the divergence, there is still limited work on a more fundamental question that can be called transferability: when should one apply transfer learning? To address this challenge, this paper proposes a divergence-based transferability analysis to decide whether to apply transfer learning and autonomically adapt learning-based intrusion detectors. This work first identifies three metrics used to measure the divergence between data distributions, and then explores the relation between detector's accuracy drop and divergence in extensive temporal, spatial, and spatiotemporal experiments. Two regression models are trained to approximate the divergence-accuracy relation and then used to predict an accuracy drop which determines whether to apply transfer learning. Finally, a state-of-the-art domain adversarial neural network (DANN) classifier is adopted as the transfer learning model. Datasets from real normal operation profiles and simulated attacks are used to validate the effectiveness of the proposed transferability analysis against variations in attack timing, locations, and both. In all three scenarios, the proposed analysis demonstrated high accuracy in predicting accuracy drop from the divergence, with an RMSE lower than 4.20%, and the DANN can be timely triggered to achieve an accuracy improvement over 5.00%.

show abstract

Are Labels Always Necessary for Classifier Accuracy Evaluation?

Cited by 58 publications

References 15 publications

Predicting Out-of-Domain Generalization with Local Manifold Smoothness

Predicting Out-of-Domain Generalization with Local Manifold Smoothness

Firenze: Model Evaluation Using Weak Signals

Divergence-Based Transferability Analysis for Self-Adaptive Smart Grid Intrusion Detection With Transfer Learning

Contact Info

Product

Resources

About