A comprehensive evaluation framework for deep model robustness

Guo, Jun; Bao, Wei; Wang, Jiakai; Ma, Yuqing; Gao, Xinghai; Xiao, Gang; Liu, Aishan; Dong, Jian; Liu, Xianglong; Wu, Wenjun

doi:10.1016/j.patcog.2023.109308

Cited by 24 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several physical-world adversarial example generation methods have been proposed and demonstrated to be effective [10,13,[34][35][36][37][38][39]. However, they use different datasets for evaluation, which makes it difficult to conduct a comprehensive evaluation.…”

Section: Adversarial Robustness Benchmarkmentioning

confidence: 99%

“…Tang [42] proposed the first unified Robustness Assessment Benchmark, RobustART, which provides a standardized evaluation framework for adversarial examples. Recently, there has been a notable emergence of a series of benchmarks [39,[43][44][45][46][47][48] in different fields. These benchmarks serve the crucial function of unifying evaluation metrics, enabling a clear and systematic comparison of the strengths and weaknesses inherent in various methods.…”

Section: Adversarial Robustness Benchmarkmentioning

confidence: 99%

See 1 more Smart Citation

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

Jiang,

Zhang,

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

Adversarial attacks can compromise the robustness of real-world detection models. However, evaluating these models under real-world conditions poses challenges due to resource-intensive experiments. Virtual simulations offer an alternative, but the absence of standardized benchmarks hampers progress. Addressing this, we propose an innovative instant-level data generation pipeline using the CARLA simulator. Through this pipeline, we establish the Discrete and Continuous Instant-level (DCI) dataset, enabling comprehensive experiments involving three detection models and three physical adversarial attacks. Our findings highlight diverse model performances under adversarial conditions. YOLO v6 demonstrates remarkable resilience, exhibiting just a marginal 6.59% average drop in average precision (AP). In contrast, the ASA attack yields a substantial 14.51% average AP reduction, twice the effect of other algorithms. We also note that static scenes yield higher recognition AP values, and outcomes remain relatively consistent across varying weather conditions. Intriguingly, our study suggests that advancements in adversarial attack algorithms may be approaching its “limitation”. In summary, our work underscores the significance of adversarial attacks in real-world contexts and introduces the DCI dataset as a versatile benchmark. Our findings provide valuable insights for enhancing the robustness of detection models and offer guidance for future research endeavors in the realm of adversarial attacks.

show abstract

Section: Adversarial Robustness Benchmarkmentioning

confidence: 99%

Section: Adversarial Robustness Benchmarkmentioning

confidence: 99%

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

Jiang,

Zhang,

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…In order to comprehensively evaluate the security of deep learning models, some research [14][15][16][17][18][19][20][21][22] has proposed evaluation metrics, method libraries, and evaluation platforms. Works such as CleverHans [14] and FoolBox [15] integrated the most common attack and defense methods, but the neglect of code quality made some of them incorrectly developed 2 of 43 or even unworkable.…”

Section: Introductionmentioning

confidence: 99%

CANARY: An Adversarial Robustness Evaluation Platform for Deep Learning Models on Image Classification

Sun,

Chen,

Xia

et al. 2023

Electronics

View full text Add to dashboard Cite

The vulnerability of deep-learning-based image classification models to erroneous conclusions in the presence of small perturbations crafted by attackers has prompted attention to the question of the models’ robustness level. However, the question of how to comprehensively and fairly measure the adversarial robustness of models with different structures and defenses as well as the performance of different attack methods has never been accurately answered. In this work, we present the design, implementation, and evaluation of Canary, a platform that aims to answer this question. Canary uses a common scoring framework that includes 4 dimensions with 26 (sub)metrics for evaluation. First, Canary generates and selects valid adversarial examples and collects metrics data through a series of tests. Then it uses a two-way evaluation strategy to guide the data organization and finally integrates all the data to give the scores for model robustness and attack effectiveness. In this process, we use Item Response Theory (IRT) for the first time to ensure that all the metrics can be fairly calculated into a score that can visually measure the capability. In order to fully demonstrate the effectiveness of Canary, we conducted large-scale testing of 15 representative models trained on the ImageNet dataset using 12 white-box attacks and 12 black-box attacks and came up with a series of in-depth and interesting findings. This further illustrates the capabilities and strengths of Canary as a benchmarking platform. Our paper provides an open-source framework for model robustness evaluation, allowing researchers to perform comprehensive and rapid evaluations of models or attack/defense algorithms, thus inspiring further improvements and greatly benefiting future work.

show abstract

“…Furthermore, multiple objectives could be at odds with each other [21,47,54] and selecting features satisfying more than one objective still remains non-trivial. As research in responsible AI and regulatory requirements for machine learning models rapidly advance, new metrics are being developed to evaluate model performance, both within [28,41] and beyond [49] the secondary characteristics discussed Hence, we introduce the process of feature reselection. Feature reselection aims to select features to improve on secondary model performance characteristics (characteristics that become important to consider after a model is already developed) while maintaining similar performance with respect to a primary characteristic based on which features were already selected.…”

Section: Introductionmentioning

confidence: 99%

REFRESH: Responsible and Efficient Feature Reselection guided by SHAP values

Sharma,

Dutta,

Albini

et al. 2023

Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society

View full text Add to dashboard Cite

Feature selection is a crucial step in building machine learning models. This process is often achieved with accuracy as an objective, and can be cumbersome and computationally expensive for large-scale datasets. Several additional model performance characteristics such as fairness and robustness are of importance for model development.As regulations are driving the need for more trustworthy models, deployed models need to be corrected for model characteristics associated with responsible artificial intelligence. When feature selection is done with respect to one model performance characteristic (eg. accuracy), feature selection with secondary model performance characteristics (eg. fairness and robustness) as objectives would require going through the computationally expensive selection process from scratch. In this paper, we introduce the problem of feature reselection, so that features can be selected with respect to secondary model performance characteristics efficiently even after a feature selection process has been done with respect to a primary objective. To address this problem, we propose REFRESH, a method to reselect features so that additional constraints that are desirable towards model performance can be achieved without having to train several new models. REFRESH's underlying algorithm is a novel technique using SHAP values and correlation analysis that can approximate for the predictions of a model without having to train these models. Empirical evaluations on three datasets, including a large-scale loan defaulting dataset show that REFRESH can help find alternate models with better model characteristics efficiently. We also discuss the need for reselection and REFRESH based on regulation desiderata.

show abstract

A comprehensive evaluation framework for deep model robustness

Cited by 24 publications

References 6 publications

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

Exploring the Physical-World Adversarial Robustness of Vehicle Detection

CANARY: An Adversarial Robustness Evaluation Platform for Deep Learning Models on Image Classification

REFRESH: Responsible and Efficient Feature Reselection guided by SHAP values

Contact Info

Product

Resources

About