Towards Resilient Artificial Intelligence: Survey and Research Issues

Eigner, Oliver; Eresheim, Sebastian; Kieseberg, Peter; Klausner, Lukas Daniel; Pirker, Martin; Priebe, Torsten; Tjoa, Simon; Marulli, Fiammetta; Mercaldo, Francesco

doi:10.1109/csr51186.2021.9527986

Cited by 17 publications

(12 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data from unlabeled data buffer can be moved to the labeled data queue after the feedback on their actual affiliation with the classes is received. The priority of specific samples being recommended for manual labeling depends on the value of the membership function (1).…”

Section: Methodsmentioning

confidence: 99%

“…These presuppose the existence of mechanisms of perturbation absorption, perturbation detection, graceful degradation, restoration of productivity and improvement. Research [1,2,3] studied vulnerability of artificial intelligence algorithms, identifying the following destructive effects: noise and adversarial attacks, faults and fault injection in the environment of intelligent algorithm deployment, concept drift and emergence of novelty, i.e., test examples that out of distribution of training data.…”

Section: Review Of the Literaturementioning

confidence: 99%

“…Therefore, there is a need to ensure the resilience of artificial intelligence algorithms to destructive perturbations such. In the case of artificial intelligence for image classification, specific perturbations such as adversarial attacks or noise, faults or fault injection attacks, as well as concept drift and out-of-distribution increase aleatoric and epistemic uncertainty and its involve a decrease in the productivity of the intellectual algorithm [1][2][3].…”

Section: Introductionmentioning

confidence: 99%

“…The resilience of the image classifier to perturbations is primarily ensured by achieving robustness for absorption of a certain level of destructive influences and implementing the graceful degradation mechanism to achieve the most effective behavior in conditions of incomplete certainty [1]. Data analysis models need to be continuously improved to take into account the nonstationary environment and new challenges.…”

Section: Introductionmentioning

confidence: 99%

“…Separate components of resilience to certain types of destructive influences have been researched in many scientific papers, but the complex influence of multiple destructive factors at once had still not been considered [1][2][3]. In addition, machine learning algorithms for classification analysis of images that simultaneously implement such components of resilience as robustness, graceful degradation, recovery and improvement have not yet been proposed.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Image Classifier Resilient to Adversarial Attacks, Fault Injections and Concept Drift – Model Architecture and Training Algorithm

Москаленко

Moskalenko

Korobov

et al. 2022

RIC

View full text Add to dashboard Cite

Context. The problem of image classification algorithms vulnerability to destructive perturbations has not yet been definitively resolved and is quite relevant for safety-critical applications. Therefore, object of research is the process of training and inference for image classifier that functioning under influences of destructive perturbations. The subjects of the research are model architecture and training algorithm of image classifier that provide resilience to adversarial attacks, fault injection attacks and concept drift. Objective. Stated research goal is to develop effective model architecture and training algorithm that provide resilience to adversarial attacks, fault injections and concept drift. Method. New training algorithm which combines self-knowledge distillation, information measure maximization, class distribution compactness and interclass gap maximization, data compression based on discretization of feature representation and semi-supervised learning based on consistency regularization is proposed. Results. The model architecture and training algorithm of image classifier were developed. The obtained classifier was tested on the Cifar10 dataset to evaluate its resilience over an interval of 200 mini-batches with a training and test size of mini-batch equals to 128 examples for such perturbations: adversarial black-box L∞-attacks with perturbation levels equal to 1, 3, 5 and 10; inversion of one randomly selected bit in a tensor for 10%, 30%, 50% and 60% randomly selected tensors; addition of one new class; real concept drift between a pair of classes. The effect of the feature space dimensionality on the value of the information criterion of the model performance without perturbations and the value of the integral metric of resilience during the exposure to perturbations is considered. Conclusions. The proposed model architecture and learning algorithm provide absorption of part of the disturbing influence, graceful degradation due to hierarchical classes and adaptive computation, and fast adaptation on a limited amount of labeled data. It is shown that adaptive computation saves up to 40% of resources due to early decision-making in the lower sections of the model, but perturbing influence leads to slowing down, which can be considered as graceful degradation. A multi-section structure trained using knowledge self-distillation principles has been shown to provide more than 5% improvement in the value of the integral mectric of resilience compared to an architecture where the decision is made on the last layer of the model. It is observed that the dimensionality of the feature space noticeably affects the resilience to adversarial attacks and can be chosen as a tradeoff between resilience to perturbations and efficiency without perturbations.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Review Of the Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Image Classifier Resilient to Adversarial Attacks, Fault Injections and Concept Drift – Model Architecture and Training Algorithm

Москаленко

Moskalenko

Korobov

et al. 2022

RIC

View full text Add to dashboard Cite

show abstract

Sustainability Effects of Robust and Resilient Artificial Intelligence

Priebe,

Kieseberg,

Adrowitzer

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

It is commonly understood that the resilience of critical information technology (IT) systems based on artificial intelligence (AI) must be ensured. In this regard, we consider resilience both in terms of IT security threats, such as cyberattacks, as well as the ability to robustly persist under uncertain and changing environmental conditions, such as climate change or economic crises. This paper explores the relationship between resilience and sustainability with regard to AI systems, develops fields of action for resilient AI, and elaborates direct and indirect influences on the achievement of the United Nations Sustainable Development Goals. Indirect in this case means that a sustainability effect is reached by taking resilience measures when applying AI in a sustainability-relevant application area, for example precision agriculture or smart health.

show abstract