An Algorithm for Generating Invisible Data Poisoning Using Adversarial Noise That Breaks Image Classification Deep Learning

Chan-Hon-Tong, Adrien

doi:10.3390/make1010011

Cited by 9 publications

(19 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Determine robustness: A major risk occurs if ML applications are not robust (in terms of not being able to perform appropriately), if data is perturbed, e.g., noisy or wrong, or manipulated in advance to fool the system (e.g., adversarial attacks) as shown by Chang [131]. This requires methods to statistically estimate the model's local and global robustness.…”

Section: Discussionmentioning

confidence: 99%

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Studer

Bui

Drescher

et al. 2021

MAKE

130

View full text Add to dashboard Cite

Machine learning is an established and frequently used technique in industry and academia, but a standard process model to improve success and efficiency of machine learning applications is still missing. Project organizations and machine learning practitioners face manifold challenges and risks when developing machine learning applications and have a need for guidance to meet business expectations. This paper therefore proposes a process model for the development of machine learning applications, covering six phases from defining the scope to maintaining the deployed machine learning application. Business and data understanding are executed simultaneously in the first phase, as both have considerable impact on the feasibility of the project. The next phases are comprised of data preparation, modeling, evaluation, and deployment. Special focus is applied to the last phase, as a model running in changing real-time environments requires close monitoring and maintenance to reduce the risk of performance degradation over time. With each task of the process, this work proposes quality assurance methodology that is suitable to address challenges in machine learning development that are identified in the form of risks. The methodology is drawn from practical experience and scientific literature, and has proven to be general and stable. The process model expands on CRISP-DM, a data mining process model that enjoys strong industry support, but fails to address machine learning specific tasks. The presented work proposes an industry- and application-neutral process model tailored for machine learning applications with a focus on technical tasks for quality assurance.

show abstract

Section: Discussionmentioning

confidence: 99%

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Studer

Bui

Drescher

et al. 2021

MAKE

130

View full text Add to dashboard Cite

show abstract

“…Determine robustness: A major risk occurs if ML applications are not robust to perturbed, e.g. noisy or wrong, or even designed adversarial input data as show by Chan-Hon-Tong [111]. This requires methods to statistically estimate the model's local and global robustness.…”

Section: Discussionmentioning

confidence: 99%

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Studer¹,

Bui²,

Drescher³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning is an established and frequently used technique in industry and academia but a standard process model to improve success and efficiency of machine learning applications is still missing. Project organizations and machine learning practitioners have a need for guidance throughout the life cycle of a machine learning application to meet business expectations. We therefore propose a process model for the development of machine learning applications, that covers six phases from defining the scope to maintaining the deployed machine learning application. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications, as the risk of model degradation in a changing environment is eminent. With each task of the process, we propose quality assurance methodology that is suitable to address challenges in machine learning development that we identify in form of risks. The methodology is drawn from practical experience and scientific literature and has proven to be general and stable. The process model expands on CRISP-DM, a data mining process model that enjoys strong industry support but lacks to address machine learning specific tasks. Our work proposes an industry and application neutral process model tailored for machine learning applications with focus on technical tasks for quality assurance.

show abstract

“…A smaller but non negligible issue is poisoning [17], [18]. Data poisoning (which also works [19] on support vector machine SVM [20]) is known as the goal of finding small modification of training data (testing data being unchanged) changing the model behaviour on test e.g.…”

Section: B Poisoningmentioning

confidence: 99%

“…δ is constrained by a L 1 norm, and, goal of the hacker is to take advantage of sensibility to small perturbation: goal is to produce symmetric poisoning with only small modification of all training data instead of heavy modification of few training samples. Typically [18] introduces a symmetric adversarial poisoning attack (SAP) based on energetic landscape hacking.…”

Section: Adversarial Poisoningmentioning

confidence: 99%

“…Precisely, [18] presents a SAP attacks on classical computer vision benchmarks targeting a frozen DL + SVM pipeline (f is a deep network but only last layer weights are updated during training resulting in a pipeline sensible to small perturbation but with convex training). This attack called energetic level attack is based on the assumption that the more CE(f, w, T rain) is high/less, the less/high is the probability that w will be returned by SGD when trained on Data.…”

Section: Adversarial Poisoningmentioning

confidence: 99%

See 1 more Smart Citation

Symmetric adversarial poisoning against deep learning

Chan-Hon-Tong¹

2020

2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA)

Self Cite

View full text Add to dashboard Cite

Data poisoning is known as the goal of finding small modification of training data which make them not suitable anymore for training the targeted model. Recently, an efficient symmetric poisoning attack targeting frozen deep features plus support vector machine has been found.However, new experiments presented in this paper shows that this attack is not symmetric anymore on unfrozen/real deep networks.Then, several extensions of this attack are considered on CIFAR10/CIFAR100 with both VGG and ResNet backbone leading to a symmetric attack. On VGG/CIFAR10 setting, this extended attack makes performances moving by -60%,+5% from native accuracy using perturbations invisible to human eyes.A discussion on this success and on the failure of the 3 other methods is also presented.

show abstract

An Algorithm for Generating Invisible Data Poisoning Using Adversarial Noise That Breaks Image Classification Deep Learning

Cited by 9 publications

References 19 publications

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Symmetric adversarial poisoning against deep learning

Contact Info

Product

Resources

About