Natural Attack for Pre-trained Models of Code

Abstract

Pre-trained models of code have achieved success in many important software engineering tasks. However, these powerful models are vulnerable to adversarial attacks that slightly perturb model inputs to make a victim model produce wrong outputs. Current works mainly attack models of code with examples that preserve operational program semantics but ignore a fundamental requirement for adversarial example generation: perturbations should be natural to human judges, which we refer to as naturalness requirement.In this paper, we propose ALERT (Naturalness Aware Attack), a black-box attack that adversarially transforms inputs to make victim models produce wrong outputs. Different from prior works, this paper considers the natural semantic of generated examples at the same time as preserving the operational semantic of original inputs. Our user study demonstrates that human developers consistently consider that adversarial examples generated by ALERT are more natural than those generated by the state-of-the-art work by Zhang et al. that ignores the naturalness requirement. On attacking CodeBERT, our approach can achieve attack success rates of 53.62%, 27.79%, and 35.78% across three downstream tasks: vulnerability prediction, clone detection and code authorship attribution. On GraphCodeBERT, our approach can achieve average success rates of 76.95%, 7.96% and 61.47% on the three tasks. The above outperforms the baseline by 14.07% and 18.56% on the two pretrained models on average. Finally, we investigated the value of the generated adversarial examples to harden victim models through an adversarial fine-tuning procedure and demonstrated the accuracy of CodeBERT and GraphCodeBERT against ALERT-generated adversarial examples increased by 87.59% and 92.32%, respectively. CCS CONCEPTS• Software and its engineering → Software testing and debugging; Search-based software engineering; • Computing methodologies → Neural networks.

A Retrospective on ICSE 2022

“…There was also quite a bit of research centered on leveraging variables and their names to develop effective models of code. Yang et al focused on adversarially training models of code by changing variable names [37]. Variable names are changed by greedily selecting more tokens that were more influential for correct predictions.…”

ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning

“…However, it is challenging to improve the robustness of pretrained models. Although the latest work by Yang et al [40] proposed some attack strategies to make CodeBERT and GraphCodeBERT have poor performance on adversarial samples. They further combined adversarial samples with original samples to fine-tune pre-trained models without any changes to the model architecture to improve prediction robustness on downstream tasks.…”

Learning program semantics via exploring program structures with deep learning

“…The research about adversarial robustness analysis on the models of code has attracted the attention [218,[243][244][245][246][247]. Generally, these works can be categorized into two groups:…”

“…However, it is challenging to improve the robustness of pre-trained models. Although the latest work by Yang et al [218] proposed some attack strategies to make CodeBERT and…”

“…In terms of the black-box works, HMH[247] generated adversarial examples of the source code by conducting iterative identifier renaming and evaluated on source code functionality classification task. The latest work by Yang et al[218] proposed ALERT to transform the inputs while preserving the optional semantics of original inputs by replacing the variables with the substitutes. Their experiments are conducted on the pre-trained models CodeBERT and GraphCodeBERT.…”