Generating Adversarial Examples for Holding Robustness of Source Code Processing Models

Zhang, Huangzhao; Li, Zhuo; Li, Ge; Ма, Лей; Liu, Yang; Jin, Zhi

doi:10.1609/aaai.v34i01.5469

Cited by 77 publications

(90 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since we are the irst to consider adversarial examples for code comment generation tasks, the literature is short of algorithms for direct comparison. To demonstrate the efectiveness of our approach, we adopt two algorithms as the baseline, i.e., the random substitute algorithm and the algorithm based on Metropolis-Hastings sampling [49].…”

Section: Evaluation 41 Experiments Setupmentioning

confidence: 99%

“…Metropolis-Hastings algorithm. The Metropolis-Hastings sampling based algorithm was recently used to generate adversarial examples for attacking source code classiiers [49]. Recall that the Metropolis-Hastings algorithm is a classical Markov Chain Monte Carlo sampling approach, which can generate desirable examples given the targeted stationary distribution and the transition proposal.…”

Section: Evaluation 41 Experiments Setupmentioning

confidence: 99%

“…Bielik et al [7] improved the adversarial robustness of models for the task of type inference by learning to abstain if uncertain. Zhang et al [49] studied the problem for the code classiication tasks where they proposed a sampling based method to generate adversarial examples. Note that this work is still of classiication nature whereas our work focuses on comment generation, which is of language translation or generation nature.…”

Section: Adversarial Examples Generation Adversarial Examples Were Irst Proposed By Szegedy Et Al In Image Classiicationmentioning

confidence: 99%

See 2 more Smart Citations

Adversarial Robustness of Deep Code Comment Generation

Zhou

Zhang

Shen

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Deep neural networks (DNNs) have shown remarkable performance in a variety of domains such as computer vision, speech recognition, and natural language processing. Recently they also have been applied to various software engineering tasks, typically involving processing source code. DNNs are well-known to be vulnerable to adversarial examples, i.e., fabricated inputs that could lead to various misbehaviors of the DNN model while being perceived as benign by humans. In this paper, we focus on the code comment generation task in software engineering and study the robustness issue of the DNNs when they are applied to this task. We propose ACCENT ( A dversarial C ode C omment g EN era T or), an identifier substitution approach to craft adversarial code snippets, which are syntactically correct and semantically close to the original code snippet, but may mislead the DNNs to produce completely irrelevant code comments. In order to improve the robustness, ACCENT also incorporates a novel training method, which can be applied to existing code comment generation models. We conduct comprehensive experiments to evaluate our approach by attacking the mainstream encoder-decoder architectures on two large-scale publicly available datasets. The results show that ACCENT efficiently produces stable attacks with functionality-preserving adversarial examples, and the generated examples have better transferability compared with the baselines. We also confirm, via experiments, the effectiveness in improving model robustness with our training method.

show abstract

Section: Evaluation 41 Experiments Setupmentioning

confidence: 99%

Section: Evaluation 41 Experiments Setupmentioning

confidence: 99%

Section: Adversarial Examples Generation Adversarial Examples Were Irst Proposed By Szegedy Et Al In Image Classiicationmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Robustness of Deep Code Comment Generation

Zhou

Zhang

Shen

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Each of the transformation operators above is designed to change the structure representation of the source code differently. For example, with VR, we want the NN to understand that even the change in textual information does not affect the semantic meaning of the source code, inspired by a recent finding of Zhang et al [74]. It is suggested that the source code model should be equipped with adversarial examples of token changes to make the model become more robust.…”

Section: Program Transformation Operatorsmentioning

confidence: 99%

Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations

Bui¹,

Yu²,

Jiang

2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

We propose Corder, a self-supervised contrastive learning framework for source code model. Corder is designed to alleviate the need of labeled data for code retrieval and code summarization tasks. The pre-trained model of Corder can be used in two ways: (1) it can produce vector representation of code which can be applied to code retrieval tasks that do not have labeled data; (2) it can be used in a fine-tuning process for tasks that might still require label data such as code summarization. The key innovation is that we train the source code model by asking it to recognize similar and dissimilar code snippets through a contrastive learning objective. To do so, we use a set of semantic-preserving transformation operators to generate code snippets that are syntactically diverse but semantically equivalent. Through extensive experiments, we have shown that the code models pretrained by Corder substantially outperform the other baselines for code-to-code retrieval, text-to-code retrieval, and code-to-text summarization tasks. CCS Concepts• Software and its engineering → Software libraries and repositories; • Information systems → Information retrieval;

show abstract

“…The security in deep learning has been a hot topic recently. Works [223][224][225][226][227][228][229][230] have been proposed to prevent adversarial attacks in deep learning models. On the other hand, we can use deep learning in security to help us capture the semantic of the program.…”

Section: Binary Function Semantics Capturingmentioning

confidence: 99%

Semantic driven vulnerability detection and patch analysis

Xu¹

View full text Add to dashboard Cite

First, I would like to express my very great appreciation to my Ph.D. supervisor Prof. Liu Yang. He has been always supportive to all my researches and gave me valuable suggestions on all my projects. He is passionate about researches. He always tries to find time to synchronize the project progress with me, despite that he has many other projects and teams to be supervised. We often stay up late to discusses research ideas and methodologies. I am appreciated that he can sacrifice his after-work time to join the meetings. I have learned a lot from Prof. Liu Yang, not only the way to do researches but also the positive attitude towards work and life.Second, I would like to offer my great appreciation to Prof. Chen Bihuan, Prof.Meng Guozhu and Prof. Xue Yingxing. They co-worked with me to publish several papers. They provides help when I first start my Ph.D. journey. I learned from them gradually to become a independent research, who can lead a research project. I would like to thank Dr. Xie Xiaofei, Prof. Wang Zhi and Prof. Song Fu for their help in my researches.

show abstract

Generating Adversarial Examples for Holding Robustness of Source Code Processing Models

Cited by 77 publications

References 0 publications

Adversarial Robustness of Deep Code Comment Generation

Adversarial Robustness of Deep Code Comment Generation

Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations

Semantic driven vulnerability detection and patch analysis

Contact Info

Product

Resources

About