Code Difference Guided Adversarial Example Generation for Deep Code Models

Tian, Zhao; Chen, Junjie; Jin, Zhi

doi:10.1109/ase56229.2023.00149

Cited by 6 publications

(1 citation statement)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 presents the datasets and models we used in the study. We follow the previous works [21], [22], [23] and consider five code tasks and one dataset per task including a multiclass classification task (problem classification), two binary classification tasks (clone detection and vulnerability detection), and two non-classification tasks (code summarization and code translation).…”

Section: Dataset and Modelmentioning

confidence: 99%

Active Code Learning: Benchmarking Sample-Efficient Training of Code Models

Hu,

Guo,

Xie

et al. 2024

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

The costly human effort required to prepare the training data of machine learning (ML) models hinders their practical development and usage in software engineering (ML4Code), especially for those with limited budgets. Therefore, efficiently training models of code with less human effort has become an emergent problem. Active learning is such a technique to address this issue that allows developers to train a model with reduced data while producing models with desired performance, which has been well studied in computer vision and natural language processing domains. Unfortunately, there is no such work that explores the effectiveness of active learning for code models. In this paper, we bridge this gap by building the first benchmark to study this critical problem -active code learning. Specifically, we collect 11 acquisition functions (which are used for data selection in active learning) from existing works and adapt them for code-related tasks. Then, we conduct an empirical study to check whether these acquisition functions maintain performance for code data. The results demonstrate that feature selection highly affects active learning and using output vectors to select data is the best choice. For the code summarization task, active code learning is ineffective which produces models with over a 29.64% gap compared to the expected performance. Furthermore, we explore future directions of active code learning with an exploratory study. We propose to replace distance calculation methods with evaluation metrics and find a correlation between these evaluation-based distance methods and the performance of code models.

show abstract

Section: Dataset and Modelmentioning

confidence: 99%