Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation

Karmakar, Anjan; Prenner, Julian Aron; D’Ambros, Marco; Robbes, Romain

doi:10.48550/arxiv.2212.02684

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Manuscript received --/revised --1. https://en.wikipedia.org/wiki/Clever Hans [17] in their input? Are source code models "stochastic parrots" [18], that overly rely on memorization [19]? There are many such questions about source code models that can be asked, all of which impact our understanding of source code models and their capabilities.…”

Section: Introductionmentioning

confidence: 99%

What do pre-trained code models know about code?

Karmakar

Robbes

2021

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

Pre-trained models of source code have recently been successfully applied to a wide variety of Software Engineering tasks; they have also seen some practical adoption in practice, e.g. for code completion. Yet, we still know very little about what these pre-trained models learn about source code. In this article, we use probing-simple diagnostic tasks that do not further train the models-to discover to what extent pre-trained models learn about specific aspects of source code. We use an extensible framework to define 15 probing tasks that exercise surface, syntactic, structural and semantic characteristics of source code. We probe 8 pre-trained source code models, as well as a natural language model (BERT) as our baseline. We find that models that incorporate some structural information (such as GraphCodeBERT) have a better representation of source code characteristics. Surprisingly, we find that for some probing tasks, BERT is competitive with the source code models, indicating that there are ample opportunities to improve source-code specific pre-training on the respective code characteristics. We encourage other researchers to evaluate their models with our probing task suite, so that they may peer into the hidden layers of the models and identify what intrinsic code characteristics are encoded.

show abstract

Section: Introductionmentioning

confidence: 99%

What do pre-trained code models know about code?

Karmakar

Robbes

2021

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

show abstract

“…Additionally, memorisation can reduce the ability of the model to adapt its output to specific use cases. For example, when slightly changing HackerRank problems, CodeX [14] struggles to produce a correct solution, instead regurgitating solutions for the original problem [32,47].…”

Section: Background and Related Work 21 Memorisationmentioning

confidence: 99%

“…Besides the privacy concerns explained in section 1, memorisation also causes an overestimation of performance. It has, for instance, been observed that CodeX can complete HackerRank problems without receiving the full task description [32].…”

Section: Background and Related Work 21 Memorisationmentioning

confidence: 99%

See 1 more Smart Citation

Traces of Memorisation in Large Language Models for Code

Al-Kaswan,

Izadi,

van Deursen

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

Large language models have gained significant popularity because of their ability to generate human-like text and potential applications in various fields, such as Software Engineering. Large language models for code are commonly trained on large unsanitised corpora of source code scraped from the internet. The content of these datasets is memorised and can be extracted by attackers with data extraction attacks. In this work, we explore memorisation in large language models for code and compare the rate of memorisation with large language models trained on natural language. We adopt an existing benchmark for natural language and construct a benchmark for code by identifying samples that are vulnerable to attack. We run both benchmarks against a variety of models, and perform a data extraction attack. We find that large language models for code are vulnerable to data extraction attacks, like their natural language counterparts. From the training data that was identified to be potentially extractable we were able to extract 47% from a CodeGen-Mono-16B code completion model. We also observe that models memorise more, as their parameter count grows, and that their pre-training data are also vulnerable to attack. We also find that data carriers are memorised at a higher rate than regular code or documentation and that different model architectures memorise different samples. Data leakage has severe outcomes, so we urge the research community to further investigate the extent of this phenomenon using a wider range of models and extraction techniques in order to build safeguards to mitigate this issue. CCS CONCEPTS• Security and privacy; • Software and its engineering; • Computing methodologies → Machine learning;

show abstract