Identification of ATP2C1 mutations in the patients of Hailey-Hailey disease

li, xiaoli; zhang, dingwei; ding, jiahui; li, li; wang, zhenghui

doi:10.21203/rs.3.rs-16069/v6

Cited by 1 publication

(2 citation statements)

References 22 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In light of emergence [50], larger models are however often preferable. We are already able to extract 73% and 47% of the text and code samples, even larger models like CodeX [14] or Starcoder [34] might memorise even more data.…”

Section: Discussionmentioning

confidence: 99%

“…This risk extends to pre-trained models, as some pre-training corpora, including the Pile [22], also contain code licenced under non-permissive licences [2]. The risk can be avoided by training models with code licenced under permissive licences (such as BSD-3 or MIT) or providing provenance information to trace the code back to its source so that the user of the output can abide by the original licence [23,34].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Traces of Memorisation in Large Language Models for Code

Al-Kaswan,

Izadi,

van Deursen

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

Large language models have gained significant popularity because of their ability to generate human-like text and potential applications in various fields, such as Software Engineering. Large language models for code are commonly trained on large unsanitised corpora of source code scraped from the internet. The content of these datasets is memorised and can be extracted by attackers with data extraction attacks. In this work, we explore memorisation in large language models for code and compare the rate of memorisation with large language models trained on natural language. We adopt an existing benchmark for natural language and construct a benchmark for code by identifying samples that are vulnerable to attack. We run both benchmarks against a variety of models, and perform a data extraction attack. We find that large language models for code are vulnerable to data extraction attacks, like their natural language counterparts. From the training data that was identified to be potentially extractable we were able to extract 47% from a CodeGen-Mono-16B code completion model. We also observe that models memorise more, as their parameter count grows, and that their pre-training data are also vulnerable to attack. We also find that data carriers are memorised at a higher rate than regular code or documentation and that different model architectures memorise different samples. Data leakage has severe outcomes, so we urge the research community to further investigate the extent of this phenomenon using a wider range of models and extraction techniques in order to build safeguards to mitigate this issue. CCS CONCEPTS• Security and privacy; • Software and its engineering; • Computing methodologies → Machine learning;

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%