Identification of ATP2C1 mutations in the patients of Hailey-Hailey disease

li, xiaoli; zhang, dingwei; ding, jiahui; li, li; wang, zhenghui

doi:10.21203/rs.3.rs-16069/v4

Cited by 5 publications

(8 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In light of emergence [50], larger models are however often preferable. We are already able to extract 73% and 47% of the text and code samples, even larger models like CodeX [14] or Starcoder [34] might memorise even more data.…”

Section: Discussionmentioning

confidence: 99%

“…This risk extends to pre-trained models, as some pre-training corpora, including the Pile [22], also contain code licenced under non-permissive licences [2]. The risk can be avoided by training models with code licenced under permissive licences (such as BSD-3 or MIT) or providing provenance information to trace the code back to its source so that the user of the output can abide by the original licence [23,34].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Traces of Memorisation in Large Language Models for Code

Al-Kaswan,

Izadi,

van Deursen

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

Large language models have gained significant popularity because of their ability to generate human-like text and potential applications in various fields, such as Software Engineering. Large language models for code are commonly trained on large unsanitised corpora of source code scraped from the internet. The content of these datasets is memorised and can be extracted by attackers with data extraction attacks. In this work, we explore memorisation in large language models for code and compare the rate of memorisation with large language models trained on natural language. We adopt an existing benchmark for natural language and construct a benchmark for code by identifying samples that are vulnerable to attack. We run both benchmarks against a variety of models, and perform a data extraction attack. We find that large language models for code are vulnerable to data extraction attacks, like their natural language counterparts. From the training data that was identified to be potentially extractable we were able to extract 47% from a CodeGen-Mono-16B code completion model. We also observe that models memorise more, as their parameter count grows, and that their pre-training data are also vulnerable to attack. We also find that data carriers are memorised at a higher rate than regular code or documentation and that different model architectures memorise different samples. Data leakage has severe outcomes, so we urge the research community to further investigate the extent of this phenomenon using a wider range of models and extraction techniques in order to build safeguards to mitigate this issue. CCS CONCEPTS• Security and privacy; • Software and its engineering; • Computing methodologies → Machine learning;

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Traces of Memorisation in Large Language Models for Code

Al-Kaswan,

Izadi,

van Deursen

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…To address IRQ1, we evaluate GPT-2 on a simplified task, next token prediction, a widely used task for pre-training LCMs [18,22,37]. In this task, our objective is to find the minimum number of layers in the GPT-2 model necessary to achieve accurate predictions for the next token in code completion.…”

Section: Irq 1: Number Of Indispensable Layersmentioning

confidence: 99%

“…Limited by our devices, we only conduct experiments on two LCMs: GPT-2 and CodeGen, both containing hundreds of millions of parameters. Nevertheless, their architectures serve as prevalent templates for current LCMs, including newer models like StarCoder [18], which achieved state-of-the-art performance by directly adopting the GPT-2 architecture with increased parameters and training samples. Remarkably, while the model size is expanding, the fundamental issue of computation waste and unhelpful completions caused by the fixed inference process remains unaddressed.…”

Section: Threats To Validitymentioning

confidence: 99%

When Neural Code Completion Models Size up the Situation: Attaining Cheaper and Faster Completion through Dynamic Model Inference

Sun,

Du,

Song

et al. 2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

Self Cite

View full text Add to dashboard Cite

Leveraging recent advancements in large language models, modern neural code completion models have demonstrated the capability to generate highly accurate code suggestions. However, their massive size poses challenges in terms of computational costs and environmental impact, hindering their widespread adoption in practical scenarios. Dynamic inference emerges as a promising solution, as it allocates minimal computation during inference while maintaining the model's performance. In this research, we explore dynamic inference within the context of code completion. Initially, we conducted an empirical investigation on GPT-2, focusing on the inference capabilities of intermediate layers for code completion. We found that 54.4% of tokens can be accurately generated using just the first layer, signifying significant computational savings potential. Moreover, despite using all layers, the model still fails to predict 14.5% of tokens correctly, and the subsequent completions continued from them are rarely considered helpful, with only a 4.2% Acceptance Rate. These findings motivate our exploration of dynamic inference in code completion and inspire us to enhance it with a decisionmaking mechanism that stops the generation of incorrect code. We thus propose a novel dynamic inference method specifically tailored for code completion models. This method aims not only to produce correct predictions with largely reduced computation but also to prevent incorrect predictions proactively. Our extensive evaluation * Both authors contributed equally to this research.

show abstract

“…Many studies on these two diseases have revealed their close relationship, such as hyperbilirubinemia is observed in erythropoietic porphyrias [32]. For coronary artery disease and familial benign chronic pemphigus, it has been detected that mutations in exons of ATP2C1 gene in the patients of familial benign chronic pemphigus [33], Nassa et al found that ATP2C1 may induce coronary artery disease [34].…”

Section: Effectivenessmentioning

confidence: 99%

A multi-network integration approach for measuring disease similarity based on ncRNA regulation and heterogeneous information

Zhang

Zang

2022

BMC Bioinformatics

View full text Add to dashboard Cite

Background Measuring similarity between complex diseases has significant implications for revealing the pathogenesis of diseases and development in the domain of biomedicine. It has been consentaneous that functional associations between disease-related genes and semantic associations can be applied to calculate disease similarity. Currently, more and more studies have demonstrated the profound involvement of non-coding RNA in the regulation of genome organization and gene expression. Thus, taking ncRNA into account can be useful in measuring disease similarities. However, existing methods ignore the regulation functions of ncRNA in biological process. In this study, we proposed a novel deep-learning method to deduce disease similarity. Results In this article, we proposed a novel method, ImpAESim, a framework integrating multiple networks embedding to learn compact feature representations and disease similarity calculation. We first utilize three different disease-related information networks to build up a heterogeneous network, after a network diffusion process, RWR, a compact feature learning model composed of classic Auto Encoder (AE) and improved AE model is proposed to extract constraints and low-dimensional feature representations. We finally obtain an accurate and low-dimensional feature representation of diseases, then we employed the cosine distance as the measurement of disease similarity. Conclusion ImpAESim focuses on extracting a low-dimensional vector representation of features based on ncRNA regulation, and gene–gene interaction network. Our method can significantly reduce the calculation bias resulted from the sparse disease associations which are derived from semantic associations.

show abstract

Identification of ATP2C1 mutations in the patients of Hailey-Hailey disease

Cited by 5 publications

References 1 publication

Traces of Memorisation in Large Language Models for Code

Traces of Memorisation in Large Language Models for Code

When Neural Code Completion Models Size up the Situation: Attaining Cheaper and Faster Completion through Dynamic Model Inference

A multi-network integration approach for measuring disease similarity based on ncRNA regulation and heterogeneous information

Contact Info

Product

Resources

About