Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

Fröhling, Leon; Zubiaga, Arkaitz

doi:10.7717/peerj-cs.443

Cited by 54 publications

(33 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research has shown that Chat-GPT expresses significantly less negative emotion and hate speech compared to human-authored texts. Stylistic features or stylometry, including repetitiveness, lack of purpose, and readability, are also known to harbor valuable signals for detecting LLM-generated texts [13]. In addition to analyzing single texts, numerous linguistic patterns can be found in multi-turn conversations [3].…”

Section: Statistical Disparitiesmentioning

confidence: 99%

“…Some of the commonly used algorithms are Support Vector Machines, Naive Bayes, and Decision Trees. For instance, Fröhling et al utilized linear regression, SVM, and random forests models built on statistical and linguistic features to successfully identify texts generated by GPT-2, GPT-3, and Grover models [13]. Similarly, Solaiman et al achieved solid performance in identifying texts generated by GPT-2 through a combination of TF-IDF unigram and bigram features with a logistic regression model [35].…”

Section: Traditional Classification Algorithms Traditional Classifica...mentioning

confidence: 99%

See 1 more Smart Citation

The Science of Detecting LLM-Generated Texts

Tang¹,

Yu-Neng²,

Hu³

2023

Preprint

View full text Add to dashboard Cite

The emergence of large language models (LLMs) has resulted in the production of LLM-generated texts that is highly sophisticated and almost indistinguishable from texts written by humans. However, this has also sparked concerns about the potential misuse of such texts, such as spreading misinformation and causing disruptions in the education system. Although many detection approaches have been proposed, a comprehensive understanding of the achievements and challenges is still lacking. This survey aims to provide an overview of existing LLM-generated text detection techniques and enhance the control and regulation of language generation models. Furthermore, we emphasize crucial considerations for future research, including the development of comprehensive evaluation metrics and the threat posed by open-source LLMs, to drive progress in the area of LLMgenerated text detection.

show abstract

Section: Statistical Disparitiesmentioning

confidence: 99%

Section: Traditional Classification Algorithms Traditional Classifica...mentioning

confidence: 99%

The Science of Detecting LLM-Generated Texts

Tang¹,

Yu-Neng²,

Hu³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…ChatGPT and other language generation models based on deep learning techniques, such as GPT-3, can be used for various natural language processing tasks, including medical writing. However, it is essential to note that using AI-generated text in the medical field requires careful consideration and review by medical experts to ensure the accuracy and reliability of the generated text [ 13 ].…”

Section: Editorialmentioning

confidence: 99%

Chat GPT and Artificial Intelligence in Medical Writing: Concerns and Ethical Considerations

Doyal¹,

Sender²,

Nanda³

et al. 2023

Cureus

View full text Add to dashboard Cite

Artificial intelligence (AI) language generation models, such as ChatGPT, have the potential to revolutionize the field of medical writing and other natural language processing (NLP) tasks. It is crucial to consider the ethical concerns that come with their use. These include bias, misinformation, privacy, lack of transparency, job displacement, stifling creativity, plagiarism, authorship, and dependence. Therefore, it is essential to develop strategies to understand and address these concerns. Important techniques include common bias and misinformation detection, ensuring privacy, providing transparency, and being mindful of the impact on employment. The AI-generated text must be critically reviewed by medical experts to validate the output generated by these models before being used in any clinical or medical context. By considering these ethical concerns and taking appropriate measures, we can ensure that the benefits of these powerful tools are maximized while minimizing any potential harm. This article focuses on the implications of AI assistants in medical writing and hopes to provide insight into the perceived rapid rate of technological progression from a historical and ethical perspective.

show abstract

“…Furthermore, as this approach was only applied to text produced by a particular version of Google Translate, these methods have been untested against current state-of-art text generation networks. Recent work has used a feature-based approach to detection and characterization of GPT-2, GPT-3 and Grover datasets using a variety of text features, but intentionally avoids modern neural language models in the analysis, and does not consider adversarial robustness [20].…”

Section: B Detection Of Computer-generated Textmentioning

confidence: 99%

Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers

Crothers¹,

Japkowicz²,

Viktor³

et al. 2022

Preprint

View full text Add to dashboard Cite

The detection of computer-generated text is an area of rapidly increasing significance as nascent generative models allow for efficient creation of compelling human-like text, which may be abused for the purposes of spam, disinformation, phishing, or online influence campaigns. Past work has studied detection of current state-of-the-art models, but despite a developing threat landscape, there has been minimal analysis of the robustness of detection methods to adversarial attacks. To this end, we evaluate neural and non-neural approaches on their ability to detect computer-generated text, their robustness against text adversarial attacks, and the impact that successful adversarial attacks have on human judgement of text quality. We find that while statistical features underperform neural features, statistical features provide additional adversarial robustness that can be leveraged in ensemble detection models. In the process, we find that previously effective complex phrasal features for detection of computer-generated text hold little predictive power against contemporary generative models, and identify promising statistical features to use instead. Finally, we pioneer the usage of ∆MAUVE as a proxy measure for human judgement of adversarial text quality.

show abstract

Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

Cited by 54 publications

References 23 publications

The Science of Detecting LLM-Generated Texts

The Science of Detecting LLM-Generated Texts

Chat GPT and Artificial Intelligence in Medical Writing: Concerns and Ethical Considerations

Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers

Contact Info

Product

Resources

About