GPTZero Performance in Identifying Artificial Intelligence-Generated Medical Texts: A Preliminary Study

Habibzadeh, Farrokh

doi:10.3346/jkms.2023.38.e319

Cited by 19 publications

(7 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These scholars have done so in varying degrees and by focusing on different types of AI detection tools. The detection tools explored include single detection tools (Habibzadeh, 2023;Perkins et al, 2023;Subramaniam, 2023); two detection tools (Bisi et al, 2023;Desaire et al, 2023;Ibrahim, 2023); three detection tools (Cingillioglu, 2023;Elali & Rachid, 2023;Gao et al, 2023;Homolak, 2023;Ladha et al, 2023;Wee & Reimer, 2023); four detection tools (Abani et al, 2023;Alexander et al, 2023;Anil et al, 2023); and multiple detection tools (Chaka, 2023c;Odri & Yoon, 2023;Santra & Majhi, 2023;Walters, 2023). But more scholarly papers published in this area are preprints, which, at the moment, tend to outnumber journal articles and book chapters.…”

Section: Introductionmentioning

confidence: 99%

Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review

2024

JALT

View full text Add to dashboard Cite

The purpose of this study was to review 17 articles published between January 2023 and November 2023 that dealt with the performance of AI detectors in differentiating between AI-generated and humanwritten texts. Employing a slightly modified version of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol and an aggregated set of quality evaluation criteria adapted from A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool, the study was conducted from 1 October 2023 to 30 November 2023 and guided by six research questions. The study conducted its searches on eleven online databases, two Internet search engines, and one academic social networking site. The geolocation and authorship of the 17 reviewed articles were spread across twelve countries in both the Global North and the Global South. ChatGPT (in its two versions, GPT-3.5 and GPT-4) was the sole AI text generator used or was one of the AI text generators in instances where more than one AI text generator had been used. Crossplag was the top-performing AI detection tool, followed by Copyleaks. Duplichecker and Writer were the worst-performing AI detection tools in instances in which they had been used. One of the major aspects flagged by the main findings of the 17 reviewed articles is the inconsistency of the detection efficacy of all the tested AI detectors and all the tested anti-plagiarism detection tools. Both sets of detection tools were found to lack detection reliability. As a result, this study recommends utilising both contemporary AI detectors and traditional anti-plagiarism detection tools, together with human reviewers/raters, in an ongoing search for differentiating between AI-generated and humanwritten texts.

show abstract

Section: Introductionmentioning

confidence: 99%

Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review

2024

JALT

View full text Add to dashboard Cite

show abstract

“…Since the integration of ChatGPT in education, academicians have shared their mixed predictions about the capabilities and potential vulnerabilities due to its seamless ability to write and summarise articles (Li, 2023), answer complex questions, translate language (Xiao & Zhi, 2023) and solve complex mathematical problems (Wardat et al, 2023). Consequently, this immediately raises concerns related to academic fraud, resulting in numerous articles and tools like GPTZero (Habibzadeh, 2023) advocating whether to use ChatGPT in academia.…”

Section: Discussionmentioning

confidence: 99%

Emerging Dynamics of ChatGPT in Academia: A Scoping Review

Mahrishi,

Abbas,

Radovanović

2024

JUTLP

View full text Add to dashboard Cite

Large Language Models (LLMs) and Generative AI tools are revolutionising every aspect of academia, including medical, physical, and STEM education. They have also proved their mettle in blended learning systems and distance learning programs by improving educational sustainability, accessibility, and engagement. Through this scoping review, we aim to provide an essential overview of the state of the art of ChatGPT from the standpoint of exploring its development, analysing its current trajectory, and emerging dynamics in active research responsible for defining regulations and protocols. The PRISMA benchmarking was used on the Scopus dataset, with 109 papers from 2022 to 2023. Interactive results and bibliographic maps are generated using the Bibliometrix library of R-Studio. The findings are aligned with the research questions and represent exceptional growth in scientific production. Furthermore, relevant avenues for research publications, leading countries, and institutions in the area are also listed. The thematic and trends analysis anticipated that artificial intelligence and generative AI will substantially influence nearly every dimension in the coming years. The review also identified the possible advantages and disadvantages of adopting ChatGPT in higher education and analysed its deployment, considering ethical issues. The research concludes that, apart from ChatGPT, other large language models are also transforming artificial intelligence in education. However, ethical concerns and implications in education highlight vital issues for further research to ensure AI's responsible and ethical use.

show abstract

“…Detection is performed based on the idea that humans tend to mix long and short sentences, while AI-produced sentences are more uniform. Recent research [51] presents encouraging findings in the identification of AI-generated texts using GPTZero. The study reveals an accuracy of 0.80 with a 95% confidence interval, a specificity of 0.90, and a sensitivity of 0.65.…”

Section: Detection Mechanismsmentioning

confidence: 99%

Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries

Nitu,

Dascalu

2024

Future Internet

View full text Add to dashboard Cite

Machine-generated content reshapes the landscape of digital information; hence, ensuring the authenticity of texts within digital libraries has become a paramount concern. This work introduces a corpus of approximately 60 k Romanian documents, including human-written samples as well as generated texts using six distinct Large Language Models (LLMs) and three different generation methods. Our robust experimental dataset covers five domains, namely books, news, legal, medical, and scientific publications. The exploratory text analysis revealed differences between human-authored and artificially generated texts, exposing the intricacies of lexical diversity and textual complexity. Since Romanian is a less-resourced language requiring dedicated detectors on which out-of-the-box solutions do not work, this paper introduces two techniques for discerning machine-generated texts. The first method leverages a Transformer-based model to categorize texts as human or machine-generated, while the second method extracts and examines linguistic features, such as identifying the top textual complexity indices via Kruskal–Wallis mean rank and computes burstiness, which are further fed into a machine-learning model leveraging an extreme gradient-boosting decision tree. The methods show competitive performance, with the first technique’s results outperforming the second one in two out of five domains, reaching an F1 score of 0.96. Our study also includes a text similarity analysis between human-authored and artificially generated texts, coupled with a SHAP analysis to understand which linguistic features contribute more to the classifier’s decision.

show abstract

GPTZero Performance in Identifying Artificial Intelligence-Generated Medical Texts: A Preliminary Study

Cited by 19 publications

References 37 publications

Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review

Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review

Emerging Dynamics of ChatGPT in Academia: A Scoping Review

Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries

Contact Info

Product

Resources

About