The death of the short-form physics essay in the coming AI revolution

Abstract

The latest AI language modules can produce original, high quality full short-form (300-word) Physics essays within seconds. These technologies such as ChatGPT and davinci-003 are freely available to anyone with an internet connection. In this work, we present evidence of AI generated short-form essays achieving First-Class grades on an essay writing assessment from an accredited, current university Physics module. The assessment requires students answer five open-ended questions with a short, 300-word essay each. Fifty AI answers were generated to create ten submissions that were independently marked by five separate markers. The AI generated submissions achieved an average mark of 71 ± 2 % , in strong agreement with the current module average of 71 ± 5 % . A typical AI submission would therefore most-likely be awarded a First Class, the highest classification available at UK universities. Plagiarism detection software returned a plagiarism score between 2 ± 1 % (Grammarly) and 7 ± 2 % (TurnitIn). We argue that these results indicate that current natural language processing AI represent a significant threat to the fidelity of short-form essays as an assessment method in Physics courses.

Comparing the Performance of College Chemistry Students with ChatGPT for Calculations Involving Acids and Bases

“…Chatbots, like ChatGPT, are an artificial intelligence resource that have diverse potential uses by instructors and students, such as being used to create assignments or to provide the learner with feedback and explanations. , ChatGPT’s functionality as an automated feedback system parallels that of other systems but with superior natural language processing (NLP) ability. − Nascent research into its use in the chemistry ,− and physics − education research communities have examined its ability to answer questions, produce explanations, and contribute to written work like lab reports.…”

Physics language and language use in physics—What do we know and how AI might enhance language-related research and instruction

“…ChatGPT was then found to excel in short-form physics essays, i.e. perform on par with students [168]. Instruction with ChatGPT could increase students' perception of ChatGPT, and it is expected that ChatGPT can play an important role in instruction [30,33].…”

ChatGTP: What is it and how can nursing and health science education use it?

“…Contrary to some literature emerging on this topic that stipulates plagiarism detection software is ineffective in detecting ChatGPT generated content (e.g., Gleason, 2022), ChatGPT itself indicates that software exists that is sensitive to ChatGPT generated text (OpenAI, 2023). A team of investigators in the United Kingdom investigated, running student short form essay responses through the plagiarism software Grammarly and TurnitIn with scores of 2% ± 1% and 7% ± 2%, respectively (Yeadon et al, 2022). Based on these scores and the favourable grading that these essays received, these authors concluded that ChatGPT is a major threat to fidelity in their academic context.…”