Examining ChatGPT’s validity as a source for scientific inquiry and its misconceptions regarding cell energy metabolism

Abstract

Many people use technological tools that are widely accessible, respond quickly, and have extensive information networks today. Due to recent technological advances in education and the increasing acceptance of Artificial Intelligence (AI) technologies, the issues regarding their implementation in education require identification and analysis. ChatGPT (Chat Generative Pre-trained Transformer), an artificial intelligence program that emerged in 2022, contains notable characteristics. OpenAI created ChatGPT and released it to users in 2022. ChatGPT is a machine learning-powered chatbot that can deliver detailed responses to inquiries. This research aims to evaluate the validity of ChatGPT-generated responses when scientific questions related to the biochemistry discipline are posed. A document analysis was conducted to determine the scientific validity of responses produced by ChatGPT for five questions. Five questions originating from bio-chemistry content were asked to ChatGPT in a written format. The AI’s generated answers were saved and analyzed depending on their scientific validity. As a result of the study, it was detected that ChatGPT responded with scientifically incorrect or incomplete answers to the five questions asked. Besides, when asked the reason for ChatGPT’s response, it is seen that AI insisted on its invalid answers. Following prompts for certainty, the AI’s performance was evaluated. It provided scientifically correct answers to the first two questions, partially correct answers to the third, and consistently offered invalid solutions for the remaining questions. Ultimately, ChatGPT’s capabilities are limited in providing scientifically rigorous responses. To obtain accurate and appropriate answers, it is imperative to pose comprehensive and detailed inquiries that facilitate a more precise and informed response. Scholars and researchers must acknowledge that ChatGPT harbors certain misconceptions and consequently only constitutes a somewhat dependable and scientifically validated resource.

Pre-service Physical Sciences Teachers’ Views on Integrating ChatGPT into Teaching: A Case Study

“…This means that learners would get incorrect responses most of the time when they attempted to use ChatGPT. Poor performance in responding to biology questions was also observed by Elmas et al (2024).…”

“…The major limitation that ChatGPT currently possesses, which was of great concern to preservice physical sciences teachers, was that sometimes ChatGPT provided inaccurate information (Elmas et al, 2024). While the teachers claimed they could notice inaccurate information, they feared that their learners would be unable to differentiate between accurate and inaccurate information.…”