Response accuracy of ChatGPT 3.5 Copilot and Gemini in interpreting biochemical laboratory data a pilot study

Abstract

With the release of ChatGPT at the end of 2022, a new era of thinking and technology use has begun. Artificial intelligence models (AIs) like Gemini (Bard), Copilot (Bing), and ChatGPT-3.5 have the potential to impact every aspect of our lives, including laboratory data interpretation. To assess the accuracy of ChatGPT-3.5, Copilot, and Gemini responses in evaluating biochemical data. Ten simulated patients' biochemical laboratory data, including serum urea, creatinine, glucose, cholesterol, triglycerides, low-density lipoprotein (LDL-c), and high-density lipoprotein (HDL-c), in addition to HbA1c, were interpreted by three AIs: Copilot, Gemini, and ChatGPT-3.5, followed by evaluation with three raters. The study was carried out using two approaches. The first encompassed all biochemical data. The second contained only kidney function data. The first approach indicated Copilot to have the highest level of accuracy, followed by Gemini and ChatGPT-3.5. Friedman and Dunn's post-hoc test revealed that Copilot had the highest mean rank; the pairwise comparisons revealed significant differences for Copilot vs. ChatGPT-3.5 (P = 0.002) and Gemini (P = 0.008). The second approach exhibited Copilot to have the highest accuracy of performance. The Friedman test with Dunn's post-hoc analysis showed Copilot to have the highest mean rank. The Wilcoxon Signed-Rank Test demonstrated an indistinguishable response (P = 0.5) of Copilot when all laboratory data were applied vs. the application of only kidney function data. Copilot is more accurate in interpreting biochemical data than Gemini and ChatGPT-3.5. Its consistent responses across different data subsets highlight its reliability in this context.

AI for Biophysical Phenomena: A Comparative Study of ChatGPT and Gemini in Explaining Liquid–Liquid Phase Separation

“…In contrast, Gemini tends to provide more data-driven and less conversational responses. As both models continue to evolve, they promise even more sophisticated applications in the future [10,11]. By subjecting these LLMs to a diverse array of queries ranging from basic principles to complex mechanisms of LLPS, we aim to evaluate their capability to deliver precise explanations, highlighting their strengths, limitations, and potential insights into these biological phenomena.…”

Letter to the Editor of the Journal of Medical Systems: Regarding “Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis”

“…The authors failed to provide the specific dates when the chatbots were queried and analyzed, leaving the rationale for counting one chatbot as two unclear. Recent research articles on the utilization of chatbots powered by large language models (LLMs) have noted that Bing has been rebranded as Copilot [ 3 ].…”