Assessment of Artificial Intelligence Platforms With Regard to Medical Microbiology Knowledge: An Analysis of ChatGPT and Gemini

Abstract

The performance of two artificial intelligence (AI) platforms, ChatGPT 3.5 (OpenAI, California, United States) and Gemini (Google AI, California, United States) was assessed by answering 200 questions of microbiology drawn from validated sources. The questions were selected from topics such as General Microbiology, Immunology, and Microbiology Applied to Infectious Diseases. The study was conducted from December 2023 to March 2024, and the responses of the different AI platforms were compared with an answer key. Statistical analysis was performed to assess accuracy. ChatGPT 3.5 and Gemini had comparable accuracy with correct response scores of 71% and 70.5%, respectively. Their performance varied across different sections. Gemini performed better in General Microbiology and Immunology, and ChatGPT 3.5 had a better score in the Applied Microbiology section. The study's findings highlight that AI platforms such as ChatGPT and Gemini can be utilized in microbiology and medical education. The evolution and continuous updating of AI platforms are required to improve their performance.