Appropriateness of Answers to Common Preanesthesia Patient Questions Composed by the Large Language Model GPT-4 Compared to Human Authors

Abstract

M any surgical patients will not interact with anesthesiologists until minutes before surgery, and the internet has become a common source of medical information. The use of large language models such as GPT-4, which are "generative artificial intelligence" tools capable of creating natural, human-sounding prose in response to a plain-language query, and their incorporation into search engines, promise to make it easier for patients to directly ask questions related to preanesthetic preparation. The accuracy of large language models in answering medical questions has generally been impressive 1-3 but has not been evaluated for preanesthetic queries. We evaluated the ability of the widely accessible model GPT-4 to provide reasonable responses to common preanesthetic patient questions, compared to online published resources. Our hypothesis was that GPT-4 was at least as reasonable as published resources.The study was approved by the Wake Forest University School of Medicine institutional review board and completion of the survey was deemed to indicate informed consent. Sixteen common preanesthetic questions were drawn from websites of academic anesthesiology departments (table 1; Supplemental Content 1, https://links. lww.com/ALN/D360). The online answers and the answers to the same questions provided by GPT-4 via the ChatGPT Pro interface (chat.openai.com), queried on two dates in independent sessions in April 2023, were obtained. Two sessions with ChatGPT were used because the software regenerates new responses when reprompted, and the responses may differ in quality. 1 Survey participants were preoperative anesthesia experts known to the Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal's Web site (www.anesthesiology.org).