Large Language Models Encode Clinical Knowledge

Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine

“…The LLM, in conjunction with other algorithms (a detailed description of the technology is given in the next section), will instantly and automatically generate a response to the prompt that, in case of text-to-text transformation and judging by style, grammar, presentation, and often also content, seems indistinguishable from the output a human counterpart might have produced. For example, Med-PaLM, an LLM custom developed by Google and DeepMind for the medical field recently reported USMLE accuracy results which started to approach performance levels of human clinical experts 6 and were replicated by ChatGPT. 7 BioGPT, a LLM developed by Microsoft and trained on biomedical data, has achieved human parity in certain tasks of biomedical text generation and mining.…”

“…28 At the time of publication of this essay, ChatGPT had successfully passed AWS Cloud Certification 29 and Med-PaLM has started to approach human expert level performance on a subset of the USMLE exam. 6 Microsoft is reportedly in negotiations with OpenAI about a multi-billion-dollar follow-up investment in integrating ChatGPT into MS Office and has already embedded it in its search engine Bing. 30 Digital health AI guru Eric Topol has shared his encouraging views on the role LLM-powered generative AI applications could play in healthcare and medicine.…”

The impending impacts of large language models on medical education

“…LLMs not only have information about language, but also implicitly contain general information embedded within their parameters. A recent study showed that an LLM with instruction prompt tuning can perform medical question answering and reasoning, and also showed an accuracy of 67.6% on MedQA (US Medical License Exam) questions [3]. In the near future, medical students will be able to access highly sophisticated LLM-based medical knowledge bases that allow for the creation of dynamic learning materials tailored to their specific needs and questions.…”

“…LLMs are given specific prompts to guide them in producing step-by-step explanations that lead to a conclusion. Recent research has demonstrated that LLMs are also capable of performing reasoning in the medical field and are able to answer medical questions with a rather high level of accuracy [3,7]. Clinical reasoning is a crucial skill that medical education aims to cultivate in students.…”

ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes center

“…2 Potential applications of ChatGPT have also received attention from the medical research community with numerous editorials and commentaries published on the topic in major scientific journals, while some authors already used and attributed co-authorship to ChatGPT. 3-6 Recent studies have demonstrated that ChatGPT and similar AI-based systems reached a level to be able to pass components or the full United States Medical Licensing Exam 7-9 and can answer questions in specific medical specialties like genetics and ophthalmology. 10,11 Such proof of clinical knowledge is promising, but studies and applications in patient-centered scenarios are still rare.…”