Modern Artificial Intelligence and Large Language Models in Graduate Medical Education: A Scoping Review of Attitudes, Applications & Practice

Abstract

Background Artificial intelligence (AI) holds transformative potential for graduate medical education (GME), yet, a comprehensive exploration of AI's applications, perceptions, and limitations in GME is lacking. Objective To map the current literature on AI in GME, identifying prevailing perceptions, applications, and research gaps to inform future research, policy discussions, and educational practices through a scoping review. Methods Following the Joanna Briggs Institute guidelines and the PRISMA-ScR checklist a comprehensive search of multiple databases up to February 2024 was performed to include studies addressing AI interventions in GME. Results Out of 1734 citations, 102 studies met the inclusion criteria, conducted across 16 countries, predominantly from North America (72), Asia (14), and Europe (6). Radiology had the highest number of publications (21), followed by general surgery (11) and emergency medicine (8). The majority of studies were published in 2023. Following key themes were identified: · Adoption Perceptions: Initially mixed attitudes, have shifted towards favorable perceptions, with increasing support for integrating AI education. · Assessments: AI can differentiate skill levels and provide feedback · Evaluations: AI can effectively analyze narrative comments to assess resident performance. · Recruitment: AI tools analyze letters of recommendation, applications, and personal statements, identifying biases and enhancing equity. · Standardized Examinations: AI models consistently outperformed average candidates in board certification and in-training exams. · Clinical Decision-Making: AI tools can support trainees with diagnostic accuracy and efficiency. Conclusions This scoping review provides a comprehensive overview of applications and limitations of AI in GME but is limited with potential biases, study heterogeneity, and evolving nature of AI.