Healthcare Professionals’ Expectations of Medical Artificial Intelligence and Strategies for its Clinical Implementation: A Qualitative Study

Abstract

Objectives: Although medical artificial intelligence (AI) systems that assist healthcare professionals in critical care settings are expected to improve healthcare, skepticism exists regarding whether their potential has been fully actualized. Therefore, we aimed to conduct a qualitative study with physicians and nurses to understand their needs, expectations, and concerns regarding medical AI; explore their expected responses to recommendations by medical AI that contradicted their judgments; and derive strategies to implement medical AI in practice successfully.Methods: Semi-structured interviews were conducted with 15 healthcare professionals working in the emergency room and intensive care unit in a tertiary teaching hospital in Seoul. The data were interpreted using summative content analysis. In total, 26 medical AI topics were extracted from the interviews. Eight were related to treatment recommendation, seven were related to diagnosis prediction, and seven were related to process improvement.Results: While the participants expressed expectations that medical AI could enhance their patients’ outcomes, increase work efficiency, and reduce hospital operating costs, they also mentioned concerns regarding distortions in the workflow, deskilling, alert fatigue, and unsophisticated algorithms. If medical AI decisions contradicted their judgment, most participants would consult other medical staff and thereafter reconsider their initial judgment.Conclusions: Healthcare professionals wanted to use medical AI in practice and emphasized that artificial intelligence systems should be trustworthy from the standpoint of healthcare professionals. They also highlighted the importance of alert fatigue management and the integration of AI systems into the workflow.

Attitudes towards artificial intelligence in emergency medicine

“…7 Comparison to previous qualitative studies (emergency clinicians) Previous qualitative interview-based studies of emergency clinicians' attitudes towards AI have had a smaller number of participants and mostly focused on attitudes towards specific AI-based tools, such as detecting pathology in chest X-rays, diagnosing aortic dissection, or predicting 30-day mortality. [30][31][32][33] In these studies, attitudes towards AI were generally positive, with clinicians viewing AI as a tool to supplement clinical expertise and help inexperienced clinicians. [30][31][32][33] However, there were also concerns that AI could bias clinical decisions, and that inexperienced clinicians could become over-reliant on such tools.…”

“…[30][31][32][33] In these studies, attitudes towards AI were generally positive, with clinicians viewing AI as a tool to supplement clinical expertise and help inexperienced clinicians. [30][31][32][33] However, there were also concerns that AI could bias clinical decisions, and that inexperienced clinicians could become over-reliant on such tools. [31][32][33] Other concerns included the trustworthiness of AI systems, alarm fatigue, the medicolegal risk of documenting AI-based predictions, the impact on clinician autonomy, and the multiple human factors that could be overlooked by AI.…”

“…[30][31][32][33] However, there were also concerns that AI could bias clinical decisions, and that inexperienced clinicians could become over-reliant on such tools. [31][32][33] Other concerns included the trustworthiness of AI systems, alarm fatigue, the medicolegal risk of documenting AI-based predictions, the impact on clinician autonomy, and the multiple human factors that could be overlooked by AI. [31][32][33] There were conflicting expectations that AI should increase work efficiency, but also concerns that additional work will be required to interact with the algorithms.…”

“…[31][32][33] Other concerns included the trustworthiness of AI systems, alarm fatigue, the medicolegal risk of documenting AI-based predictions, the impact on clinician autonomy, and the multiple human factors that could be overlooked by AI. [31][32][33] There were conflicting expectations that AI should increase work efficiency, but also concerns that additional work will be required to interact with the algorithms. [31][32][33] These findings are consistent with the results of our study.…”

“…[31][32][33] There were conflicting expectations that AI should increase work efficiency, but also concerns that additional work will be required to interact with the algorithms. [31][32][33] These findings are consistent with the results of our study.…”