Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study

Carayon, Pascale; Hoonakker, Peter; Hundt, Ann Schoofs; Salwei, Megan E.; Wiegmann, Douglas A.; Brown, Roger; Kleinschmidt, Peter; Novak, Clair; Pulia, Michael S.; Wang, Yudi; Wirkus, Emily J.; Patterson, Brian W.

doi:10.1136/bmjqs-2019-009857

Cited by 45 publications

(51 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study codesigned an AI chatbot with 20 students and performed a formative evaluation to better understand their experience of using the tool [ 72 ]. Two recent studies measured the perceived usability of AI-based decision-making tools: Ada, an AI tool that helps patients navigate the right type of care [ 69 ], and PE-Dx CDS, a tool for diagnosing pulmonary embolism [ 66 ]. However, in another study, the researchers primarily focused on developing the algorithm for assessing the severity of pulmonary obstruction and obtained users’ feedback on the end product [ 74 ].…”

Section: Resultsmentioning

confidence: 99%

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

Asan¹,

Choudhury²

2021

JMIR Hum Factors

View full text Add to dashboard Cite

Background Despite advancements in artificial intelligence (AI) to develop prediction and classification models, little research has been devoted to real-world translations with a user-centered design approach. AI development studies in the health care context have often ignored two critical factors of ecological validity and human cognition, creating challenges at the interface with clinicians and the clinical environment. Objective The aim of this literature review was to investigate the contributions made by major human factors communities in health care AI applications. This review also discusses emerging research gaps, and provides future research directions to facilitate a safer and user-centered integration of AI into the clinical workflow. Methods We performed an extensive mapping review to capture all relevant articles published within the last 10 years in the major human factors journals and conference proceedings listed in the “Human Factors and Ergonomics” category of the Scopus Master List. In each published volume, we searched for studies reporting qualitative or quantitative findings in the context of AI in health care. Studies are discussed based on the key principles such as evaluating workload, usability, trust in technology, perception, and user-centered design. Results Forty-eight articles were included in the final review. Most of the studies emphasized user perception, the usability of AI-based devices or technologies, cognitive workload, and user’s trust in AI. The review revealed a nascent but growing body of literature focusing on augmenting health care AI; however, little effort has been made to ensure ecological validity with user-centered design approaches. Moreover, few studies (n=5 against clinical/baseline standards, n=5 against clinicians) compared their AI models against a standard measure. Conclusions Human factors researchers should actively be part of efforts in AI design and implementation, as well as dynamic assessments of AI systems’ effects on interaction, workflow, and patient outcomes. An AI system is part of a greater sociotechnical system. Investigators with human factors and ergonomics expertise are essential when defining the dynamic interaction of AI within each element, process, and result of the work system.

show abstract

Section: Resultsmentioning

confidence: 99%

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

Asan¹,

Choudhury²

2021

JMIR Hum Factors

View full text Add to dashboard Cite

show abstract

“…This may have negative implications when the CDS is implemented since users have to use both the Wells' and PERC, which may hinder physician workflow and efficiency. However, this negative impact may be minimized since the total time to complete the patient scenarios decreased or stayed the same from session 1 to session 2 in 82% of cases using this workflow (Carayon et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

“…This study was part of a larger study evaluating the usability of a CDS to support PE diagnosis (Carayon et al, 2019), which took place in the ED of a large academic teaching hospital in the US. We enrolled 32 emergency medicine physicians in the study: 8 year-one residents, 8 yeartwo residents, 8 year-three residents, and 8 attending physicians.…”

Section: Setting and Participantsmentioning

confidence: 99%

Assessing workflow of emergency physicians in the use of clinical decision support

Salwei

Carayon

Hoonakker

et al. 2019

Proceedings of the Human Factors and Ergonomics Society Annual

Self Cite

View full text Add to dashboard Cite

The emergency department (ED) is a complex environment where diagnoses must often be made quickly, based on incomplete information. Pulmonary embolism (PE) is an especially challenging diagnosis that is frequently delayed or missed due to its non-specific symptoms, and can be life-threatening when not treated. Clinical decision supports (CDS) have the potential to improve these difficult decisions; however, previous efforts to implement CDS in the ED have faced challenges due to poor usability and lack of workflow integration. The objective of this study is to identify potential barriers to workflow integration from the technology’s implementation and inform the CDS design; this is achieved by analyzing ED physicians’ workflow during a usability evaluation of two different CDS, a web-based risk scoring CDS and a CDS designed using an human-centered design (HCD) process and human factors (HF) design principles. The number of cases matching the guideline-based workflow and the percent of correct diagnostic decisions increased from the use of the HF-based CDS, but varied depending on the patient scenario. We identified three workflow variations, which had both positive and negative implications for the CDS design and implementation. The workflow analysis can be used to inform the CDS design and improve the technology’s usability and integration in physician workflow prior to implementation.

show abstract

“…2 3 In particular, user-centered approaches are needed to design and evaluate technologies before they are actually implemented. While such approaches have been used to develop health IT to support the work of individuals (e.g., individual clinician interactions with clinical decision support systems 4 and computerized medication alerts, 5 limited research has tackled the design of team-oriented health IT. 1 By team-oriented health IT, we mean the use of health IT to support the information needed by a team of people to care for a single patient.…”

Section: Background and Significancementioning

confidence: 99%

Scenario-Based Evaluation of Team Health Information Technology to Support Pediatric Trauma Care Transitions

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background Clinicians need health information technology (IT) that better supports their work. Currently, most health IT is designed to support individuals; however, more and more often, clinicians work in cross-functional teams. Trauma is one of the leading preventable causes of children's death. Trauma care by its very nature is team based but due to the emergent nature of trauma, critical clinical information is often missed in the transition of these patients from one service or unit to another. Teamwork transition technology can help support these transitions and minimize information loss while enhancing information gathering and storage. In this study, we created a large screen technology to support shared situational awareness across multiple clinical roles and departments. Objectives This study aimed to examine if the Teamwork Transition Technology (T3) supports teams and team cognition. Methods We used a scenario-based mock-up methodology with 36 clinicians and staff from the different units and departments who are involved in pediatric trauma to examine T3. Results Results of the evaluation show that most participants agreed that the technology helps achieve the goals set out in the design phase. Respondents thought that T3 organizes and presents information in a different way that was helpful to them. Conclusion In this study, we examined a health IT (T3) that was designed to support teams and team cognition. The results of our evaluation show that participants agreed that T3 does support them in their work and increases their situation awareness.

show abstract

Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study

Cited by 45 publications

References 71 publications

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

Assessing workflow of emergency physicians in the use of clinical decision support

Scenario-Based Evaluation of Team Health Information Technology to Support Pediatric Trauma Care Transitions

Contact Info

Product

Resources

About