Designing Visual Markers for Continuous Artificial Intelligence Support

Berkel, Niels van; Ahmad, Omer F.; Stoyanov, Danail; Lovat, Laurence; Blandford, Ann

doi:10.1145/3422156

Cited by 18 publications

(26 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to these works, all of which focus on intermittent interaction, our work aims to explore AI-user interaction in a continuous scenario. The constraints of this setting, such as the direct control over a live video feed, the need to keep focus on the endoscopic image at all times, and therefore the limited amount of information that can be overlaid on the image, impose a different way of working, for example, on endoscopists [40]. Prior work on continuous interaction stresses that the HCI and digital health community "need to ensure that guidelines on the design of AI systems accurately Initial Responses to False Positives in AI-Supported Continuous Interactions 2:5 reflect user needs when the user is not necessarily the starting nor the end point of an interaction, but instead operates along a continuum" [40].…”

Section: Designing For Error In Ai Supportmentioning

confidence: 99%

“…Through a bespoke Python script, we rendered the AI support visuals 'on top' of the annotated areas. The design of the AI indicator (i.e., a circle encompassing the annotated area, its size adjusting in alignment with the annotation) is based on earlier recommendations on continuous AI support [40]. Out of the total of 14 videos, 6 videos contain an actual polyp (i.e., true positives).…”

Section: Video Materialsmentioning

confidence: 99%

“…The majority of HCI literature on AI support has focused on intermittent scenarios [4,7,12], such as pathology classification on a stationary image, in which the interaction between user and AI can be defined as a turn-taking process. Continuous interaction scenarios, in which user input is sustained over a period of time and may receive AI input at any time, remain underexplored despite the unique challenges faced in terms of user interaction [40]. Clinical decision support during colonoscopy, the case presented in this article, is indicative of this different type of human-AI interaction, as AIpowered suggestions may repeatedly appear and disappear on a screen while a clinician navigates through a patient's colon.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Initial Responses to False Positives in AI-Supported Continuous Interactions: A Colonoscopy Case Study

Berkel

Opie

Ahmad

et al. 2022

ACM Trans. Interact. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

The use of artificial intelligence (AI) in clinical support systems is increasing. In this article, we focus on AI support for continuous interaction scenarios. A thorough understanding of end-user behaviour during these continuous human-AI interactions, in which user input is sustained over time and during which AI suggestions can appear at any time, is still missing. We present a controlled lab study involving 21 endoscopists and an AI colonoscopy support system. Using a custom-developed application and an off-the-shelf videogame controller, we record participants’ navigation behaviour and clinical assessment across 14 endoscopic videos. Each video is manually annotated to mimic an AI recommendation, being either true positive or false positive in nature. We find that time between AI recommendation and clinical assessment is significantly longer for incorrect assessments. Further, the type of medical content displayed significantly affects decision time. Finally, we discover that the participant’s clinical role plays a large part in the perception of clinical AI support systems. Our study presents a realistic assessment of the effects of imperfect and continuous AI support in a clinical scenario.

show abstract

Section: Designing For Error In Ai Supportmentioning

confidence: 99%

Section: Video Materialsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Initial Responses to False Positives in AI-Supported Continuous Interactions: A Colonoscopy Case Study

Berkel

Opie

Ahmad

et al. 2022

ACM Trans. Interact. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…From the IT perspective, clinicians needed transparency on DST output to feel confident in adopting them. This touches on key topics about explainability and trustworthiness of AI systems in healthcare [28,56]. Healthcare is a particularly challenging space, where the demand for explanation is much higher compared to other applications [2], and where explanations depend more profoundly on domain knowledge and the needs of the specific context.…”

Section: Opportunities To Adopt Dsts In Ad Clinical Practicementioning

confidence: 99%

“…Another important point refers to Human Factors, such as stakeholders' needs and beliefs [61]. Involving users early in the design process [43,56,57,64] can fulfil these requirements, saving time and resources, and avoiding critical errors. Their engagement can guide in defining the intended use of DSTs.…”

Section: Opportunities To Adopt Dsts In Ad Clinical Practicementioning

confidence: 99%

Opportunities and Barriers for Adoption of a Decision-Support Tool for Alzheimer’s Disease

Bellio¹,

Furniss²,

Oxtoby³

et al. 2021

ACM Trans. Comput. Healthcare

Self Cite

View full text Add to dashboard Cite

Clinical decision-support tools (DSTs) represent a valuable resource in healthcare. However, lack of Human Factors considerations and early design research has often limited their successful adoption. To complement previous technically focused work, we studied adoption opportunities of a future DST built on a predictive model of Alzheimer’s Disease (AD) progression. Our aim is two-fold: exploring adoption opportunities for DSTs in AD clinical care, and testing a novel combination of methods to support this process. We focused on understanding current clinical needs and practices, and the potential for such a tool to be integrated into the setting, prior to its development. Our user-centred approach was based on field observations and semi-structured interviews, analysed through workflow analysis, user profiles, and a design-reality gap model. The first two are common practice, whilst the latter provided added value in highlighting specific adoption needs. We identified the likely early adopters of the tool as being both psychiatrists and neurologists based in research-oriented clinical settings. We defined ten key requirements for the translation and adoption of DSTs for AD around IT, user, and contextual factors. Future works can use and build on these requirements to stand a greater chance to get adopted in the clinical setting.

show abstract

Making AI Work

Berkel¹

2023

AI in Clinical Medicine

View full text Add to dashboard Cite

Learning Objectives• Understand the field of human-computer interaction (HCI), as well as have an overview of relevant work in this area for further reading. • Understand some of the factors that support the alignment of AI systems with the needs of medical professionals, patients, and other relevant stakeholders.• Recognize the importance of stakeholder involvement when designing human-AI systems in healthcare. • Consider the role and importance of a study's ecological validity, longitudinal assessments, validation across diverse datasets, and iterative development when designing human-AI systems in healthcare. • Discuss recommendations for the development of AI systems in healthcare, with lessons learned from prior deployments. IntroductionPromising stories of AI have set sky-high expectations for its ability to transform medical practice. However, living up to even the more modest expectations will require careful consideration of how medical professionals will interact with this technology and how we can successfully embed AI systems within day-to-day clinical practice. Previous attempts at introducing technology into the medical context, such as electronic health records [33], have not been unanimously successful [23]. This is despite what is often initial optimism regarding a technology's ability to transform healthcare. Similarly, in the recent and ongoing battle against COVID-19, many AI systems were developed to support the diagnosis and triaging of patients, the impact of which was ultimately limited [37]. These challenges highlight the difficulties of designing and evaluating digital systems for use in the healthcare context, which may be further exacerbated if the technology does not meet the needs of medical professionals. Yet, the potential for AI systems to support medical professionals is vast, covering areas such as diagnosis, prognosis, guided surgery, and assisting in the training of medical personnel. This chapter provides a starting point for developers and clinicians interested in the design and evaluation of AI systems in a healthcare context. The field of human-computer interaction (HCI) has developed methods and guidelines on how to design and evaluate digital systems so that they

show abstract

Designing Visual Markers for Continuous Artificial Intelligence Support

Cited by 18 publications

References 53 publications

Initial Responses to False Positives in AI-Supported Continuous Interactions: A Colonoscopy Case Study

Initial Responses to False Positives in AI-Supported Continuous Interactions: A Colonoscopy Case Study

Opportunities and Barriers for Adoption of a Decision-Support Tool for Alzheimer’s Disease

Making AI Work

Contact Info

Product

Resources

About