Digital video analysis of health professionals’ interactions with an electronic whiteboard: A longitudinal, naturalistic study of changes to user interactions

Abstract

As hospital departments continue to introduce electronic whiteboards in real clinical settings a range of human factor issues have emerged and it has become clear that there is a need for improved methods for designing and testing these systems. In this study, we employed a longitudinal and naturalistic method in the usability evaluation of an electronic whiteboard system. The goal of the evaluation was to explore the extent to which usability issues experienced by users change as they gain more experience with the system. In addition, the paper explores the use of a new approach to collection and analysis of continuous digital video recordings of naturalistic "live" user interactions. The method developed and employed in the study included recording the users' interactions with system during actual use using screen-capturing software and analyzing these recordings for usability issues. In this paper we describe and discuss both the method and the results of the evaluation. We found that the electronic whiteboard system contains system-related usability issues that did not change over time as the clinicians collectively gained more experience with the system. Furthermore, we also found user-related issues that seemed to change as the users gained more experience and we discuss the underlying reasons for these changes. We also found that the method used in the study has certain advantages over traditional usability evaluation methods, including the ability to collect analyze live user data over time. However, challenges and drawbacks to using the method (including the time taken for analysis and logistical issues in doing live recordings) should be considered before utilizing a similar approach. In conclusion we summarize our findings and call for an increased focus on longitudinal and naturalistic evaluations of health information systems and encourage others to apply and refine the method utilized in this study.

Enhancing the Effectiveness of Consumer-Focused Health Information Technology Systems Through eHealth Literacy: A Framework for Understanding Users' Needs

“…The work of Kushniruk and Turner [ 23 ] extends and expands the concept of the user-task matrix to consider the key aspect of context of use. The user-task-context framework has also proven useful in the modeling of user requirements for different health care applications, mostly involving development of systems for health professionals [ 25 , 26 ].…”

An Evaluation Study of an Electronic Whiteboard System for Nursing Care in the Emergence Unit

“…With the informatization of hospital environments, traditional boards (manual) are being replaced by digital ones [Rasmussen et al 2010, Rasmussen andKushniruk 2013]. As a collaborative technology, the electronic whiteboards (TV, monitor or any electronic device) are placed in strategic locations in the hospitals to provide information, such as the patient's general status, bed, responsible physician, medications administered and next actions [Rasmussen et al 2010].…”

Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine

“…Regarding EHR-powered applications in EDs, tools with iconic GUIs significantly outperformed (using Student's t-tests) the non-iconic version considering the Nielsen's heuristic evaluation, effectiveness, and user satisfaction [86]. A very interest-ing usability study was conducted by the authors of [110] where the clinicians interaction with electronic whiteboards were analysed using a "naturalistic" approach. Live videos of the users while interacting with electronic whiteboards were collected, along with screen captures of the whiteboards themselves, to record actual system interaction.…”