Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes

Abstract

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems' main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.

Building Usability Knowledge for Health Information Technology: A Usability-Oriented Analysis of Incident Reports

“…However, since there are no observations of technology in use, only hypotheses can be drawn about the effect of usability flaws. 14 As for user-based evaluations (e.g., usertesting, 15 think-aloud protocols 16 ) where representative end-users interact with the technology in a controlled environment, they offer insights about how usability flaws can impair work (i.e., usage problems): however, hypotheses must still be drawn on their potential negative outcomes on the work system (including patient safety). 14 One way to examine the contribution of usability flaws to negative outcomes including patient harm is by field observations and interviews (e.g., see refs.…”

“…14 One way to examine the contribution of usability flaws to negative outcomes including patient harm is by field observations and interviews (e.g., see refs. 14,[17][18][19][20][21] ). These study designs enable connection of usability flaws with their effects on users and even with patient safety issues (e.g., see ref.…”

“…The data extraction was based on an existing usability framework 14,32 that describes the chain of latent consequences that leads from a usability flaw to a usage problem and then a negative outcome (►Fig. 1).…”

Challenges And Opportunities For Improving Patient Safety Through Human Factors And Systems Engineering

“…heuristic evaluation) early in the design process, user testing in simulated environments, and monitoring of safety problems of technology-in-use. (15, 52)…”

Board 127: Collaborative Multidisciplinary Engineering Design Experiences for Teachers (CoMET) Train-the-Trainer Model of Supports

“…This theory uses rapid prototyping cycles followed by end-user evaluation at periodic intervals throughout the process. User Interface testing during the iterative design process allows data and feedback to provide information through multiple pathways using cyclic methodologies that meet each user's unique needs [10].…”