Review of health information technology usability study methodologies

Abstract

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user–task interaction, n=42; stage 2, system–task interaction, n=310; stage 3, user–task–system interaction, n=69; stage 4, user–task–system–environment interaction, n=54; and stage 5, user–task–system–environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.

Supporting the Process of Help-Seeking by Caregivers of Functionally Dependent Seniors Through eHealth: Protocol for a Multicentric Codesign (Preprint)

“…The two methods employed are the think-aloud approach, to measure usability, and the questionnaire, to measure user satisfaction. First, in a process of digital tool development, the think-aloud method is frequently used to reveal usability problems that the user might encounter with the tool [39]. Generally, this method aims to capture a systematic process of thinking aloud, and analyze this process to gain a deeper appreciation of any problems, which could arise during the use of the digital tool.…”

“…The standardized questionnaires used most often are those for user interaction satisfaction, the modified technology acceptance model questionnaire, and the IBM usability questionnaire [39]. The modified technology acceptance model questionnaire was conceived in a telemonitoring context and is less relevant to the present study [41].…”

“…As for user satisfaction, (the second method), the most common method used is the questionnaire [39]. The standardized questionnaires used most often are those for user interaction satisfaction, the modified technology acceptance model questionnaire, and the IBM usability questionnaire [39].…”

“…Two distinct methods will be used for the study of usability [39] in the context of an individual encounter with the participant in a location of their choice, for a maximum of one hour. The two methods employed are the think-aloud approach, to measure usability, and the questionnaire, to measure user satisfaction.…”

A Systematic Approach for Using DICOM Structured Reports in Clinical Processes: Focus on Breast Cancer

“…Therefore, structured reports have been often criticized as impractical for complex cases [23]. Usability evaluates whether users perform tasks in an effective manner through a tool in a specific environment [24]. In this work, usability was used to measure the success of the clinical processes redesign.…”