The use of think-aloud and instant data analysis in evaluation research: Exemplar and lessons learned

Joe, Jonathan; Chaudhuri, Shomir; Le, Thai; Thompson, Hilaire J.; Demiris, George

doi:10.1016/j.jbi.2015.06.001

Cited by 59 publications

(54 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Observing users in a more natural use environment can lead to richer data, but the data can be harder to quantify effectively. In early instances of user testing, the administrator will often ask the subject to think aloud, allowing the observer to gain an insight into the train of thought the user is employing as they encounter and attempt to overcome usability and human factors problems [ 62 , 63 ] (Requirements 1 and 5). Cameras, audio recorders, and note taking are employed to record the user behavior.…”

Section: Methodsmentioning

confidence: 99%

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology

Harte¹,

Glynn²,

et al. 2017

View full text Add to dashboard Cite

BackgroundDesign processes such as human-centered design, which involve the end user throughout the product development and testing process, can be crucial in ensuring that the product meets the needs and capabilities of the user, particularly in terms of safety and user experience. The structured and iterative nature of human-centered design can often present a challenge when design teams are faced with the necessary, rapid, product development life cycles associated with the competitive connected health industry.ObjectiveWe wanted to derive a structured methodology that followed the principles of human-centered design that would allow designers and developers to ensure that the needs of the user are taken into account throughout the design process, while maintaining a rapid pace of development. In this paper, we present the methodology and its rationale before outlining how it was applied to assess and enhance the usability, human factors, and user experience of a connected health system known as the Wireless Insole for Independent and Safe Elderly Living (WIISEL) system, a system designed to continuously assess fall risk by measuring gait and balance parameters associated with fall risk.MethodsWe derived a three-phase methodology. In Phase 1 we emphasized the construction of a use case document. This document can be used to detail the context of use of the system by utilizing storyboarding, paper prototypes, and mock-ups in conjunction with user interviews to gather insightful user feedback on different proposed concepts. In Phase 2 we emphasized the use of expert usability inspections such as heuristic evaluations and cognitive walkthroughs with small multidisciplinary groups to review the prototypes born out of the Phase 1 feedback. Finally, in Phase 3 we emphasized classical user testing with target end users, using various metrics to measure the user experience and improve the final prototypes.ResultsWe report a successful implementation of the methodology for the design and development of a system for detecting and predicting falls in older adults. We describe in detail what testing and evaluation activities we carried out to effectively test the system and overcome usability and human factors problems.ConclusionsWe feel this methodology can be applied to a wide variety of connected health devices and systems. We consider this a methodology that can be scaled to different-sized projects accordingly.

show abstract

Section: Methodsmentioning

confidence: 99%

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology

Harte¹,

Glynn²,

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The team would then identify end‐user specific needs (i.e., how technology can aid in the design of interventions to facilitate patient or caregiver–provider communication and reduce symptom burden) and then collect iterative feedback through focus groups or interviews using participatory design sessions that engage end users. One strategy to guide this process is “design thinking,” which is a user‐centered process that provides an organizational framework, leverages collective expertise, and encourages innovation (Gibbons, ; Joe, Chaudhuri, Le, Thompson, & Demiris, ). Data from focus groups should be analyzed using qualitative research techniques to establish themes, including individual values related to technology, to guide the next step of the roadmap.…”

Section: A Roadmap For Technology Use In Precision Healthmentioning

confidence: 99%

The Use of Technology to Support Precision Health in Nursing Science

Starkweather

Jacelon

Bakken

et al. 2019

J of Nursing Scholarship

View full text Add to dashboard Cite

Purpose This article outlines how current nursing research can utilize technology to advance symptom and self‐management science for precision health and provides a roadmap for the development and use of technologies designed for this purpose. Approach At the 2018 annual conference of the National Institute of Nursing Research (NINR) Research Centers, nursing and interdisciplinary scientists discussed the use of technology to support precision health in nursing research projects and programs of study. Key themes derived from the presentations and discussion were summarized to create a proposed roadmap for advancement of technologies to support health and well‐being. Conclusions Technology to support precision health must be centered on the user and designed to be desirable, feasible, and viable. The proposed roadmap is composed of five iterative steps for the development, testing, and implementation of technology‐based/enhanced self‐management interventions. These steps are (a) contextual inquiry, focused on the relationships among humans, and the tools and equipment used in day‐to‐day life; (b) value specification, translating end‐user values into end‐user requirements; (c) design, verifying that the technology/device can be created and developing the prototype(s); (d) operationalization, testing the intervention in a real‐world setting; and (e) summative evaluation, collecting and analyzing viability metrics, including process data, to evaluate whether the technology and the intervention have the desired effect. Clinical Relevance Interventions using technology are increasingly popular in precision health. Use of a standard multistep process for the development and testing of technology is essential.

show abstract

“…After each round of user testing, the study team employed the best practices for instant data analysis of usability data for each PRO [ 54 , 55 ]. Task success was calculated as the percentage of PRO questions that were answered correctly without errors.…”

Section: Methodsmentioning

confidence: 99%

Investigation of a Mobile Health Texting Tool for Embedding Patient-Reported Data Into Diabetes Management (i-Matter): Development and Usability Study

Schoenthaler¹,

Cruz²,

Payano³

et al. 2020

JMIR Form Res

View full text Add to dashboard Cite

Background Patient-reported outcomes (PROs) are increasingly being used in the management of type 2 diabetes (T2D) to integrate data from patients’ perspective into clinical care. To date, the majority of PRO tools have lacked patient and provider involvement in their development, thus failing to meet the unique needs of end users, and lack the technical infrastructure to be integrated into the clinic workflow. Objective This study aims to apply a systematic, user-centered design approach to develop i-Matter (investigating a mobile health [mHealth] texting tool for embedding patient-reported data into diabetes management), a theory-driven, mobile PRO system for patients with T2D and their primary care providers. Methods i-Matter combines text messaging with dynamic data visualizations that can be integrated into electronic health records (EHRs) and personalized patient reports. To build i-Matter, we conducted semistructured group and individual interviews with patients with T2D and providers, a design thinking workshop to refine initial ideas and design the prototype, and user testing sessions of prototypes using a rapid-cycle design (ie, design-test-modify-retest). Results Using an iterative user-centered process resulted in the identification of 6 PRO messages that were relevant to patients and providers: medication adherence, dietary behaviors, physical activity, sleep quality, quality of life, and healthy living goals. In user testing, patients recommended improvements to the wording and timing of the PRO text messages to increase clarity and response rates. Patients also recommended including motivational text messages to help sustain engagement with the program. The personalized report was regarded as a key tool for diabetes self-management by patients and providers because it aided in the identification of longitudinal patterns in the PRO data, which increased patient awareness of their need to adopt healthier behaviors. Patients recommended adding individualized tips to the journal on how they can improve their behaviors. Providers preferred having a separate tab built into the EHR that included the personalized report and highlighted key trends in patients’ PRO data over the past 3 months. Conclusions PRO tools that capture patients’ well-being and the behavioral aspects of T2D management are important to patients and providers. A clinical trial will test the efficacy of i-Matter in 282 patients with uncontrolled T2D. Trial Registration ClinicalTrials.gov NCT03652389; https://clinicaltrials.gov/ct2/show/NCT03652389

show abstract

The use of think-aloud and instant data analysis in evaluation research: Exemplar and lessons learned

Cited by 59 publications

References 40 publications

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology

The Use of Technology to Support Precision Health in Nursing Science

Investigation of a Mobile Health Texting Tool for Embedding Patient-Reported Data Into Diabetes Management (i-Matter): Development and Usability Study

Contact Info

Product

Resources

About