In the wild pilot usability assessment of a connected health system for stroke self management

Abstract

This paper reports on the findings of a pilot study for the formative "in the wild" assessment of the usability of CONSULT, a research-led connected health system for stroke self-management and prevention. CONSULT integrates data from commercial wellness sensors, electronic health records and clinical guidelines and enables users to monitor their vital signs to support self-monitoring and provision of tailored advice. The CONSULT system includes a dashboard and a chatbot. To assess the usability of our system, six volunteers were recruited to interact with CONSULT over a period of seven days. System logs confirmed that participants interacted with the CONSULT system throughout. CONSULT's ability to integrate data from different sensors was an aspect of this systems that all participants liked and kept them motivated to track their vital signs. The study also revealed several usability issues that designers of this type of systems should consider. Some of the most prevalent issues were: information overload, data misinterpretation, need for more anthropomorphic conversational capabilities for the chatbot; lack of visibility of the data transmission status. This paper concludes with reflections on the importance of these findings when assessing the usability of connected health systems, like CONSULT.

EQRbot: A chatbot delivering EQR argument-based explanations

“…. ] the lack of a more natural conversation flow when interacting with the chatbot (e.g., close to the one that they [the patients] would have with their GP)" (Balatsoukas et al, 2020).…”

“…A seven day within-subjects mixed-methods run in-the-wild (Waterson et al, 2002 ) study has been conducted to assess the usability and acceptability of the CONSULT system with two different versions: with and without a chatbot. Such a pilot study demonstrated that real users could employ the application over an extended period (Balatsoukas et al, 2020 ). Connie, the conversational agent previously equipped with the cDSS at the time of the experiment, accommodates the patients willing to seek immediate evidence-based advice about a specific health problem.…”

“…This option triggers one last response from the chatbot, thus providing the explanation behind the rationale of the proposed recommendation. Nonetheless, Connie presents some limitations, as summarized by the result of the pilot study: “[…] the lack of a more natural conversation flow when interacting with the chatbot (e.g., close to the one that they [the patients] would have with their GP)” (Balatsoukas et al, 2020 ).…”

The relationship of socio-demographic factors and patient attitudes to connected health technologies: A survey of stroke survivors

“…The CONSULT system was developed and co-designed with Patient and Health Care Professionals (HCP) focus groups 22 , and includes a dashboard and a chatbot that were pilot-tested on healthy volunteers 23, 24 . The prototype system uses commercial wireless sensor devices that make it possible for patients to monitor a range of health and wellness data at home, including: blood pressure using a wireless cuff; and heart function using a smartwatch and a disposable ECG patch.…”