A Case Study Investigating a User-Centred and Expert Informed 'Companion Guide' for a Complex Sensor-based Platform

Abstract

We present a case study that informs the creation of a 'companion guide' providing transparency to potential non-expert users of a ubiquitous machine learning (ML) platform during the initial onboarding. Ubiquitous platforms (e.g., smart home systems, including smart meters and conversational agents) are increasingly commonplace and increasingly apply complex ML methods. Understanding how non-ML experts comprehend these platforms is important in supporting participants in making an informed choice about if and how they adopt these platforms. To aid this decision-making process, we created a companion guide for a home health platform through an iterative user-centred-design process, seeking additional input from platform experts at all stages of the process to ensure the accuracy of explanations. This user-centred and expert informed design process highlights the need to present the platform's entire ecosystem at an appropriate level for those with differing backgrounds to understand, in order to support informed consent and decision making.

Explanation before Adoption: Supporting Informed Consent for Complex Machine Learning and IoT Health Platforms

“…These insight on an informed consent framing of the onboarding process complement prior work by Gilpin et al [23], Lombrozo [46], Burrows et al [9] and Lakoff et al [40]. Findings cast light on practical communication methods used by platform experts, such as tailoring language use or the level of detail offered to the individual explainee, comparing parts of the platform to commercial products or the natural world, using metaphors to describe the physical hardware, and using physical props [15] as well as gesture to aid in those conversations. The experts also scoped the explainee's level of knowledge and experience, and made use of shared questioning, common ground and effective linguistic repair.…”

“…This is particularly significant in reasoning about complex platforms with several sensor or analytical methods that might be beneficial to health if adopted and accepted, but which also impose limitations and potential risks. In our findings, the complex nature of the platform was addressed during both studies using multiple information types (spoken word strategies, sections 4.1.1 to 4.1.3; physical props, [15] and section 4.1.4; physical gestures, section 4.1.7), the availability of demos tailored to approach complex topics such as data fusion and participant privacy (section 4.1.4) and interactive approaches to explanation that importantly encourage open discussion around relevant topics.…”

“…To support the onboarding session, we used an A4-printed document [15] previously produced by the SPHERE team to aid in the explanation of the SPHERE platform as a whole. The designers of this A4 document [15] (fig. 1) used a user-centred design process to present the elements of SPHERE in a highly visual way (i.e.…”

“…sensor devices; data extraction; platform installation and removal; training of the data; data processing and the ML algorithms which combine and interpret activities). As this document [15] was a tool that the SPHERE team had developed to support in the onboarding explanation of the platform, we included the use of this document within our study. The participants were given the option of either using a digital or a printed version of the document with all opting for the printed version, which was sent by post.…”

“…4b) were too high-level and did not provide a good overview of the platform. However, the printed document used during the observational studies, as P4 noted (Eardley et al [15], fig. 1) provided a good balance to the verbal presentation.…”