An Analysis of Usability Evaluation Practices And Contexts of Use In Wearable Robotics

Abstract

Background : User-centered design approaches have gained attention over the past decade, aiming to tackle the technology acceptance issues of wearable robotic devices to assist, support or augment human capabilities. While there is a consensus that usability is key to user-centered design, dedicated usability evaluation studies are scarce and clear evaluation guidelines are missing. However, the careful consideration and integration of user needs appears to be essential to successfully develop an effective, efficient, and satisfactory human-robot interaction. Methods : Through an online survey for developers of wearable robotics, we wanted to understand how the design and evaluation in actual daily practice compares to what is reported in literature. With a total of 31 questions, we analyzed the most common wearable robotic device applications and their technology maturity, and how these influence usability evaluation practices. Results : A total of 158 responses from a heterogeneous population were collected and analyzed. The dataset representing contexts of use for augmentation (16.5%), assistance (38.0%), therapy (39.8%), as well as few other specifc applications (5.7%), allowed for an insightful analysis of the influence of technology maturity on user involvement and usability evaluation. We identifed functionality, ease of use, and performance as the most evaluated usability attributes and could specify which measures are used to assess them. Also, we could underline the frequent use of qualitative measures alongside the expected high prevalence of performance-metrics. In conclusion of the analysis, we derived evaluation recommendations to foster user-centered design and usability evaluation. Conclusion : This analysis might serve as state-of-the-art comparison and recommendation for usability studies in wearable robotics. We believe that by motivating for more balanced, comparable and user-oriented evaluation practices, we may support the wearable robotics field in tackling the technology acceptance limitations.

A Survey on the Influence of CYBATHLON on the Development and Acceptance of Advanced Assistive Technologies

“…This is in contrast to the PIL perceptions, which indicate an encouraging level of satisfaction (QUEST mean = 3.98) with the AAT usability [30]. Given that the QUEST is one of the most used scales for the subjective rating of perceived usability [31,33], the mean QUEST scores from our data might additionally serve as a benchmark for other advanced assistive technologies. Still, besides all other means of measuring user experience and usability, the actual daily use -and hence technology acceptance and adoption -is arguably the most reliable usability outcome.…”

“…A reduced involvement of the PIL in the more technical phases of device development might be a reason for partial disagreement of the PIL's personal impact and consideration in the nal design, as observed in the Likert Scale answers. Only in a few teams, the PIL simultaneously are among the leading engineers, or researchers behind the technical solutions to implement their needs and wishes directly [32,33]. The employment status and actual workload of PIL within development teams remain somewhat undocumented, restricting a more detailed analysis.…”

“…In the majority of AAT developments, target users only get involved once the engineering teams have spent substantial resources developing and testing with neurologically intact, mock-users [33]. At this point, there is a high probability that the designs shot past their target of solving user-de ned problems of daily life, and to do so with satisfactory end-user usability.…”