Evaluation of Methodologies and Measures on the Usability of Social Robots: A Systematic Review

Jung, Minjoo; Lazaro, May Jorella; Yun, Myung Hwan

doi:10.3390/app11041388

Cited by 28 publications

(14 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The users also mentioned the better user experience, since accessing the information as a part of a group allowed them to discover facts, data and information more easily, and discussing with the near-by users improved the social experience. This will be further investigated as part of our future research, seeking the social usability of our multi-platform cultural product [27].…”

Section: Evaluation Results On the Interactive Touchscreen Tablementioning

confidence: 99%

User Evaluation of a Multi-Platform Digital Storytelling Concept for Cultural Heritage

et al. 2021

View full text Add to dashboard Cite

Digital storytelling platforms have proven to be a great way of bringing cultural heritage closer to people. What lacks is a deeper understanding of the user experience of such systems, especially in multi-platform digital storytelling. For the last three years, we have been developing a project called Spotlight Heritage Timisoara, which is at its core a digital storytelling platform for the city of Timisoara (Romania), soon to be European Capital of Culture in 2023. The project consists of a website, mobile applications, and interactive museographic and street exhibitions. This paper presents a multi-platform usability evaluation study which employed semi-structured interviews, observations, think-aloud protocol, SUS questionnaire, Net Promoter Score and Product Reaction Cards to gather insights from 105 participants and reveal usability problems in the Spotlight Heritage context. We found out that the four platforms, i.e., interactive touchscreen table, desktop/laptop, mobile and Augmented Reality, have very good usability scores, are considered accessible and useful, work seamlessly together, and create user satisfaction and loyalty, across demographic groups, having the potential to bring people closer to cultural heritage.

show abstract

Section: Evaluation Results On the Interactive Touchscreen Tablementioning

confidence: 99%

User Evaluation of a Multi-Platform Digital Storytelling Concept for Cultural Heritage

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This study is based on a qualitative method, interviews as a single research method. While interviews are not a common evaluation method in social robots’ usability studies, they are an appropriate way to evaluate trust in robots; this approach requires a “deep dive into users” [ 38 ]. We consider the implementation of a qualitative methodology, using multiple, in-depth interviews in the wild [ 16 , 17 ], as a significant contribution to HRI research.…”

Section: Methodsmentioning

confidence: 99%

“…The conversation, and the questions and answers dialogue encourage reflection and consideration by interviewees, sharing insights that would have been lost in surveys [ 15 ]. The researcher can ask follow-up questions in response to users' answers, to better understand their intent [ 38 ]. In this context, the interview plan did not assume any predefined hypotheses regarding the HRI and the HCI (although it corresponded methodologically with previous literature and applied customary concepts) in order to deliver to the HRI community interviewees’ authentic opinions and insights.…”

Section: Methodsmentioning

confidence: 99%

Extended Interviews with Stroke Patients Over a Long-Term Rehabilitation Using Human–Robot or Human–Computer Interactions

Koren

Polak

Levy-Tzedek

2022

Int J of Soc Robotics

View full text Add to dashboard Cite

Socially assistive robots (SARs) have been proposed to assist post-stroke patients in performing their exercise during their rehabilitation process, with the trust in the robot identified as an important factor in human–robot interaction. In the current study, we aimed to identify and characterize factors that influence post-stroke patients’ trust in a robot-operated and a computer-operated rehabilitation platform during and after a long-term experience with the platform. We conducted 29 interviews with 16 stroke patients who underwent a long-term rehabilitation process, assisted by either a SAR or a computer interface. The intervention lasted 5–7 weeks per patient, for a total of 229 sessions over 18 months. By using a qualitative research method—extended interviews “in the wild” with stroke patients, over a long-term rehabilitation process—our study reveals users’ perspectives regarding factors affecting trust in the SAR or in the computer interface during their rehabilitation process. The results support the assertion that SARs have an added value in the rehabilitative care of stroke patients; It appears that personal characteristics, such as age and gender, have an effect on the users’ acceptance of a non-human operator as a practice assistant. Our findings support the notion that SARs augment rehabilitative therapies beyond a standard computer; Importantly, patients appreciated different aspects of the non-human operator in the two groups: In the SAR group, users preferred its functional performance over its anthropomorphized social skills; In the Computer group, users highlighted its contribution to the training of their memory skills.

show abstract

“…Due to the COVID-19 pandemic, researchers further explored the possibilities of remote HRI studies and noticed that this type of experiment can increase the participants' effort and their frustration [53]. In contrast to live HRI studies, video-based studies have the advantage of reaching more participants and consistent experiment conditions among participants [54].…”

Section: Related Workmentioning

confidence: 99%

Explain yourself! Effects of Explanations in Human-Robot Interaction

Ambsdorf¹,

Munir²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent developments in explainable artificial intelligence promise the potential to transform human-robot interaction: Explanations of robot decisions could affect user perceptions, justify their reliability, and increase trust. However, the effects on human perceptions of robots that explain their decisions have not been studied thoroughly. To analyze the effect of explainable robots, we conduct a study in which two simulated robots play a competitive board game. While one robot explains its moves, the other robot only announces them. Providing explanations for its actions was not sufficient to change the perceived competence, intelligence, likeability or safety ratings of the robot. However, the results show that the robot that explains its moves is perceived as more lively and human-like. This study demonstrates the need for and potential of explainable human-robot interaction and the wider assessment of its effects as a novel research direction.

show abstract

Evaluation of Methodologies and Measures on the Usability of Social Robots: A Systematic Review

Cited by 28 publications

References 62 publications

User Evaluation of a Multi-Platform Digital Storytelling Concept for Cultural Heritage

User Evaluation of a Multi-Platform Digital Storytelling Concept for Cultural Heritage

Extended Interviews with Stroke Patients Over a Long-Term Rehabilitation Using Human–Robot or Human–Computer Interactions

Explain yourself! Effects of Explanations in Human-Robot Interaction

Contact Info

Product

Resources

About