Lessons Learned About Designing and Conducting Studies From HRI Experts

Abstract

The field of human-robot interaction (HRI) research is multidisciplinary and requires researchers to understand diverse fields including computer science, engineering, informatics, philosophy, psychology, and more disciplines. However, it is hard to be an expert in everything. To help HRI researchers develop methodological skills, especially in areas that are relatively new to them, we conducted a virtual workshop, Workshop Your Study Design (WYSD), at the 2021 International Conference on HRI. In this workshop, we grouped participants with mentors, who are experts in areas like real-world studies, empirical lab studies, questionnaire design, interview, participatory design, and statistics. During and after the workshop, participants discussed their proposed study methods, obtained feedback, and improved their work accordingly. In this paper, we present 1) Workshop attendees’ feedback about the workshop and 2) Lessons that the participants learned during their discussions with mentors. Participants’ responses about the workshop were positive, and future scholars who wish to run such a workshop can consider implementing their suggestions. The main contribution of this paper is the lessons learned section, where the workshop participants contributed to forming this section based on what participants discovered during the workshop. We organize lessons learned into themes of 1) Improving study design for HRI, 2) How to work with participants - especially children -, 3) Making the most of the study and robot’s limitations, and 4) How to collaborate well across fields as they were the areas of the papers submitted to the workshop. These themes include practical tips and guidelines to assist researchers to learn about fields of HRI research with which they have limited experience. We include specific examples, and researchers can adapt the tips and guidelines to their own areas to avoid some common mistakes and pitfalls in their research.

Workshop YOUR study design! Participatory critique and refinement of participants' studies

“…We plan to document the workshop via a paper that includes a meta-analysis of current practices in HRI and a discussion of the outcomes of the workshop. We hope to leverage the lessons learnt during the breakout sessions and the mini discussions to write a follow-up paper to the existing discussions on study design guidelines [3], [4]. We expect that this paper will extend prior work by providing guidelines specific to the sub-fields of HRI such as child-robot interaction, trust in HRI etc.…”

Biological Motion in Robots and Users’ Action Understanding

“…After 200 trials, participants still could not effectively imitate the nonbiological motion, and their performance did not significantly differ from their first attempt. In sum, this body of human-robot interaction research has been hampered by the vast heterogeneity of robots and paradigms (Fraune et al, 2022), with statistical reporting standards unevenly applied.…”

Towards Improved Replicability of Human Studies in Human-Robot Interaction

“…An example of one such attempt is seen in Fraune et al, [10]. Within this paper, the authors summarize the central insights gained from the recent Workshop Your Study Design (WYSD) workshop held at the 2021 International Conference on Human-Robot Interaction 6 .…”