Metrics and benchmarks in human-robot interaction: Recent advances in cognitive robotics

Abstract

Robots are having an important growing role in human social life, which requires them to be able to behave appropriately to the context of interaction so as to create a successful long-term human-robot relationship. A major challenge in developing intelligent systems, which could enhance the interactive abilities of robots, is defining clear metrics and benchmarks for the different aspects of humanrobot interaction, like human and robot skills and performances, which could facilitate comparing between systems and avoid application-biased evaluations based on particular measures. The point of evaluating robotic systems through metrics and benchmarks, in addition to some recent frameworks and technologies that could endow robots with advanced cognitive and communicative abilities, are discussed in this technical report that covers the outcome of our recent workshop on current advances in cognitive robotics: Towards Intelligent Social Robots-Current Advances in Cognitive Robotics, in conjunction with the 15th IEEE-RAS Humanoids Conference-Seoul-South Korea-2015 (https://intelligent-robots-ws.ensta-paristech.fr/). Additionally, a summary of an interactive discussion session between the workshop participants and the invited speakers about different issues related to cognitive robotics research is reported.

Measuring Progress in Robotics: Benchmarking and the ‘Measure-Target Confusion’

“…The issues have become more urgent as robotics has become more complex (Antonelli, 2015, p. 1) and continues to move into more complex environments and involve more humancomputer interaction (Aly, Griffiths, & Stramandinoli, 2017).…”

An Experimental Study of Embodied Interaction and Human Perception of Social Presence for Interactive Robots in Public Settings

“…There have been several efforts to identify and standardize a set of metrics and measurements in HRI research [18,[27][28][29]. One approach takes an ontological view [29].…”

Call to Redefine Human-Robot Interaction: From Social Robots to Gun Shooting Humanoids

“…The abundance of metrics in measuring human-robot interaction presents numerous challenges (Aly, Griffiths, & Stramandinoli, 2016). There is the infamous Negative Attitudes towards Robots Scale (NARS) developed by Nomura, Suzuki, Kanda, and Kato (2006), which was further improved by accounting for the touch distance (Walker et al, 2015).…”