Telerobotic Pointing Gestures Shape Human Spatial Cognition

Abstract

This paper aimed to explore whether human beings can understand gestures produced by telepresence robots. If it were the case, they can derive meaning conveyed in telerobotic gestures when processing spatial information. We conducted two experiments over Skype in the present study. Participants were presented with a robotic interface that had arms, which were teleoperated by an experimenter. The robot could point to virtual locations that represented certain entities. In Experiment 1, the experimenter described spatial locations of fictitious objects sequentially in two conditions: speech condition (SO, verbal descriptions clearly indicated the spatial layout) and speech and gesture condition (SR, verbal descriptions were ambiguous but accompanied by robotic pointing gestures). Participants were then asked to recall the objects' spatial locations. We found that the number of spatial locations recalled in the SR condition was on par with that in the SO condition, suggesting that telerobotic pointing gestures compensated ambiguous speech during the process of spatial information. In Experiment 2, the experimenter described spatial locations nonsequentially in the SR and SO conditions. Surprisingly, the number of spatial locations recalled in the SR condition was even higher than that in the SO condition, suggesting that telerobotic pointing gestures were more powerful than speech in conveying spatial information when information was presented in an unpredictable order. The findings provide evidence that human beings are able to comprehend telerobotic gestures, and importantly, integrate these gestures with co-occurring speech. This work promotes engaging remote collaboration among humans through a robot intermediary.

A Review of Mobile Robotic Telepresence

“…The participants also perceived both the embodied collaborator and the colleagues involved in the interaction as being more composed and involved when the embodied collaborator took a leadership role in the interaction. Spatial location recalling with a similar type of system as MeBot making use of Skype and arms was investigated in [99]. It was found that participants facing a system with verbal location descriptions accompanied by robotic pointing gestures remembered more locations in comparison with participants facing a system that only provided verbal descriptions when the locations were presented nonsequentially.…”

Combining Robotic Persuasive Strategies: The Persuasive Power of a Storytelling Robot that Uses Gazing and Gestures

“…Next to social robots that assist people in medical situations [1][2][3], social robots can also help improve the user's spatial cognition through pointing gestures [4][5][6][7], improve the imitation skills of children with autism [8][9][10][11], or greet and communicate with people [12][13][14][15]. Therefore, we argue that one of the most important current challenges in research on social robotics is to develop social robots that can effectively influence human behavior or attitudes.…”

Speech and Gesture Emphasis Effects for Robotic and Human Communicators

“…Previous work in human robot-interaction on how coverbal gestures affect comprehension has largely focused on pointing (deictic) gestures [5][23] [28], revealing that better understanding of relative locations of referents was achieved by supplementing speech information with deictic gestures. This provides some first evidence for speech and gesture integration.…”