A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction

Abstract

Robots are progressively moving into spaces that have been primarily shaped by human agency; they collaborate with human users in different tasks that require them to understand human language so as to behave appropriately in space. To this end, a stubborn challenge that we address in this paper is inferring the syntactic structure of language, which embraces grounding parts of speech (e.g., nouns, verbs, and prepositions) through visual perception, and induction of Combinatory Categorial Grammar (CCG) in situated humanrobot interaction. This could pave the way towards making a robot able to understand the syntactic relationships between words (i.e., understand phrases), and consequently the meaning of human instructions during interaction, which is a future scope of this current study.

Evaluation of Word Representations in Grounding Natural Language Instructions Through Computational Human-Robot Interaction

“…Thereupon, a logical explanation could be that the short sentences did not allow the model to create enough samples for learning [16]. This hypothesis is concordant with the findings of Aly et al [3], where longer sentences with more than one object allowed a similar model to appropriately learn the Object modality 12 . However, this requires further investigations through a different experimental setup and a new study.…”

“…While the sentences used by Aly et al[3] are longer, the employed short sentences in our current study are more intuitiv and near to the daily language used by human users to interact with robots, which constituted our motivation for the experimental setup of this study.…”

On Designing Expressive Robot Behavior: The Effect of Affective Cues on Interaction

“…Moreover, we are considering to ameliorate the affective expressivity of speech and facial expressions to make the generated multimodal robot behavior more persuasive and natural. Besides, we are considering to integrate language models that can help the robot to understand human language with a wider scope instead of parsing keywords as with the employed system in the paper [10,11].…”

“…"Affective Speech Synthesis"). 11 These studies discuss the characteristics of body behavior in different emotions employing arm gestures. McColl and Nejat [55] used the gesture hanging arms to express the sadness emotion using the robot Brian-2, while Wallbott [89] used the gesture crossed in front of chest to describe the disgust emotion.…”