A Brain-Inspired Multi-Modal Perceptual System for Social Robots: An Experimental Realization

Al-Qaderi, Mohammad K.; Rad, A.B.

doi:10.1109/access.2018.2851841

Cited by 11 publications

(2 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since intelligent machines, including SARs, are becoming more prominent, it is essential for them to maintain long-term, meaningful and engaging interactions with humans [8], whereas state-of-the-art SARs face serious challenges regarding this [5,4,7,3]. Particularly, rudimentary social skills displayed by SARs in interaction with humans negatively impact the human engagement, and thus the effectiveness of SARs [7,4,5,9]. Although nonverbal cues, e.g., joint attention (i.e., drawing the attention of others to an object or person by looking or pointing at it) [4], eye contact [5], and facial expressions [3], have been implemented for SARs, these robots often fail to recognize the best time to display these cues in social interactions, due to a lack of (deep) understanding of the human cognition [7,10].…”

Section: Introductionmentioning

confidence: 99%

“…The key to successful social interactions by humans, based on ToM [11], is to build cognitive models of each other and to interact based on these models. In order to interact as humanly as possible, SARs should exhibit similar understanding of rational agents [8,7,10,12,9]. Thus, we focus on developing dynamic mathematical models for perception, cognition, and decision-making of humans that can be used by intelligent machines in human-machine interactions to estimate the mental states of humans and to predict their behavior.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Mathematical Models of Theory of Mind for Socially Assistive Robots

Patrício,

Jamshidnejad

2023

IEEE Access

View full text Add to dashboard Cite

Interactive machines should establish and maintain meaningful social interactions with humans. Thus, they need to understand and predict the mental states and actions of humans. Based on Theory of Mind (ToM), in order to understand and interact with each other, humans develop cognitive models of one another. Our main goal is to provide a mathematical framework based on ToM to improve the understanding of interactive machines regarding the perception, cognition, and decision-making of humans. Most stateof-the-art models of behavioral theories based on machine learning are focused on input-output black-box representations. Thus, they lack transparency and generalizability, and exhaustive training procedures are needed to personalize them for various humans. Moreover, these models lack dynamics, i.e., they do not mathematically describe the evolution of the mental states and actions of humans in time. Following a systems-and-control-theoretic point-of-view, we represent for the first time the perception, cognition, and decision-making of humans via a dynamic, mathematical framework by introducing a novel formalization and an extension to Fuzzy Cognitive Maps (FCMs). The resulting models are given in a general state-space representation, which can be used by interactive machines within known model-based state estimation and control methods. In a case study, the resulting models were identified and validated for 21 participants, in scenarios where predicting the intentions and behavior of the participants required understanding the dynamics of their mental procedures. The results of these experiments show that our model is capable of incorporating the dynamics to estimate the intentions and predict the behavior of the participants, with an accuracy of, respectively, 81.55% and 66.06%. Moreover, we compared our model with a state-of-the-art formalization of human cognition, which was made dynamic using our introduced FCM framework. Our model, which in addition to the elements of the state-of-the-art model included emotions, personality traits, and biases (thus providing a more transparent insight about the mental procedures of the participants) showed 6.25% and 2.45% more accuracy in, respectively, estimating the intentions and predicting the behavior of the participants.INDEX TERMS Dynamic mathematical models, long-term interactions of rational agents, state-space models of cognition, theory of mind.

show abstract