Perceptual and Semantic Processing in Cognitive Robots

Abstract

The challenge in human–robot interaction is to build an agent that can act upon human implicit statements, where the agent is instructed to execute tasks without explicit utterance. Understanding what to do under such scenarios requires the agent to have the capability to process object grounding and affordance learning from acquired knowledge. Affordance has been the driving force for agents to construct relationships between objects, their effects, and actions, whereas grounding is effective in the understanding of spatial maps of objects present in the environment. The main contribution of this paper is to propose a methodology for the extension of object affordance and grounding, the Bloom-based cognitive cycle, and the formulation of perceptual semantics for the context-based human–robot interaction. In this study, we implemented YOLOv3 to formulate visual perception and LSTM to identify the level of the cognitive cycle, as cognitive processes synchronized in the cognitive cycle. In addition, we used semantic networks and conceptual graphs as a method to represent knowledge in various dimensions related to the cognitive cycle. The visual perception showed average precision of 0.78, an average recall of 0.87, and an average F1 score of 0.80, indicating an improvement in the generation of semantic networks and conceptual graphs. The similarity index used for the lingual and visual association showed promising results and improves the overall experience of human–robot interaction.

Artificial Subjectivity: Personal Semantic Memory Model for Cognitive Agents

“…This content space was used by the second phase for personal semantic formulation. Semantic Memory is long-term memory that holds the general knowledge that plays a key role in the perceptual semantic phase [10,31]. Here the semantic knowledge helped in analyzing the positivity and negativity of action performed for example if the agent receives external information "John helps Mark", how the agent can evaluate that some positive action has been performed by the John?…”