Foundations of the Socio-Physical Model of Activities (SOMA) for Autonomous Robotic Agents1

Abstract

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave many pieces of information necessary for performing the task unspecified. Humans can solve such problems fast as we reduce the search space by recourse to prior knowledge such as a connected collection of plans that describe how certain goals can be achieved at various levels of abstraction. Rather than enumerating fine-grained physical contexts SOMA sets out to include socially constructed knowledge about the functions of actions to achieve a variety of goals or the roles objects can play in a given situation. As the human cognition system is capable of generalizing experiences into abstract knowledge pieces applicable to novel situations, we argue that both physical and social context need be modeled to tackle these challenges in a general manner. The central contribution of this work, therefore, lies in a comprehensive model connecting physical and social entities, that enables flexibility of executions by the robotic agents via symbolic reasoning with the model. This is, by and large, facilitated by the link between the physical and social context in SOMA where relationships are established between occurrences and generalizations of them, which has been demonstrated in several use cases in the domain of everyday activites that validate SOMA.

Affordances and their ontological core

“…Şahin et al (2007) argue for the relevance of the observer perspective to affordance-based robot control. Although not being referred to in their argument, examples to illustrate the observer perspective may include Ortmann & Kuhn's (2010 extension of their ontology of observation to Turvey's (1992) dispositional account of affordances and its application to what Ortmann et al (2014) call an "egocentric semantic reference system for affordances"; as well as Beßler et al's (2020) definition of affordances as "descriptions of dispositions" (rather than as dispositions themselves) in ameliorating autonomous robot control (Beßler et al, 2021).…”

“…tools) can be distinguished and identified. Against this theoretical background, the notion of affordance has been investigated in a number of different agency-related domains: for example, it has been extensively utilized in robotics to foster flexible planning and behaviors of robots (Şahin et al, 2007;Chemero & Turvey, 2007;Min et al, 2016;Yamanobe et al, 2017;Beßler et al, 2020Beßler et al, , 2021.…”

“…In fact, a dispositional view of affordances has been recently philosophically defended by Vetter (2018) and Heras-Escribano (2019) (albeit not necessarily exactly in the same way as by Turvey). It has been also formally explored by Ortmann & Kuhn (2010 in compliance with the upper ontology the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE; Masolo et al, 2003;Borgo & Masolo, 2010;Borgo et al, 2022) and, in this DOLCE-based direction of research, it has been reinterpreted and implemented in robotics (Beßler et al, 2020(Beßler et al, , 2021.…”

Deep understanding of everyday activity commands for household robots

“…The direction is thus prototypical of everyday activities and the directions DLU handles. In summary, this work builds upon the existing SOMA ontology [6], the semantic parser SCG [9,10], and the human computation game Kitchen Clash [34]. In this work, a pipeline is build around these three modules to parse directions into semantic specifications.…”

“…The solution proposed in this work is the Deep Language Understanding (DLU) processing pipeline. This pipeline employs the ontological Socio-physical Model of Activities (SOMA) [6], which serves not only to define the interfaces of the multi-component pipeline but also to connect numeric data and simulations with symbolic reasoning processes.…”